DT[TBL, a:=i.b] update join with 'i.' prefix fails when 'i' is a tibble

[MichaelChirico]

library(data.table)
library(tibble)

DT1 = data.table(a=1, b=2)
DT2 = data.table(a=1, c=3)
DF = data.frame(a=1, d=4)
TBL = tibble(a=1, e=5)

# fine
DT1[DT2, on='a', c := i.c]

# also fine
DT1[DF, on='a', d := i.d]

# error
DT1[TBL, on='a', e := i.e]
# Error in eval(jsub, SDenv, parent.frame()) : object 'i.e' not found

DT1
#        a     b     c     d
#    <num> <num> <num> <num>
# 1:     1     2     3     4

This is inconsistent -- either we should require i to be a data.table (so DT1[DF, ...] would fail), or we should treat all is.data.frame(i) && !is.data.table(i) cases the same.

[AngelFelizR]

we should require i to be a data.table

[trobx]

I think that would be a retrograde step, and wouldn't it break a lot of code? We'd lose the ability to do ad hoc joins on the fly via lists, for example.

iiuc i just needs "read-only access" to the underlying list of vectors, so taking a shallow copy of it with unclass() works:

# fine
DT1[unclass(TBL), on='a', e := i.e][]        # or lapply(TBL, \(x) x)
#        a     b     c     d     e
#    <num> <num> <num> <num> <num>
# 1:     1     2     3     4     5

address(unclass(TBL)$a)==address(TBL$a) # TRUE

Obviously this is just a workaround (internals are beyond me). @MichaelChirico, any idea why the problem occurs?

[EDIT: Actually there's no mention of disallowing lists, is there - sorry. But I still think it would break things to disallow plain data.frames, tibbles, sf etc. in i, and the more interoperability there is, the better.]

[MichaelChirico]

I agree we should not change the behavior -- not just for back-compatibility or convenience, but because it better fits the data.table mental model: a list in i is used to "look up" rows in x. The S3 class of that list shouldn't particularly matter, as long as it's unambiguous which rows are being "searched".

[MichaelChirico]

Our first hint at the issue: this branch. Could explain why tbl doesn't work, but I don't yet understand why data.frame does:

data.table/R/data.table.R

Lines 680 to 683 in 5f4258e

	if (is.data.table(i)) {
	idotprefix = paste0("i.", names_i)
	xdotprefix = paste0("x.", names_x)
	} else idotprefix = xdotprefix = character(0L)

[MichaelChirico]

OK, here's part II: we apply as.data.table(i) if i is a "pure" data.frame:

data.table/R/data.table.R

Line 442 in 5f4258e

else if (identical(class(i),"data.frame")) { i = as.data.table(i) } # TO DO: avoid these as.data.table() and use a flag instead

Now, I'm not sure what breaks if we change either of these two steps.