nsqd: ephemeral channels don't work with --mem-queue-size=0

[herry-go]

When mem-queue-size= 0 is set, the message is sent to a temporary channel (#ephemeral), and it is found that the message can never be consumed. Is it because the mem-queue-size is exceeded and the message is directly discarded?

[ploxiln]

That's correct.

#ephemeral means there is no disk-queue, only mem-queue. mem-queue-size=0 means there is no mem-queue. So, there is no queue :)

[herry-go]

However, I found that I did not have this problem with version 1.2.0, and the messages of 1.2.1 temporary channel could not be consumed. Whether this is a vulnerability is the same as the delayed message problem?（ #1375 ）

[herry-go]

Thank you very much for your careful answer， I have a few questions：
1.When mem-queue-size=0, set memorymsgchan to nil. Is this reasonable for temporary channels?
2.At the same time, I found that even if mem-queue-size=0 is set, he temporary channel message is not lost, but it is persisted to the disk， because I found xxxx.diskqueue.000000.dat files and xxxx.diskqueue.meta.dat files
3.I'm in protocol_ v2.go In the messagepump function of the go file, "case B: = < - backendmsgchan:", why does it not work

[ploxiln]

You may see disk queue files because the topic is not ephemeral, but the channel being ephemeral would still have this problem with mem-queue-size=0. The topic and each channel has its own set of memory and disk queues. Usually, if there are any channels, the topic queues do not accumulate any messages.

Indeed, this is related to the change in 1.2.1 that completely disabled the mem-queue channel for mem-queue-size=0 (instead of leaving it a zero-buffer channel). Previously, it was possible for a single message to jump through the topic to the channels, and then through the channels to a consumer, if all are ready and poised to receive a message. But now all messages are forced through the "backend" disk queue, in each stage.

So in your case the message is forced through the disk-queue for the topic, and then again it's forced through the disk-queue for the channel, but for an ephemeral channel that doesn't exist.

mem-queue-size=0 is not recommended, it doesn't work well in general.

[herry-go]

Will this problem be fixed in Version 1.2.1?

[ploxiln]

eh, well, the issue was introduced in nsq v1.2.1 ... will it be "fixed" in v1.2.2 or v1.3.0? I don't know, but I have an unapproved pull-request which attempts to fix it, which you could try: #1376 (or just go back to nsq v1.2.0)

[herry-go]

@mreiferson Sorry for bothering but is there any progress?
Our team has some resources to help fix this problem, that is, you can simply point out the potential problem location.
We want to help.

[ploxiln]

You can try using #1376 and leave some review about how it behaves for you.

[herry-go]

Well, thank you very much.