Activity for Isync

Oswald Buddenhagen posted a comment on ticket #15

Oswald Buddenhagen — Sun, 18 Oct 2026 18:19:01 -0000

oh, that's good. however, the request as such is still valid, only with a very low priority.

Mark Diekhans posted a comment on ticket #15

Mark Diekhans — Sun, 18 Oct 2026 17:54:45 -0000

It appear that gmail now correctly handles custom keywords, so I withdraw this request as unnecessary

Oswald Buddenhagen modified ticket #91

Oswald Buddenhagen — Fri, 25 Sep 2026 14:49:14 -0000

mbsync: "Warning: SASL wants more steps despite successful IMAP authentication. Ignoring..."

Space created ticket #91

Space — Fri, 25 Sep 2026 10:52:08 -0000

mbsync: "Warning: SASL wants more steps despite successful IMAP authentication. Ignoring..."

Stephan Gerth posted a comment on ticket #90

Stephan Gerth — Thu, 17 Sep 2026 18:21:25 -0000

FWIW: Source server is Dovecot 2.3.13 (Debian 11), target server is Dovecot 2.4.1-4 (Debian 13).

Oswald Buddenhagen modified ticket #90

Oswald Buddenhagen — Thu, 17 Sep 2026 17:53:52 -0000

Connection loss for IMAP2IMAP sync on large mailboxes

Oswald Buddenhagen posted a comment on ticket #90

Oswald Buddenhagen — Thu, 17 Sep 2026 17:53:32 -0000

ok, thanks, something is obviously going wrong. first it queues 22 fetches (which is incompatible with a pipeline depth of 1), then it queries the buffer size for all remaining messages (which is zero, because no appends have been scheduled), then it queues all remaining fetches. then the first fetches are completing, upon which it correctly schedules the first append, and the next one only once the first one completes (this is how pipelinedepth 1 should work). one notable observation is that the...

Stephan Gerth posted a comment on ticket #90

Stephan Gerth — Thu, 17 Sep 2026 15:00:39 -0000

I've sent you a log of a -Ddn run with BufferLimit 1, PipelineDepth 1 on the target and FSync no to the maintainer address.

Oswald Buddenhagen posted a comment on ticket #90

Oswald Buddenhagen — Wed, 16 Sep 2026 21:17:47 -0000

the only legitimate way how that can happen is that the IMAP commands were actually all already sent out, and the queuing is happening on the target server. -Ddn should confirm that. the way to work around that would be limiting PipelineDepth to something reasonable, say 50.

Stephan Gerth posted a comment on ticket #90

Stephan Gerth — Wed, 16 Sep 2026 18:54:14 -0000

I think the disk setup is not the actual problem since the network connection could always be the main bottleneck (imagine syncing over a 50MBit internet connection). But I think I figured out why the BufferLimit has no effect (lines refer latest git 42cfa1): Line sync.c:1468 checks via get_memory_usage if BufferLimit has been already reached or overrun and handles that. However, according to the logs all calls to get_memory_usage returned no usage at all: N: Called get_memory_usage, ret=0 This explains...

Oswald Buddenhagen posted a comment on ticket #90

Oswald Buddenhagen — Wed, 16 Sep 2026 12:36:21 -0000

the fs load result is surprising. what are the actual numbers? network bw, disk bw, fs type & mount options, etc. that effect of bufferlimit sounds just wrong. please mail me the log.

Stephan Gerth posted a comment on ticket #90

Stephan Gerth — Wed, 16 Sep 2026 07:55:11 -0000

I had a look at iotop and I could see that the write rate was 1,5x compared the Fsync yes setting. So it seems to make a difference but does not increase write rate enough. Setting BufferLimit 1 pushes message storage further to the end in the log file (-Dd) but it still fetches all messages so that the connection to the source server runs into the timeout (while it is working on storing all mails into the target mailbox folder (about 120k lines only enter/leave store_msg lines).

Oswald Buddenhagen posted a comment on ticket #90

Oswald Buddenhagen — Tue, 15 Sep 2026 19:07:03 -0000

Setting FSYNC no did not make any difference. weird. have a look at iotop on the target server with the two settings. also check the mount options of the volume. it's kinda unexpected that the process is disk-bound to such a degree. Would it make sense to set it to some smaller value than the default value so that it cannot fetch the whole mailbox folder [...]? yes. you could actually use something ridiculously small (even 1) to avoid that multiple STOREs are queued up at all.

Stephan Gerth posted a comment on ticket #90

Stephan Gerth — Tue, 15 Sep 2026 17:40:25 -0000

I've tried with the -Dd option: The output makes sense so far: A lot of interleaved messages of such as this snippet (within the large mailbox folder already): F: [ 120772] Callback enter fetch_msg, sts=0, flags=, date=1789297202, size=110049 N: [ 228630] Enter store_msg, flags=, date=1789297202, size=110071, to_trash=no N: [ 228630] Leave store_msg F: [ 120772] Callback leave fetch_msg F: [ 120773] Callback enter fetch_msg, sts=0, flags=, date=1789279202, size=122022 N: [ 228631] Enter store_msg,...

Oswald Buddenhagen modified a comment on ticket #90

Oswald Buddenhagen — Tue, 15 Sep 2026 09:30:30 -0000

there is no such mechanism; i just didn't anticipate this situation. assuming you are using the default BufferLimit, the timeout cannot possibly occur while messages are being transferred. (*) so it must be during message enumeration. but that should be also on the order of 10 MB with your mailbox size, so i dunno. have you tried the -Dd log? if you can't make sense of it, you can mail it to me privately. EDIT: (*) well, actually, if the target server is using a legacy fs that does fsync inefficiently,...

Oswald Buddenhagen posted a comment on ticket #90

Oswald Buddenhagen — Tue, 15 Sep 2026 08:50:11 -0000

there is no such mechanism, i just didn't anticipate this situation. assuming you are using the default BufferLimit, the timeout cannot possibly occur while messages are being transferred. so it must be during message enumeration. but that should be also on the order of 10 MB with your mailbox size, so i dunno. have you tried the -Dd log? if you can't make sense of it, you can mail it to me privately.

Stephan Gerth posted a comment on ticket #90

Stephan Gerth — Tue, 15 Sep 2026 06:44:26 -0000

Regarding the working of isync this was just guessing from what I've observed, sorry! The mailbox folder in question has about 108k emails, so that's far from millions of messages. I've tried to speed-up the target server by putting the target mailbox on a ramdisk. This lets the synchronization finish before the timeout hits which solves it in my case. However, for larger mailboxes or slower servers the issue would surely occur again. How is the general mechanism working when target server takes...

Oswald Buddenhagen posted a comment on ticket #90

Oswald Buddenhagen — Mon, 14 Sep 2026 18:26:22 -0000

that's just not how isync works. you can watch it by adding -Dd. -Dn (or just -D) will produce even more output, but based on the info so far i expect that to be somewhat useless due to the sheer volume of repetetive content. how large is "quite large"? at normal speeds, such a duration would translate to millions of messages.

Stephan Gerth posted a comment on ticket #90

Stephan Gerth — Mon, 14 Sep 2026 17:35:42 -0000

No, this happens somewhere in the middle (on the 43rd folder of 48 to be specific which happens to be quite large). I watched some numbers this time while running the synchronization: fetching from the source seems to be faster than pushing to the target mbsync seems to cache the whole(?) mailbox folder (memory usage went to max. 10G) when it tries to push the remaining mails to the target while the source mails have been already cached, the source connection is not seeing any activity During the...

Oswald Buddenhagen posted a comment on ticket #90

Oswald Buddenhagen — Mon, 14 Sep 2026 09:55:38 -0000

at which point does that happen? while one side is still FETCHing the message list? for half an hour?!

Stephan Gerth created ticket #90

Stephan Gerth — Mon, 14 Sep 2026 05:33:26 -0000

Connection loss for IMAP2IMAP sync on large mailboxes

Oswald Buddenhagen committed [42cfa1] on isync

Oswald Buddenhagen — Tue, 01 Sep 2026 07:48:23 -0000

limit length of credentials supplied to LOGIN

Oswald Buddenhagen modified ticket #89

Oswald Buddenhagen — Tue, 11 Aug 2026 09:23:55 -0000

Fails to build against OpenSSL4

Oswald Buddenhagen updated merge request #10

Oswald Buddenhagen — Tue, 11 Aug 2026 09:23:03 -0000

Fix for OpenSSL4 ompatibility

Oswald Buddenhagen committed [aebbf2] on isync

Oswald Buddenhagen — Tue, 11 Aug 2026 09:22:43 -0000

mention that --dry-run isn't quite read-only

Oswald Buddenhagen committed [bdd9ff] on isync

Oswald Buddenhagen — Tue, 11 Aug 2026 09:22:43 -0000

fix build with openssl 4

Finn Gärtner posted a comment on merge request #10

Finn Gärtner — Tue, 11 Aug 2026 08:05:36 -0000

Here is the relevant source code bit at the OpenSSL_1_1_0 tag: https://github.com/openssl/openssl/blob/abd30777cc72029e8a44e4b67201cae8ed3d19c1/crypto/asn1/asn1_lib.c#L366

Oswald Buddenhagen posted a comment on merge request #10

Oswald Buddenhagen — Mon, 10 Aug 2026 21:19:52 -0000

thanks, this looks reasonable. one thing: i'd like to see a somewhat definitive statement (with source) that the function is actually available since 1.1.

Finn Gärtner created merge request #10 on isync

Finn Gärtner — Mon, 10 Aug 2026 19:49:38 -0000

Fix for OpenSSL4 ompatibility

Finn Gärtner created ticket #89

Finn Gärtner — Mon, 10 Aug 2026 19:20:41 -0000

Fails to build against OpenSSL4

Oswald Buddenhagen committed [7cbbc6] on isync