Missing snapshots when using cron

Heyo!
First of all, I'd like to say thank you for your work and bringing us LuckyBackup! It's been awesome so far :)

Right, I'm using UnRaid as my server OS, and I have LB installed on our main server.
I've got another, offsite backup server, at a remote (duh) place, where LB is making automatic backups.

For whatever reason, the scheduled task's isn't working inside of LB. I've spoken with the guy who made the docker image of LB for UnRaid, and he's told me he's trying to fix the issue, but so far hasn't been able to.

Worry not, as I've set up a custom script within UnRaid, that runs the backup profile automatically every day.

It seems to be working fine, but one fairly critical feature is missing: Snapshots!
When I do a manual backup, within LB, the snapshots are created just fine. But when the script runs the backup profile, I'm not seeing the snapshots through the GUI atleast.

I've read about some permission issues from these forums earlier, and I tried out different user names, just to be sure. And it seems that with any other username, I get an error when the backup is ran. Only with one username (called luckyBackup) it actually works. The error is about missing directories, which makes me believe that atleast the user is correct, and the backup profile is being found correctly.

I'm currently on my phone writing this, so unfortunately I don't have any version numbers right at hand, but I'll give you any needed information once I get back to the office ! :)

Any help would be appreciated, thank you!

Hello Reiiou,
For a start, thank you for the kind words :)

I guess that from within the gui (task->manage backup), you can only see the very last backup that run via cron and no previous snapshots. Correct or do you see the last backup run manually?
Can you please also have a look at the destination directory of any task that uses many snapshots. There should be a (hidden folder) named ".luckybackup-snaphots/", yes this is a typo, I know!
Inside this, you will find directory "<name_of_profile>.profile". Does this contain many logfiles inside "logs" and many snapshot files inside "snaps"?
Also, inside the ".luckybackup-snaphots/", if there was some change at a source file (modified or deleted), you should find folders named as "YYYYMMDDhhmmss". Is that the case?</name_of_profile>

Hey,
Sorry for the late response! Still on phone, will get you more info tomorrow! :)

From the "manage backups" I only see the backups from the manual runs.
All of my task's use multiple snapshots.

I'll go through the folders and logs tomorrow!

ps. oh, and it's "rellu", just stylized (=

Hey, got my arse to the office now.

I'll attach a screenshot of the default.profile/snaps folder (what a original profile name, I know!)

It does indeed look like there is multiple (way more than there should be?) backup snapshots in there.
For this particular task, I've set the snapshots to be 4.

It also does look like the automatic cron-jobs have created backups. Manual backups are done usually in the evening, and the cron ones are done at 00.00 (See the timestamp at the right)

As for the second question, yeah, there are folders in the .luckybackup-snaphots folder. I'll attach a screenshot of that as well.

Cheers!

So, it looks like the snapshot backups are performed normally, either when manual or scheduled.
But the reference at the gui is missing which is quite of annoying, especially when the time comes for a restore of an older snapshot. This is not normal LB behavior, all snapshots should be visible via the "manage backup" dialog.
Just to make sure, could you please,for a start, double check that the gui only displays the manual performed snapshots.

Hey,

Exactly. Restoring a backup through GUI would just be so much nicer!
And apologies for not posting a screenshot of the "manage backups" menu! (I'm in the software business myself, so should've known better hah!)

I'll post one as an attachment now. Again, look at the timestamps.

Cheers

I think I found the culprit!
There is file that is not updated for some reason when the profile runs via cron.
One of the reasons this can happen is that the main LB gui is already open and it has control of that file.
I really hope this the case, so if you could just confirm it (or not)...
By the way, the file is the normal ".profile" text file, located at ~/.luckyBackup/profiles/ that contains all the needed profile info.

I'm not too sure how to confirm this :/
I've tried to "quit" the GUI of LB (with the "quit" button) and run the script, but the result seems to be the same (but from what I can see, it kind of just restarts the GUI after the quit is pressed)

I checked the security details of the profile file within Windows, and it shows two users there, called "nobody (Unix User/nobody)" and "users (Unix Group/users)" that have permissions to read and execute, read, write, and special permissions.

Could this have anything to do with the fact that LB is running inside of a Docker within UnRaid?

EDIT: No idea if this is of any use, or if this is for a completely different issue, but I spotted this message within the log that LuckyBackup is sending to UnRaid (see the attachment)

Please excuse my late reply rellu :(

I'm not too sure how to confirm this :/

Just make sure that the LB main window is not open while a schedule is running
If the gui is closed and the result is the same then there's something else to blame.

In case the above is confirmed, all the info you are providing will hopefully help to figure out what's causing the issue. Though I'm not familiar with docker or UnRaid.

Hey,
don't worry at all about the slow response! I'm getting better customer service here than some paid services, so you're good in my books! :)

The computer is always shut off when the timed cron job is done. So the GUI should be closed, but! With that said, I'm not too sure if UnRaid actually keeps the GUI open while the docker is running.. 🤔

Usually when I've tried to manually run the script, I've had the GUI on a browser tab, open. I can try closing said tab next monday when I get to the office again, but the result should be the same, as, like I said, the timed cron runs when the computer (that's being used to access the GUI) is shut off.

I've been in talks with the guy over at UnRaid forums, who's made the docker image of LB, I could ask him how the GUI is handled, and if there's a way to properly close it.

Cheers, have a good weekend! :)

EDIT: Ok, so the answer is:
The GUI is never shut down, as the container requires it :/

@luckyb69 Hi, I'm the creator of the luckyBackup Docker container for Unraid. :)
First of all let me say thank you for creating such a superb frontend for rsync, I use it for backing up my whole server over SSH to another server and everything simply works.

The problem that I have is that cron doesn't works properly or at least you can get it to work but not reliably in a Docker container and the next thing that @REllU has already mentioned is that I simply can't end the main process, wich is luckyBackup since otherwise the container will stop.

I also looked into various other time based solutions for Docker that are similar to cron but these are all not working with the real cron that luckyBackup uses.
Have to say that I used DirSyncPro befor I used luckyBackup and DirSyncPro had a schedule engine built into the application itself but DirSyncPro was based on Java and really memory hungry if you have big sync jobs like Server to Server backup.

So from a techincal standpoint it's not possible to run a cron job and also see how it progressed or if it's finished in luckyBackup iself?

I also thought about adding a logreader that watches for logs and outputs the results if a new log file was found.

Hello Christoph,
well, the appreciation is mutual. Thank you as well for maintaining LB as a docker container :)

Now everything makes sense, which is good news.
The bad news is that I don't see the scheduling feature working normally haha

...so as far as I've understood from your comment as well as Rellu's input:

1. The cron job might run (and it has, multiple times) but it's not to be trusted.
2. Even if the cron job or any other scheduling solution succeeds, the backup will be performed normally, snapshots, if chosen, will be created but the gui (task->manage backup) will not reflect the new run. This is because the non-stop running instance of LB will not let the scheduled instance update a file (~/.luckyBackup/profiles/PROFILENAME.profile).

I could fix No.2 at some newer version but unfortunately I have stopped the development of LB quite some time now and I cannot foresee when or if I will start again.
As for No.1, LB could run via any other scheduling solution, it's just not very straightforward. The user will have to manually input the LB command (copy from " schedule->view current crontab") to the scheduling tool.

I understand that this doesn't look good but any workaround brainstorming is more than welcome :)

Thank you for the detailed explanation. :)

Exactly cron is not working reliably in Docker since it's also not persistent and I also tried to work around that "problem" but nothing is working so far and I have real trouble to get it to work.
I gave @REllU the workaround that he can create a cron job on the host with the method you described.

Just a little side note, I think luckyBackup popular in the Unraid community because it has at time of writing 77.183 pulls (I know Docker Hub only tells 50K+ but this is the exact number): https://hub.docker.com/r/ich777/luckybackup

If you start on updating/maintaining luckyBackup again I think very much users will appreciate that. :)
Also if you release new versions I will change the way that the Docker Container works now and use also another scheduling engine because now the container doesn't have luckyBackup built in.
I build it first on my build server so that it fits the needs how the Container works and then it downloads it on the first Container start and checks on every Container start/restart if a newer version is available.

If you don't plan to maintain or further develop luckyBackup I will add a note how to add a cron job from the host so that schedules work.

It's really a excellent frontend for rsync and I really love it, there is nothing similar out there I think. :)

Many greetings from Austria,
Christoph

You are correct Christoph!
It's a real surprise, after so many years of inactivity, for LB to be still preferred by users as a backup solution. Maybe there is a software gap in this field. I still use it as well :D
I just wish I had the time to go no with this project.
Many greetings to Austria from Luxembourg :)

Sorry for the late response but my job is currently exhausting my a bit
Yes, the number of downloads is getting higher and higher (79.603 as time of writing), I also don't think the number is only dedicated to unRAID users, since everyone can download from Docker Hub.
If I can make three feature requests, if you got somehow time somehow it would be:

1. Checkbox for Automatic start of a sync job after the program is started, if possible with a delay of x seconds
2. Run a terminal command when the job is finished
3. Checkbox for Automatic shutdown from luckyBackup when the job is finished, eventually with a delay of x seconds

A little bit more explanation on this:
1.: Users could run a cron schedule on the host to start the container and then the container automatically starts to sync after x seconds
2.: Let the user know via Pushover/Gotify that the sync is finished (since Push services getting more and more popular than Email nowadays)
3.: If luckyBackup ends itself the container would be also shutdown automatically (actually this could be done with 2. too but I think it would be more convenient to have a checkbox for this)

I think that would help users a lot especially for the Container version.

Also I completely understand that time is rare these days... See this requests as wishes from my side... :D

You can also look at the source code for my container here: https://github.com/ich777/docker-luckybackup
And I push the custom builds for the container from luckyBackup to this repository at the Releases: https://github.com/ich777/luckyBackup

As said I would switch how the container is currently created to include the binary into the container itself and not download it on start if you have somehow time for it to modify it a bit (I actually know it is really hard to get back to a project that you haven't touched for a long time).

Best,
Christoph

Hello Christoph,
well, good news. 2 out of the 3 feature requests are kinda already there !!

1. Run a terminal command when the job is finished

Have a look at the "also execute" commands. You can run anything before a task starts or when it's finished. You can even mount disks, check stuff or even add delays with the sleep command. If a command does not work directly at the LB gui field, put it inside a script and use the script name instead :)

1. Checkbox for Automatic shutdown from luckyBackup when the job is finished, eventually with a delay of x seconds

During a run, there is a shutdown button available. If pressed, the system will shutdown when all tasks finish running. See here, just before the "all done" paragraph.

Have a look at the "also execute" commands. You can run anything before a task starts or when it's finished.

Sorry completely forgot about that feature... so one "request" solved...

shutdown

Actually a shutdown won't work in the container because a container can't shutdown the host, a exit from the application will do the job just fine...
But that will be also solvable with something like:

kill $(pidof luckybackup)

with the "also execute" commands on finish.

But my thought was to make it a little more user firendly... :)

Hey,

Just spitballing here.. Would it be somehow possible to add in the paths to the cron-made backups?
Since the backup itself is clearly working as it should, and right now it looks like we've narrowed it down to permission issue, where the file that has the paths to the backup snapshots cannot be edited since LuckyBackup GUI is on.

I'm explaining it terribly, let me try again..

Since the snapshot-files seem to be there, wouldn't we e just need to point these files to LuckyBackup, correct?

Something like..

1. A timed cron job is started.
2. LuckyBackup container is started
3. LuckyBackup runs a profile
4. LB is shut down
5. The cron script "manually" edits the file that points out where the snapshot files are

LB is shut down

This won't work because the container will be stopped if luckyBackup is stopped.

The cron script "manually" edits the file that points out where the snapshot files are

This is a ruff workaround...

@Loukas I have a crazy idea would it be possible to write the schedules to a file instead to cron?
I could implement a function and add "at" to the container so that the schedule works and every start/restart from the container is checked if every "at" entry from the file is present or if you can write it directly to a file I can also check if the cron jobs are actually in the cron table then it will also work.

@Loukas Looks like I've got cron running in the container just fine regarding to this post: Click
Looks like you have to activate console mode so that it is executed properly.