Rerunning a backup downloads unmodified images again

[Solliko]

Hello Tommaso,

Rerunning a backup re-downloads unmodified images and overwrites identical files already on disk.

Have been using this wonderful app for years and this has never happened before. To verify I downloaded a fresh copy of my smugmug account (about 400 GB) which completed successfully in 2 hours. Then I ran it again and it proceeded to download files again.

Using version 1.5.0 from Aug 2023 on MacOS 14.6.
Here is my config.toml file:

[store]
destination = "/Volumes/Photos Backup/Album"
file_names = "{{.FileName}}"
use_metadata_times = true
force_metadata_times = true
concurrent_downloads = 10
concurrent_albums = 10

[tommyblue]

@Solliko does it download everything every time or just some photos?

I noticed that on my account recently, it kept downloading a bunch of photos. After some investigation I found out that those photos were duplicated in their folders (mostly they were photos I rotated with Preview) and they had different sizes. Since the APIs return albums' photos in random order, every time the order changed then the size check failed and they were downloaded again. I removed the duplicated photos and everything is clean now.

Can you check if you're keeping downloading the photos with the same name?

[Solliko]

I just ran it again and it's currently downloading from multiple folders. Some folders were created last month and had files moved to them, some folders were created 2 years ago and have not changed since.

This is on a new backup created today.

[Solliko]

I have let it run for 20 minutes now and it appears to be re-downloading everything (60GB so far). I see it downloading unmodified folders from 2019.

Also, I tried to re-open issue #15. I left you a new message there. Thanks!

[tommyblue]

2 questions: which version are you using? Can you paste some logs so I can try to understand what's going on?

[Solliko]

Using version 1.5.0 from Aug 2023 on MacOS 14.6.
How do I create logs?

[tommyblue]

Using version 1.5.0 from Aug 2023 on MacOS 14.6. How do I create logs?

Ok, the last version is 1.5.5 but nothing should change about what you're facing now. Logs are printed to console while running the program. How do you run it? If you run it from terminal you see them

[HenrikHoyer]

FYI: I am encountering a similar problem. Some of the files are downloaded again and again and...

It looks like the SmugMug API is returning a wrong size for the file. Example

This file:
"https://photos.smugmug.com/photos/i-nGHxx5p/0/L4KN6bNCnW2tmNZdQxS2ZW3qCgkfKDhqC9TKs5TKL/D/i-nGHxx5p-D.jpg"

is reported with a ArchivedSize of 361292 bytes

But the file returned from SmugMug is only 346846 bytes: