iCloud Photos Downloader

HN disclosure: I’m the author of Photos Backup Anywhere, but this thread mirrors the exact issues that pushed me to write it.

One thing that surprised me when digging into Apple Photos is how much state isn’t represented by just files-on-disk. Albums, Live Photos (paired assets), bursts, slo-mo, edits, and even “simple” things like adjusted capture dates are all tracked separately, and most export/backup tools end up flattening or partially reconstructing that on restore.

The approach I took was to treat Photos as the source of truth and verify restored items against it, rather than assuming filesystem metadata is enough. As far as I know, this is the only tool that restores albums and correctly round-trips all Photos item types while preserving location data, creation dates, and modification dates when restoring back into Photos.

Project page is here if it’s useful: https://photosbackup.app/

Happy to explain details if anyone’s curious — there are a lot of sharp edges in Photos once you go beyond “export originals”.

Nice to see my project on Hacker News! I started this almost 10 years ago and haven't been involved with maintenance for a long time, but I'm glad that people are still finding it useful.

Surprisingly, there is no official way to download all (400 Gb) photos from iCloud. Here is an open-source command-line tool to download all your iCloud photos.

This is awesome! This might be a great replacement to attempting to get the Windows app to work. Has anyone had luck with the iCloud app on windows?

Similar to some other folks in this thread I have ~2TB of iCloud data, a Macbook with far less than 2TB of space, an external hard drive somewhere with the external Photo Library that I need to plug in if I want to look at photos on the Macbook, and a Windows desktop with 10TB+ of rusty disks.

I was excited when they added the iCloud app + iCloud photos to Windows, but it never seems to catch up or finish what it is doing. It appears to be almost constantly download at 50MB/s, stressing both disk & internet, and yet navigating to the folder reveals that they are all 'available when online'.

It seems like there is not an option in Windows to actually grab everything in full quality (actually now that I look at it - its gotten to 944GB on disk / 1.91TB total, so it is getting there.)

I guess a real question - with these photos finally on a Windows desktop - is there a better photo browser than Microsoft photos that can show the HEIC and the Live Photo?

I run this periodically from a little shell script; I "should" automate it, but time is scarce.

  ⟩ cat ~/bin/icloud_download
  #!/bin/bash
  mkdir "$(pwd)"/{photos,cookies} 2> /dev/null
  
  if [[ -z "${ICLOUD_USERNAME}" ]]; then
      echo "need env ICLOUD_USERNAME"
      exit 1
  fi
  if [[ -z "${ICLOUD_PASSWORD}" ]]; then
      echo "need env ICLOUD_PASSWORD"
      exit 1
  fi
  
  podman container run -it --rm --name icloud \
      -v $(pwd)/photos:/data \
      -v $(pwd)/cookies:/cookies \
      -e TZ=America/Boise \
      icloudpd/icloudpd:latest \
      icloudpd --directory /data \
      --cookie-directory /cookies \
      --folder-structure {:%Y/%Y-%m-%d} \
      --username "${ICLOUD_USERNAME}" \
      --password "${ICLOUD_PASSWORD}" \
      --size original

While not free, and not for any other platform than macOS. The program Parachute[1] in the App Store is very nice in downloading both photos from your library as well as files from the various locations.

[1]: https://apps.apple.com/us/app/parachute-backup/id6748614170?...

I was there until I saw this feature:

Automatic de-duplication of photos with the same name

I recently went through a year’s worth of photos from my wife’s phone, and found three distinct “img_0001.jpg”’s just in that single year. Apple’s naming convention is so short sighted that I’d be terrified letting a piece of software try to dedupe it “by name “

I was just thinking about this today. Apples lack of any 3rd party integration for things like this and iMessage is really annoying sometimes. In addition to a secondary backup, I’d love to automatically sync some photos from a certain album to my parents photo frame. Or if I take a nice nature shot have it sync to a Samsung frame tv. I get the benefits of the walled garden but esp w photos and messaging it seems like opening up a little would allow for some innovation

I’ve been using usbmuxd+ifuse to copy the photo files straight from the phone. No need to wait for an upload/download to some remote server, just a direct cable from the phone to my computer. I get the original files, and can even move (instead of copy) to clear up the phone.

From the project describtion: "Looking for MAINTAINER for this project"

Honestly, Apple should officially maintain tools like this. However, for obvious reasons, such as the iCloud subscription revenue model, Apple will not do it. In fact, Apple may even make life harder for such tools.

Thanks for this project. Our family generates about 2TB of media a year, and it’s been like that for a while, so we’re sitting at roughly 12TB total. That’s very much the long tail of personal media.

I’m not ready to pay $60/month, but I do like iCloud’s memories and other photo features. My compromise is simple:

- I use docker-icloudpd to download our iCloud Photos to local storage over time. It’s been the most practical way I’ve found to back up multiple accounts into one place, though it does require occasional re-auth every so often. - I keep only the last ~2 years of media in iCloud and delete older ones after they’re archived locally. - For browsing and searching the older archive, I use Immich, which has been a great self-hosted personal photo cloud experience with a modern app feel.

For storage, I’ve found fast local disk matters a lot once you’re digging up photos from 5+ years ago. Something like an OWC 4M2 with M.2 drives keeps the experience snappy; a typical HDD-based NAS can feel sluggish when you just want to quickly pull up an old memory.

https://github.com/boredazfcuk/docker-icloudpd

My concern with backing up iCloud Photos with anything but Apple Photos is that there are some proprietary formats like Live Photos and slow mo video for which exports are lossy. Also, Apple Photos stores all edits non-destructively, so 'flattening' the edits into a single file for export is also a lossy operation.

It seems like an obvious improvement for Time Machine to support full backups while using optimized storage on the primary system.

It is no fun to have old iCloud photos deleted unexpectedly. Apple has provided plenty of footguns, even if they really are user errors. For examples: (1) during device restores and (2) premium subscription management fumbles.

Product idea: Apple should offer a paid service to restore the "old backups" of photos that are no longer accessible via iCloud UI/API, which were soft-removed for missing the subscription quota or whatever, if Apple happens to have that data tucked away in cold storage somewhere.

Case in point, I had some c. 2016 era photos in iMessages that I thought I handled right to not lose from iCloud, but they are apparently nowhere to be found in iCloud API based on recent checks. More than mildly irritating.

I should have used an iCloud photos backup tool like this much sooner.

Print what you want to keep onto archival paper with archival dyes. Everything else will atrophy.

Wish I’d seen this 3 days ago. Needed to backup our Shared Library and did the following (about 10K photos/videos at 300gb, had enough space so full downloaded to MacBook, not optimize. 1. Repair iPhoto library, (Wait 24 hrs to re-sync to iCloud) - initially looks like it moved all photos to personal. 2. Select small chunks, by year worked well enough - selecting All gave me the spinning ball. Then export unmodified to external hdd into folders organized by year. 3. Moved entire photo library file to another external hdd. 4. Open iPhoto and select external hdd library as primary library, let it re-sync to iCloud (Wait 24 hrs). iPhoto now running off external HDD library and I’ll backup to separate external HDD monthly. Repair function and wired Ethernet connection were biggest game changers to previous attempts.

Parachute Backup is a tool I use: https://parachuteapps.com/parachute

This great. I want to switch to android but the only thing stopping me is eight years of photos. No more.

Does anyone know if there’s a way to self host/impersonate iCloud? I’d like to back my iPhone up locally.

I'm a big fan of Parachute Backup for this purpose. https://parachuteapps.com/parachute

This looks useful. I ran into a very similar problem recently and ended up building a small tool for my own use: https://github.com/cleanexit0/darwin-photos

It's macOS-only and intentionally minimal — the goal is just to download originals from iCloud Photos to disk without syncing everything into Photos.app first.

To be clear on limitations: it doesn't preserve albums or other metadata yet, and it's not meant to replace more full-featured tools. It’s mainly for the "I just want my photos off iCloud for backup" case.

Reading the comments here, it sounds like metadata preservation is a big pain point for many people — I'd be curious whether that's the first thing folks would want added, or if simple bulk export already covers most needs.

All my vertical videos in iCloud show up cropped horizontal for some reason. If I go to edit I see the whole video. I really do not want to trust any cloud provider to maintain my years of archives of family photos and videos. Glad things like this exist. I just need properly date-foldered files, without no duplciates. Is that so hard?

Does anyone have any idea for why Apple makes it so difficult to keep photos downloaded?

For context, try tapping 'optimize photos' in iPhone storage settings and then figure out how to turn off the feature without using Google. Not only is the toggle nearly impossible to find, but it's also hidden from being searchable

Thanks. I cannot get iCloud sync to work at all. It consumes CPU, asks for logins repeatedly, etc but fails to actually do its job. When I think of its bugs and all the issues with the latest iOS (bugs and performance on recent hardware), I am thinking of exiting the Apple ecosystem entirely.

I’ve been using this for several years now on a little unraid box to download new photos nightly. There’s a few docker containers that wrap in support for notifying when 2FA is required etc. Always makes me nervous, the access it has, but I’d rather have my photos backed up somewhere I own.

I pay around 10 euros a month to apple just so I can sync my photos from iphone to mac and ipad. That’s the only reason I need the 2 TB for icloud service. With an app like this I could download and keep copies and get rid of iCloud subscription?

I used to use the Photos desktop app to move my photos (“select from the app and drop into a folder somewhere” worked best) to a separate folder a lot (and regularly) until I started using ente. Now ente not only backs up to their e2ee cloud but its desktop app keeps those media synced to the OS of my choice on my laptop.

But I can still not escape Apple’s gonorrhoeic naming and organisation.

Pro: FOSS of course; it works, with limitations (that’s mostly Apple) and glitches (that’s entirely ente)

Cons: really subpar non-native apps (desktop app is quite a dumb app as well) :( (and barely and useful additional features that lets a user do some batch/organisational changes or so)

I use this to sync my wife's photos to Immich and it works great, however the auth process is a bit of a pain (not the fault of icloudpd) and have to reauth every few months.

Is there no worry in giving the app your password? I would never just give my password, especially my Apple account password to a random app. Is this program reliable? However it looks like a program that I want to use.

Seems to be looking for a new maintainer. If anyone has the bandwidth, it’s a great piece of software

Wow, I will definitely give this a try. I have tens of thousands of photos in iCloud and I literally can’t export them all at once. Photos app chokes and crashes and manually babysitting smaller batches is a pain. It’s pretty clear they want to make it as hard as possible

Related: does anyone know of a way to delete the original videos on trimmed ones? Apparently all edits to videos are non-destructive, so the 10-minute video that I trimmed to 2 seconds still takes up 8gb.

Is there an iCloud Photos uploader?

I have a script to scan files from my camera and add a compressed copy to a folder. This folder was supposed to work with the iCloud for windows (10) program, but one day it just stopped working.

Incredible. I have such a hacked-together system to get my iCloud Photos backed up to an external drive while not filling my main laptop drive. This would be even better.

For those of you on Android / PC-Mac - Syncthing is an excellent tool for P2P syncing between multiple devices (it works over NAT and IPV6 too!).

Ok https://aionlywebsite.pythonanywhere.com/

If you are already in the progress of downloading all of your photos, this is a great time to consider completely switching over to Immich.

It's great!

Why did Apple invent this abomination of a system for storing photos?

We have had file systems for decades. They work well. They sync trivially.

There is something similar to Google Takeout for Apple Photos. Recently used it to download thousands of photos.

Anyone got any tips for programmatically downloading photos in a shared album?

iCloud Photos.app seems to not be able to show some photos

But Photometor.app (owned by Apple) can...

So that's a little annoying... I wish I had more visibility on photos not showing up in Photos.app, and what it is that stops it showing them

You know you only need to do a GDPR request to Apple (dedicated page), select images and you get a download link after a few hours

I mean can't you go to privacy.apple.com, ask for an archive of your data, and then they'll email you the link to a zip file in a week or two? I'm pretty sure this is what my girlfriend did when she transitioned from an iPhone to a Pixel. I think there's even a specific checkbox for photos/videos

It's fucking amazing sometimes how the Mac/iOS Photos app can't download a photo or small video for several minutes but you can easily watch YouTube on the same connection.

And it took Apple YEARS to give us a "Keep Downloaded" option for iCloud Drive documents in Finder.

And it's been years since I read any ebooks because the damn Mac/iOS Books app keeps removing my downloaded books even though I have several GBs of storage space left.

Goddamn Tim Cook and the other execs, do they even ever use their own products at all?

Fuck, my wife got a notice that she would have to increase her iCloud storage so last week began the process of ordering a backup of all her pictures so I could get them off iCloud and organized on some drives at home. We got 12 zips of the pictures along with csv's and some metadata, and I literally just finished iterating on the script to sort them into year-based folders and convert all the HEIC shit into JPG. It's running literally right now.

Guess I should've searched harder!

why not use rclone

I created this over the holiday break to do mostly the same:

https://github.com/rcarmo/PhotosExport

...when you try to export files using the (restricted) APIs we get, it automatically triggers a download.

Vulnerability hijack incoming with the headline calling for another maintainer

Became more fascinated with the history of my small hometown (Paris, Texas) TLDR: Much of it was wiped out by a 1916 fire. I spent some time recently vibe-coding this interactive map to provide some kind of historic visualization ( which enabled me to see the impact better ) https://gorch.com/parisfiremap/

> Disable Advanced Data Protection

Nope, bzzzzt, wrong!

You need an extra tool to download your own photos? That's... not a basic feature?

I'm always surprised what kind of antifeatures people in Apple land are willing to accept and still use those things...