Tag: apple photos

Apple Photos to Lightroom Classic CC: Step 2 [en]

[fr] Le suite de mes misères pour revenir à Lightroom après mon passage dans Apple Photos...

Here’s where I left things in March:

  • I had a catalog of my exported Apple Photos
  • I was hoping to merge them into my Lightroom master catalog.

One little problem: importing from catalog didn’t recognize any of the photos in the temporary catalog full of photos from Apple as duplicates of those already in the master catalog. So my bright idea of using this system to update the metadata of photos in the master catalog came crashing to the ground.

I figure it has something to do with the hash Lightroom uses to identify photos. Anyway… off to find another idea.

My next hope had to do with using the Syncomatic plugin to sync metadata between files with similar names or capture times. A little scary to run on the whole catalog, and I haven’t managed to get it working predictably enough to trust it. Might still come, thought?

So I ended up looking at Photosweeper and LR/Transporter. There was my solution, though it was tedious, and the result is not perfect.

Here’s a bunch of thing I did, best I remember.

I used Photosweeper to weed out duplicates from my Apple Photos. I tried various settings and experimented quite a bit. This also allowed me to get rid of “Edited in Apple Photos” photos which were so close to the original photos it wasn’t worth importing a duplicate bloated JPG just for fun.

LR/Transporter allows you to import metadata to a photo based on a filename or a capture date.

That meant that if I could export a list of keywords of my Apple Photos (with LR/Transporter) and import it onto photos that were already in the catalog, matching the keywords to files with the same name, I could avoid importing duplicate files and having to hunt them down afterwards.

I would have to find a way, however, to flag the photos in the Apple Photos folders from which I had extracted and imported the metadata — because those that remained, “non-duplicates”, would need to be moved to the main folder. Thankfully, LR/Transporter flags photos that have last been modified by it. So once I had imported the keywords from Apple Photos, I had only to:

  • use the metadata filter on my master folders to select all photos that had just been modified by LR/Transporter
  • export a list of those filenames
  • edit it in Google Sheets and add a “delete” flag column
  • import that metadata back to the Apple Photos folders, matching the delete flag column on the “job identifier” field (for example)
  • select those photos through a filter or smart collection and delete them

I did this on the Apple “original” photos I had in my collection.

You’ll not that it’s totally uninteresting to go through this exercise with Apple Photos that don’t have keywords. As I had previously keyworded all apple photos to identify if they were master photos, edited photos, master photo of an edited version, HDR, panorama, etc, I couldn’t easily filter out photos with no “real” keywords. I started out by replacing all those “status” keywords by flags in the “job identifier” field. For exemple

  • I selected all apple edited photos
  • added “apple edited” in the job identifier field
  • removed the “apple edited” keyword
  • used the “job identifier” filter to display those photos again
  • now I could see which ones had no keywords
  • selected those and gave them a slightly different job identifier, etc.

So when doing the LR/Tranporter export above, I was able to select only the photos that actually had keywords to work with.

What about those that had a different name? I used a similar process, counting on the capture time. Her’s what I did, year by year (to avoid having unending tables to open in Google Sheets — max 10k photos, roughly).

  • I selected a year of photos with keywords both in the “master” and “Apple Photos” folders
  • I exported them with LR/Transporter. Fields: size, filename, path, keywords, capture time
  • I opened the CSV in Google Sheets and added four columns to help me “sort” them and generate for the correct file in each matching pair either the updated list of keywords or a delete flag (see example spreadsheet here)
  • I replaced the path for all the files in the Apple Photos part of my directory structure with “apple” for readability’s sake (useful when scanning to check things)
  • Ordered the photos by capture date (with Apple folder photos first, so sort them A-Z by that column first) to let the formulas do their magic
  • Duplicate the sheet, then copy-“paste special” (values only) the columns containing the formulas
  • I went through the list checking that the formulas worked correctly (this is where filename and size come in handy, as well as conditional formatting in the spreadsheet to create a visual pattern that is easy to scan for anomalies), corrected manually when necessary
  • Once that was done, used some A-Z sorting to delete the lines of all the photos that did not match anything
  • Duplicated the sheet once more (trying to be safe) and removed all unnecessary columns
  • Duplicated it one last time so I could have to clean “export” sheets: one to update the keywords, one to set the delete flags
  • Import the “keywords” metadata sheet onto the photos in the master folder
  • Import the delete flags metadata sheet onto the apple photos folder photos.

Once I had done that for each year, it meant that all the Apple Photo original photos with keywords had been processed to see if they had a “twin” already in my Lightroom photos, and keywords imported accordingly (and source file removed).

I then was able to import the remaining Apple Photos files into my main folders, knowing there should be no obvious “same time” or “same name” duplicates — except of course for edited photos. (For that, I deleted them from the main catalog and imported/moved them using the standard “import”, so that they would sit in the right monthly folders.)

The saga continues, as my reduced list (12k from 20k!) of Apple Photos is now integrated with my main catalog, but there are still duplicates in there.

Ideally, I’d get syncomatic to work how I want it to, sync keywords, then use Find Duplicates 2 to delete duplicates. But that’s not going well for the moment.

In addition to this, somewhere in the process a pile of my photos have lost their capture date, or seen their capture date replaced by “today’s” date (the day I was obviously doing whatever caused them to lose their capture date). So I have about 2k badly dated (or undated) photos I need to find a solution for. Many of my videos seem not to be read correctly by Lightroom anymore, and have metadata/date issues. My catalog is also sprouting metadata conflicts, and Lightroom very helpfully (not) asks you whether you want to import metadata from the file or write Lightroom metadata to the file without showing you what the conflict is made of. Not much chance of troubleshooting what is going on in there.

So, what’s left to do?

  1. Identify duplicates with FD2 and figure out a way to be systematic about which versions I keep (I’m doing a test on a month of photos to see how it goes). The key seems to be smart collections, and using color labels like I remember doing in March: for example, color all the “most edited” duplicates identified by the plugin green, then go and look at the full list and figure out if that is a good criteria or not.
    But for that to work I need Syncomatic to work, or to mess around with LR/Transporter again, because many of these duplicates do not have synced keywords. *sigh*
  2. See what’s going on with those videos. Not too sure where to start, but I was pointed to this, and need to dig into it. Reminder, there are three problems: systematic metadata being flagged as up-to-date though in a smart collection it’s indicated as changed; mess-ups with capture times; unreadable videos.
  3. Look at the photos with a bad capture date (or none) and see what I can salvage. One idea is to find a back up of my master catalog before all this mess started happening, use LR/Transporter to match those “lost” photos to their “ancestors” in the old vers of the catalog, and use a process similar to the one described above to fix their capture times. But I’d still like to know what created this situation so I can avoid it in future.

    I’m not sure how understandable all this is, these are mainly notes for myself for when I pick this up again, but if it’s useful for you, all the better! Feel free to ask questions if certain parts are unclear and you would like explanations.

Moving From Apple Photos to Adobe Lightroom Classic CC [en]

God have mercy on me. A few months ago I decided I was coming back to Lightroom. Now is the time to actually move my stuff out of Apple Photos and into Lightroom. It’s not so much emptying Apple Photos that concerns me as transferring albums, favorites, and editing over to Lightroom.

I had foreseen the headache, and so I am documenting what I’m doing here first of all for myself (because I might end up abandoning halfway through, as usual, and picking up six months later, having forgotten everything), and also for other poor souls out there who might be in the same situation.

First, the easy part: exporting from Apple Photos.

  1. One thing I wanted to “export” was my albums. I went through each album I wanted to keep, selected all the photos in it, displayed information and added a keyword like “my cats album” to all the photos. Kludgy and a little tedious, but does the trick.
  2. When viewing photos Apple lets you display “only edited” photos. This allowed me to export both the edited photo and the unmodified original for photos I had edited in Apple Photos. I then exported the unmodified originals of photographs I hadn’t touched in Apple Photos separately.
  3. I exported these photos into three separate folders, without any subfolders: “Apple edited”, “Apple originals”, “Apple unedited”. I renamed the edited photos to avoid file name conflicts later on, but left the originals/unedited file names untouched, in the hope it would help Lightroom detect duplicates/updated photos later on.
  4. For the original files, I told Apple Photos to write IPTC to XMP. This works great for RAW files (Lightroom grabs the metadata from the XMP sidecar) but not for JPG originals (who are not supposed to have a sidecar). After fumbling around I found my solution: a simple command-line command for exiftools. The person posting had pretty much the same problem as I did, and I just used the solution offered as-is. It throws some errors (when XMP files don’t have anything interesting in them, I think) but works fine.

Now for the real fun: importing into Lightroom.

  1. For this, I used a temporary working catalog, rather than mess up my master catalog directly. I made the working catalog by exporting some photos as a catalog from the master catalog, and then removing those photos from the temporary catalog (not the files though, beware!)
  2. I started with the edited photos, followed by their original files. I moved them into a month-based folder structure parallel to the one I use for my main library (in a folder called “Apple import”). Upon importing, I gave each batch a keyword to be able to figure out who was who later on (“appleedited” and “master of apple edited”).
  3. I ran Find Duplicates 2 on those photos and it turned out quite a pile of them. Not that surprising. I decided to have a look, and saw that there were indeed a lot of “edited” photos that were so close to the original (or unimportant) that I wasn’t going to bother importing a bloated redundant JPG of those “edits”.
  4. I proceeded to cull those “duplicates”. I started out by giving all those photos a keyword to recognise them later (see how I abuse keywords?). I then rejected all the “mess” (screenshots, photos of bank statements…) that comes with exporting photos from your phone.
  5. I then went painstakingly (but as efficiently as possible) through the unflagged photos and used a label to identify those where I was indeed going to keep both the edited version and the master. I could have skipped this but I figure less bloat is better.
  6. Amongst the unflagged and unlabeled photos with the “duplicate” keyword, I filtered for those with “edited” in the file name (remember how I renamed the edited photos upon export from Apple Photos? handy; I could also have used the keyword I attributed the edited versions upon export, come to think of it. Oh well.) I rejected all those edited photos I decided not to keep.
  7. Similarly, I selected the originals for those photos and changed their keyword to indicate they were not a master photo for an edited version anymore. I also removed the duplicate tag and then cleaned up my mess of coloured labels.
  8. I am not deleting any rejected photos until I get everybody back into my master catalog. Hopefully this will clean up a bit of the “smartphone mess”…or not.
  9. I then proceeded to import the photos from Apple Photos which hadn’t been edited. Just 20k of them. It was loooooong.

Now… how to merge all this back into the master catalog without losing any information and without multiplying photos excessively… I’m not sure I have the solution, and I’m going to err on the side of not losing data. I can always hunt for duplicates later.

I picked a year where I had only a couple of hundred Apple photos, and exported a working catalog from the Apple import catalog for only that year. I then imported those photos into my master catalog, without moving the files. To my dismay Lightroom didn’t recognize any as duplicates or updated files. After looking at things manually it’s clear there are duplicates and I was very wise to not try and move the files to their right place in the catalog yet (filenames are identical!)

I set Find Duplicates loose on all the photos for that year. As I’ve previously cleaned up my whole catalog of duplicates, and marked “fake duplicates” with a keyword that allows me to filter them out, I end up with a shortlist of duplicates between my newly imported photos and those that were already in the master catalog. The “edited” photos in the duplicates are not much of a problem, as they are strictly speaking “fake duplicates”. The master photographs are more of a problem: I’d like to retain the keywords from the new photo and whatever keywords/ratings were on the old photo. I can do that by manually synchronising metadata, but it’s super tedious.

For the time being I’ll just mark those duplicates “appledupes” until I can figure out what to do with them.

Next in line:

  • moving those photos into the “final” folders (will involve renaming the Apple photos)
  • trying a year with more photos.

Back to Lightroom [en]

[fr] Retour à Lightroom après deux ans et quelques d'infidélités avec Apple Photos.

Two and a half years ago I took the plunge and started using Apple Photos “seriously”. It quickly became my main photo library, and the comfort of having photos sync seamlessly across devices became something I was not willing to do without. Lightroom was just not there yet (I tried, it was a nightmare), so Apple won.

Over my holidays I peeked back into Lightroom, which I’d neglected since then. And it clearly wins when it comes to organising and editing photos. Time has done its magic, too, and syncing across devices now works! It’s still reasonably early days, but it’s good enough for me.

Here’s what I’m looking at:

  • my main photo library is (and remains) Lightroom Classic CC — or “good old Lightroom” that we’ve known for years
  • I have the mobile version of Lightroom on my phone and tablet
  • I have created a collection called “mobile” in which I stuff the photos I want to sync with Adobe Creative Cloud and have available on phone and tablet (right now, all my 2018 photos)
  • I have set my phone to “auto-add” any new photos from the camera roll into Lightroom: this means that if I take a photo with my phone or tablet (omg), it will be added into Lightroom mobile, synced over Creative Cloud, and downloaded to the correct monthly folder on my computer (“Lightroom sync” setting in preferences in Lightroom Classic CC)
  • I have also installed Lightroom CC (desktop client built from the ground up specifically for dealing with photos stored in Creative Cloud), without making it download originals (it’s in the settings), so that I can benefit from the AI subject detection to search photos
  • I also use the web client so that I can benefit from the AI “pick my best photos” functionality — this is seriously the killer, as far as I’m concerned
  • I have a monthly “photography” subscription which includes Lightroom Classic CC, Lightroom CC, Photoshop CC (+Spark&Portfolio), and a measly 20Gb of cloud storage
  • I’ll certainly shell out what’s needed for the 1TB plan at some point, but as I’m only syncing Smart Previews to the cloud from Lightroom Classic CC, the 1000+ photos I have in CC don’t even take up 8Gb (my library is 70k, but a few thousand photos in the cloud is enough to play with it for a bit)

I do have a few headaches:

  • RAW and JPG: I’ll let you read the thread for details, but I’ve come to the conclusion that I should be working with JPG. I’m happy to not retouch photos if I can avoid it.
  • I’ve taken a lot of “RAW+JPG” photos with my camera, which means I have the JPG handy, but there is no way in Lightroom to say (like in Apple Photos) “hey, use the JPG for this one”; either the JPG is simply there as a sidecar, or it’s a separate photo, and there is no way for Lightroom to “know” that photos A and A’ are in fact the same photo in two different formats
  • I don’t like the idea of throwing away the RAW file, but the way Lightroom deals with RAW+JPG pairs is making me consider doing JPG only
  • I’ve taken some “RAW only” photos… so I’m going to have to deal with those. My photo post-processing skills aren’t great, and it’s not something I take pleasure in. I did get a Huelight camera profile for my old Lumix G2, which seems to help a bit.
  • I have a pile of albums in Apple Photos, and retouched photos, that I’d like to import into Lightroom. Apple Photos lets you export either the originals or the edited photos of any album, which can then be imported into the Lightroom catalog, and between the Find Duplicates and the Teekelesschen Duplicate Finder plugins I can figure out which version of each photo I actually want in the catalog. I’m still fiddling with the process but it’s workable. (I discovered the use of temporary working catalogs doing this, yay!)