Took your advice, ordered EHD, iMac- new questions

[mistyprochaska]

Brian, 

I asked last week if your thought my iMac needed to be replaced, and I needed a new EHD because my 3TB EHD is full of photos. So since then, I took my iMac in, everyone involved agreed it will be a great computer for many years to come for my kids, but due to the type of usage it sees with my daily and constant photography, a new machine would be better. I ordered the 27 in iMac, the fastest processor, included 16 GB of RAM, ordered an additional 16 GB of RAM from Crucial, and ordered the WD RAID drive you suggested, 12TB, which I will set up as RAID 1 (mirroring the drives). I have an additional EHD for Time Machine, and everything is backed up through CrashPlan.

This is my new question. And it is going to be about Lightroom, and I am not sure if you are also anti-LR, but bear with me. In the past I have always switched out my EHDs before they were full, because I know they typically fail and I try to avoid that by just semi-regularly replacing them. Last time, I think my main EHD started making strange loud noises. So the problem I had was moving my photos to a new EHD. I wish I would have written down what happened, but it was really, really bad. I was on the phone with Adobe CS in India, and I cried for most of a day trying to figure it out. It has been three years though, and I can no longer remember what I did or what the problem was. Per advice from Victoria Bampton, I understand that it is not advised to move a very large volume of photos from one EHD to another THROUGH LR. I think it is advised that you copy the photos from one EHD to another, then have LR reconnect with them. I an very, very anxious about having to move these photos. My EHD is coming today, and the computer will be here Thursday, and I am not even excited at all because I don't want to move the photos to the new EHD and set LR up again. 

Do you have a recommended process? Like an extremely detailed recommended process? Where should my catalog be while I am moving the photos? I don't store my LR catalog with my photos, it is on my iMac HD now. Do you think it would be ok to move the photos (140,000 of them) through LR? I have them divided into folders by year, and then within those folders by month. Thanks for any help you can provide...

[Brian]

I actually use LR, but am slowly moving away from it. Damien is persistent.

 

4 hours ago, mistyprochaska said:

Per advice from Victoria Bampton, I understand that it is not advised to move a very large volume of photos from one EHD to another THROUGH LR.

 

I agree 110%.

Large LR catalogs have a habit of corrupting, randomly. So that photographer that hasn't had one problem with a million photos in his LR catalog will convince you that there isn't a problem and just go ahead and do it. Which in my opinion, is like playing roulette. I have personally never transferred large amounts photos in LR, I've always done this outside of LR. Too risky. LR catalogs have enough problems as it is. So since you are asking me, I would do this outside of LR. The downside is LR gets really cranky when it can't find things. OMG does it b-i-t-c-h. Honestly, do you really need all of those photos in LR? Think about it. After you delivered the photos to the client from 3+ years ago, are you seriously going to dive back in and re-edit them? Most of the time, you will have your latest sessions and maybe your "Greatest HIts" photos. The others just sit there, just taking up space.

What you should do is turn on the EHD icons on your desktop, and that's in the Finder Preferences Menu. Right now I'm on my Work's laptop, so I can't do screen shots. I will post them later.

Basically what you want to do is COPY the photos from one HD to the other. You NEVER-EVER-EVER want to MOVE the photos. The reason is, when you move files and folders, you are deleting the photos from the source folder as it copies them. If something happens and there is a glitch during the data transfer, there is a very good possibility that the photos will go "Poof," which is bad and causes tears.

When you copy, you aren't deleting anything, as the source files remain intact. If something screws up, you can cancel the data transfer and start all over again. I myself used to be one of those guys that always moved files. What could happen? I'm a computer guy, I know what I'm doing!!! LOL!! Ummm....yeah. I ate Crow, not sure if it tasted more like Chicken or Turkey. I hadn't had a single problem in 20+ years...until that one data move cost a customer 100 non-replaceable files. (Which turned out to be corrupted .pdf files on the destop, but we didn't know that until we tried to transfer things.) Learned my lesson on that one.

OK, before we start moving anything, let's get that WD drive setup as a RAID 1 and make sure things are happy. THEN we will start dorking around with your photos. Sound Good?

But to answer your question, yes...you are correct. We will do a mass copy to the new EHD then load LR on your new iMac (You can have two licensed copies running with Adobe's products.) When the LR is installed and the photos are transferred, we can re-import things. I'm also sure there are presets and other stuff to move over. This whole process is a bit of a pain, and is tedious. This stuff isn't fun, but it is a way to start over fresh. I would also download the latest LR CC 2015 and PS CC 2015. El Capitan has known issues with Adobe's products, and the LR / PS CC 2015 versions have the greatest chance of being updated to fix bugs.

[Brian]

5 hours ago, mistyprochaska said:

Do you have a recommended process? Like an extremely detailed recommended process?

Yeah, it's in my head. I haven't written it down. Some members have paid me to log into their Macs and do things for them. I have been toying around with a "Macintosh Class" for awhile. We could also network the two Macs together, but that's a whole process and one I'm not willing to tell you how to do for free. It's techy-nerd-type-stuff.

 

5 hours ago, mistyprochaska said:

Where should my catalog be while I am moving the photos?

We are going to create a new one. I know you don't want to hear it, but instead of trying to link every single file and folder, all 140,000 of them, it's just faster and easier to create a new catalog. Do you really want LR to find 140,000 files? Because it's not going to do it on its own, you are gonna have to click them. That would take hours, possibly a week or two. Honestly, it's just better to start fresh and import the photos that you want / need. It already sounds like you have a data-management process / file-structure in place, so we will stick with that. If you caved in and used Bridge, as Damien recommends, this whole thing would be a non-issue.

 

5 hours ago, mistyprochaska said:

Do you think it would be ok to move the photos (140,000 of them) through LR?

Hell to the NO!! See above.

[mistyprochaska]

Oh my. I don't know about the hard drive. It is just this huge box sitting here taunting me. I for some reason thought I would just wait (and wait.. and wait). No, I thought I would wait until the new computer gets here. I am adding more photos every day (I know, I know), so I thought it would be best to do the transition when I had the new computer up and running. Or to be honest, yeah, I just don't want to face it. 

I do plan to set both computers up together, and I actually don't want to set the new computer up via Time Machine. If this current computer has weirdness or whatever, I don't want to just install it into the new machine. So I thought I would just move everything over that I wanted. So I am totally on board with fresh starts in that sense. 

As far as the photos, the thing is, until right about yesterday, when I agreed to shoot my first wedding August 1st, I have been a hobbyist. I am a really serious hobbyist, but I have not been in business. So a lot of my photos are of my kids, and of still life, some of which is used for stock photography. And I actually did just have to go back through years and years of photos to find particular photos for a stock photo project because they needed larger files than what I had originally submitted years ago. And the photos of my kids, I just have a hard time getting rid of. In summary, while I know I have a HUGE photo hoarding problem, I don't want to address it now. I will be better about culling day to day, and I will go back through old photos systematically and delete what should go, but at this point it is harder to decide what to keep than to keep everything. And when I do start shooting professionally, I will create a new catalog for each client/job. 

And now that you say the clicking thing, yes, I am having horrible flashbacks, and I know that I had to go through and click a ton of stuff to reconnect, but even getting to that point was nightmarish. But wont creating a new catalog cause a loss of all of my edits? I am open to whatever will work. I am open to compensating you for your help too- I spend half of my life giving people advice on choosing and using mirrorless cameras for no compensation- so if you are not getting paid for doing this, I totally get that. I am going to keep trying to do some research on my own as well, but I remember last time feeling fairly confident it would be ok, and also everyone said, hey, this is not a big deal, just do it! And then it was such a big deal. 

Let's say I get brave and start copy my RAW files on to the new EHD. Do you recommend copy and paste, or a program to do it? I know people have mentioned programs that are helpful for tasks like these, to make sure it is copied exactly and no files are missed.

I also know there is something I have to remember in this process about Crashplan- don't unplug the EHD, or do, or don't rename it, or something (or maybe there was some trick with the LR transfer about not renaming something). Anyway, it is a way that Crashplan knows that it is not new data, and that it is already backed up, but that it needs to start backing up from the new EHD. 

I know this was the "right" thing to do, but I really will be so happy when this is totally over and resolved. 

[Brian]

No special software. Just good 'ol copy and paste. 

Step 1: Let's get that WD configured for RAID 1 and formatted for use with the Mac. 

[Brian]

Creating a new catalog will lose your LR settings for each file. Yes. But in reality, those edits are only applied when to export from LR. 

That's one of the big problems with LR catalogs. If the Catalog corrupts itself, all that labeling and edits with the sliders are lost. 

So, what you might want to do is export those files that you go keep going back to and export them as .psd files in a special folder. Or create presets based on those edits. 

[Brian]

Crashplan is a whole other thing. We might have to create a new thread and I'll move it to the Land of Misfits for others to chime in. I have zero experience with Crashplan. 

[mistyprochaska]

I really appreciate your help! I can't lose my catalog, because I edit everything by hand and there would be endless presets I would have to create. It just wouldn't be reasonable. I will just have to follow the recommendations I can find and hope for the best. I know that as long as I have the catalog, and as long as I have my photos, I can get everything to work together eventually. I am sure it will work out, I just dread getting to that point. I appreciate your specific hard drive recommendations, and your verification that I do not want to move the files through LR. I will not do that. Soon, I will bring myself to take the EHD out of the box. I still have two days before the computer gets here...

  

[Christina Keddie]

Just so you know -- as long as you have your presets saved on your hard drive properly, and as long as you've made the LR settings change to write the XMP data separately from the raw files, you wouldn't lose anything if you lost your catalog.

Please tell me you've got your presets saved on your hard drive, and that you've changed your LR settings to write the XMP data separately.  

[mistyprochaska]

1 minute ago, Christina Keddie said:

Just so you know -- as long as you have your presets saved on your hard drive properly, and as long as you've made the LR settings change to write the XMP data separately from the raw files, you wouldn't lose anything if you lost your catalog.

Please tell me you've got your presets saved on your hard drive, and that you've changed your LR settings to write the XMP data separately.  

I do have the box UNchecked for "Automatically write changes into XMP. And my presets are on my hard drive in the library folder (aren't they installed there when I install them?), but I also have them wherever (probably in my terribly disorganized downloads folder). I basically use presets never, ever. I don't even create my own presets, and I should, at least as a starting point for some things I often do.

I think that is where I have to take a deep breath and know that everything is going to be fine. I have my catalog. I have my photos. The EHD with my photos will still be here. Not only that, but the catalog is backed up through Time Machine, and EVERYTHING is backed up through Crashplan. SO eventually it will all get set up, some how or another. I may not do it in the most efficient way, but it will eventually work out. I think. 

[Christina Keddie]

3 minutes ago, mistyprochaska said:

I do have the box UNchecked for "Automatically write changes into XMP.

WAIT WAIT WAIT -- you want that box to be CHECKED.  And then you want to make sure that all your raw files in your hard drive folders have .xmp files safely sitting right next to each of them!

[mistyprochaska]

3 minutes ago, Christina Keddie said:

WAIT WAIT WAIT -- you want that box to be CHECKED.  And then you want to make sure that all your raw files in your hard drive folders have .xmp files safely sitting right next to each of them!

Ok, I just checked the box. I wonder if I had it unchecked for a reason, or never checked it when I transitioned to CC. What will happen now? Is it going to go back through all of my files, or will this only be created going forward?

[Christina Keddie]

It's been too long since I checked that box myself so I don't remember -- best thing would be for you to open a random folder of raw files on your hard drive and see if XMP files have popped up.  (And if they aren't there, try rebooting LR -- I want to say the XMP files might be created once you load the catalog?)

[Christina Keddie]

Looked into it a bit, and I believe checking this box will only automatically create XMP files going forward.  But you can also select any files in your Library that you want XMPs of and then choose "Save metadata to files" under the Metadata menu, and that should do it for you.  I wouldn't recommend doing this for 140K files at once, though...