Help with my backup plan please

[ARoel]

Hi,

I'm formulating some sort of my own backup policy because as I just realized, my file organization for the past decades have been a mess (no backups, some even with triplicate files in one HDD) and I'm lucky I was able to recover my decades old family photos. I lost a lot of photos before just because of hardware failures. Surprisingly, not a single one was lost due to a corrupted files (at least from what I can remember). I want to make sure I avoid all that now. 

As of now, I have these:

1. Working PC equipped with 2 x 4TB HDDs, set to RAID1.

2. Home server - Truenas with NextCloud installed, equipped with 4 x 4TB HDDs, set to RAIDZ2 (I think this is similar to RAID6). Set up in the same house where my working PC is.

3. OneDrive subscription

 

My planned backup "rule" as of now is to:

1. Retain ALL raw files in my working PC, whether they be active or not. I'll archive everything there.

2. Working PC is then synced to a folder in NextCloud (Home server). No file in here will ever be touched or accessed unless my working PC gets stolen, catches fire, or things like that, unless I really have to (something which I will try to avoid at all cost). Files will never be uploaded to NextCloud  directly as well.

3. NextCloud is then synced to OneDrive. Similarly, no file will be accessed unless both my PC and Home Server get lost at the same time. No files will ever be uploaded here.

 

I'm new to this and I'm not sure if this is a good plan to follow. I think this is enough protection from hardware failure. or maybe not?

Remember I just said I never had a corrupted file? Not anymore. I just had it last night (It's in this topic:

. )

 

I'm not sure what caused the file to be corrupted but what happened eventually was: corrupted fie in main PC = corrupted file in home server = corrupted file in OneCloud. So, not a single useable file was left. Luckily it was my first file using this new system I want to follow. A blessing in disguise and a wake up call, telling me that I am not doing enough.

So, with all that, what else should I do? Basically, I just want my kids, their kids, grandkids, and so on to still be able to access these files in the future. Thank you.

[Brian]

Keep in mind, RAID1 (Mirrored) is just redundancy. If the file is corrupted on the 1st drive, it is instantly corrupted on the 2nd HD in the RAID1 configuration. Sometimes it's a fluke, other times it could be something like a bad sector on one of the HDs, in which case things can get botched up, as the data is mirrored. Keep an eye on things. Once...OK...things happen. But if you find multiple images giving you grief? Something is wrong with one of those HDs! Again, RAID allows for tolerance in keeping you up and running for as long as possible, so you can get to your data; RAID is not a backup strategy!! 

How you choose to back up your data, is really a personal thing. In reality, before going nuts with backup strategies, it is more important to have a file/directory/folder structure that makes sense to you. This way you can avoid triplicates all over the place and you can find things quickly in the event of a disaster. Secondly, testing your backups is imperative. I can't tell you how many times I've performed resurrections and voodoo chants with my Customer's Servers, only to find out that several backup tapes are no good and they haven't had a good backup in MONTHS. Then their main HD dies and I can only restore the backup from 11 months ago, the recent stuff has to be inputted in, by hand, all over again. 

So the first thing I would do, is to get your file structure files/folders straightened out, THEN purchase a larger external HD, like a 8TB (or even larger) to copy stuff to. One big data-dump. Use that as your "Workbench" to see how easy it is to navigate to, pretend your main HDs and Server was in a flood or fire. If you can figure out what is where within seconds, THEN you can pay attention to the Cloud Stuff and folders on your Server. Keep things simple. It doesn't do you any good not knowing what is and what isn't backed up when things go horribly wrong. Yes, this is going to be a PITA and a huge time-suck, but you are wasting time when things are working, rather than doing it the other way around, which is not fun at all. I speak from experience. 

Offsite Backup is imperative, Home Server is another, and having external access to that server helps. Always remember, the Cloud is "Someone else's Computer," and OneDrive isn't fool-proof. In fact, NONE of them are and I would always remain skeptical of these services. TRUST NO ONE!!  Always read the fine-print with Cloud Storage. I do not care how great a service claims to be, again...read the fine-print. That's why I recommend having a simple external HD to duplicate you main backups and then that EHD is stored off-site somewhere. For long-term storage, I recommend spinning traditional hard drives and not fancy SSD Drives for this. Why? Because Flash Media breaks down over time if the drive isn't powered on at least monthly. Digital Rot is a very real thing and a Traditional HD stores things magnetically, and lasts for years. SSD Drives? Notsomuch. Especially if you don't hook them up for a few years...the data goes "Poof."  

So fiddle around with things. Figure out a Directory / Folder Tree. You need to be able to find something within 30 seconds at all times, that's the goal. In fact, it needs to be as logical as possible, in the event that someone else needs to find things if you aren't around to do it for them. Do a data dump to a large EHD, then see how easy it is to get to your files, even using another person's computer. 

[ARoel]

Thanks Brian! Will do everything you advised.

The way I understand it for off-site backup, it would be better to just purchase a large external HDD, copy-paste all my files at home, bring it to some other place, then repeat this say every month?

[Brian]

That's what I do. Keep it simple.

Theft, Fire and Floods are all possible. Even if that "Flood" is the busted water pipe in the ceiling that's over your computer. That's why having a simple off-site backup is part of the 3-2-1 Rule. 3 Copies of your extremely valuable files, backed up on 2 different mediums and 1 is kept off site. The Cloud is great for convenience, but that only works when the internet is running and you have power with working hardware. What happens when a Tornado rips through your neighborhood? Sure your house might be standing, but you have no power for the next 3 months. Just having a simple large EHD can really help. Just make sure it is not a Flash Media Based EHD. You want a large spinning HD. SSD Drives and the like, need to be powered on, at least monthly, to keep the memory cells intact. Traditional Spinning HDs store data magnetically. 

You can still do your home server and OneDrive stuff...that's fine. I've also heard of good things with BackBlaze, but read the fine-print with them. They don't want to be dumping grounds, and if you delete the file(s) locally on your computer, they are also deleted on their Servers within 30 Days.

I like the redundancy of RAID, so keep doing that. The extra EHD is just an extra insurance policy if things go sideways at the worst possible time. Disaster Recovery should just be a PITA/Time-Suck, but you should be able to fully recover within a day or two. Not weeks.