Hm I wonder why snapshots wouldn’t satisfy 3. Copies on the same disk like /file, /backup1/file, /backup2/file should satisfy 3. Why wouldn’t snapshots be equivalent if 3 doesn’t guard against filesystem or hardware failure? Just thinking and curious to see opinion.
Does this make sense?
If Raid is backup, then Unraid is?
Try ZFS send if you have ZFS on the other side. It’s insane. No file IO, just snap and time for the network transfer of the delta.
Every hour. Could do it more frequently if needed.
It depends on how resource intensive the backup process is.
Consider an 800GB Immich instance.
Using Duplicity or rsync takes 1 hour per backup. 99% of the time is spent in traversing the directory structure and checking which files have changed. 1% is spent into transferring the difference to the backup. Any backup system that operates on top of the file system would take this much. In addition, unless you’re using something that can take snapshots of the filesystem, you have to stop Immich during the backup process in order to prevent backing up an invalid app state.
Using ZFS send on the other hand (with syncoid) takes less than 5 seconds to discover the differences and the rest of the time is spent on the data transfer, at 100MB/s in my case. Since ZFS send is based on snapshots, I don’t have to stop the service either.
When I used Duplicity to backup, I would backup once week because the backup process was long and heavy on the disk array. Since I switched to ZFS send, I do it once an hour because there’s almost no visible impact.
I’m now in the process of migrating my laptop to ZFS on root in order to be able to utilize ZFS send for regular full system backups. If successful, eventually I’ll move all my machines to ZFS on root.
What’s the second B stand for?
Why did you do that to yourself… 🥹
With 2 disks that would be type mirror in ZFS-speak, completely built-in. Equivalent to RAID1 in terms of hardware fault tolerance.
You could do a 3-disk mirror or n-disk mirror really. The RAID5/6 rough equivalents are called RAIDzN where N is the number of disk failures they tolerate. E g. RAIDz1, RAIDz2, etc. You probably want a mirror unless you need more space than a single disk provides.
Yup, turn it on, let it do a scrub, then turn it off. I’d still use redudnancy though. Not merely to cover the case of the drive failing, but also to cover the bit rot use case. It’s exceedingly unlikely bits to rot at the exact same spot on two or more disks. When ZFS finds a checksum mismatch during a scrub (which indicates bit rot), it’ll be able to trivially recover the data from the drive where the checksum matches. It’ll then rewrite the rotten part.
ZFS with automatic snapshots and scrubbing. This will keep as many and as old snapshots as your like. It’ll ensure the files don’t rot. It’ll ensure the media doesn’t die, so long as you have enough redundancy and you replace disks as they die. This is what I’d trust for long term storage because I think I understand how and why it works. It should last as long as I feed it disks. If I delete something, I should be able to restore it from a snapshot. The hardware doesn’t need to be anything fancy. Just a Pi 4/5 with a couple of WD Elements would be fine. Could add more disks for more redudnancy. I’m running 2-disk residency.
You don’t have to touch the software if it’s not exposed to the Internet. Whatever works today on it will work 20 years from now, so long as the hardware works. A couple of spare Pis, SD cards and power supplies should let it last for decades.
Use Debian LTS or Ubuntu LTS (10 years support with free Ubuntu Pro). Turn on automatic unattended updates. Upgrade OS when you’re bored one of those years.
Keywords:
I used a mix of Elements and MyBook for years. Upon opening to heatsink, I didn’t see any significant differences between them. They use ASMedia or Jmicron, mostly ASMedia. The overheating issue depends on ambient temp and load. I’ve had one machine in a basement never experience them. Either way the solution is pretty straightforward and cheap. Once heatsinked, I haven’t had a problem.
The cables they come with are good.
Absolutely can and should use docker in a VM. ☺️
Predictable cadence, stable operation, timely updates, huge community and therefore documentation. You can get up to 5 years from an LTS release of Debian or Ubuntu. With Ubuntu LTS and Ubuntu Pro (free) you could theoretically run a machine without upgrading for 10 years. If you run workloads in containers, it doesn’t matter how old the host OS is. As long as it’s security patches, you can keep on trucking.
As briefly as possible:
If you want to program something, the closest you’re gonna get to programming is Ansible and Bash scripts.
You might want to get self hosting hardware like Synology or the like if you’re not ready to dig.
Otherwise here’s some things you need to know:
Oh and use Debian or Ubuntu LTS.
Windows are privacy-invasive.
I’m trialing it but I’m also thinking of trying Linkwarden to compare.
Right so I guess the question of 3 is whether it means 3 backups or 3 copies. If we take it literally - 3 copies, then it does protect from user error only. If 3 backups, it protects against hardware failure too.
E: Seagate calls them copies and explicitly says the implementer can choose how the copies are distributed across the 2 media. The woodchipper scenario would be handled by the 2 media requirement.