Archiving HDD vs. SSD in 2020-2021?

[Reading Bug]

Hey all!

I asked this question a few years ago and wanted to ask again because... technology changes. Years ago the recommendation from fine folks here was that HDDs are more reliable than SSDs for long term storage, or "archiving." Meaning, for me at least, the traditional 5-year window between data migrations. My questions are simple. Is that still the case? If so, any idea how much longer until SSDs are equal to HDDs in this area, if ever? How do we feel about SSDs replacing HDDs in this regard? I feel like there may be some old school technical reasons to never abandon HDDs (just seems like one of those things). Is that right or can I look forward to a switch someday, price notwithstanding?

Thanks!

[JPMedia]

Much of the dabate over using HDDs vs SSDs for long-term storage revolves around the nature of how each type of drive fails over time. If HDDs experience failures they often occur over a long period of time. Also if part of a hard disk drive isn't operating properly, it can likely still power on and data can be extracted from it. SSDs fail much more suddenly and the process of recovering data from dead SSDs is difficult/much more expensive.

[Reading Bug]

Thanks! I had that general understanding. Do you have an opinion on improvements over time? I'd love to know how the gaps might be closed - improvement in circuit reliability; data retrieval costs coming down; improvements in alerting to imminent loss or corruption, and so on. Certainly prices for new SSDs coming down someday, to where they're comparable to HDDs, would make things easier for consumers to double up on redundant drives to compensate for sudden and complete losses (assuming no other improvements).

I mean, SSDs are the future. There must be some ways of improving their reliability. At least across that 5 year window.

I'm also interested in opinions on general improvements, such as read/write, wear (durability, performance, etc). Things that have typically made them less desirable compared to HDDs.

[Hushpower]

I've had dozens of HDDs over the years and only one failure (as mentioned, over a few days and I was able to have the data retrieved by an expert). I've had 8 SSDs in the last 3 years and two have died, instantly. Completely wiped in a second. I would never put anything critical for storage on an SSD.

[lordsmurf]

Nothing has changed in at least 5 years now. Some larger drives, new methods to allow it. That's really it.

Same advice still applies. Multiple copies, on multiple media types.

- use DVD-R (or +R), less so BD-R -- obviously smaller, so backup only most important, not everything
- use hard drives (multiples)
- never flash memory
- SSD should is working space, not for archive/storage

SSD have probably gotten worse in recent years, not better. Pushing more data results in slowdown and potential integrity compromises. QLC NAND is awful. I've not moved past using Samsung 850 EVO drives as a result (TLC). I don't understand why companies insist on 2.5" HDD SATA form factor when you could stick lots of TLC into a cube or thicker platter. Something larger in size, both physical and data size. NAND prices are falling, after all.

[Hushpower]

Quote:

I've not moved past using Samsung 850 EVO drives as a result (TLC)

They were the two that carked it on me.

[JPMedia]

Quote:

Originally Posted by Hushpower

They were the two that carked it on me.

I have an editing PC that now has 3 2TB Samsung 860 EVO drives in it. Originally I was using a 2TB 970 EVO to store media and project files, but it just up and died very suddenly. Extremely disappointing coming from an almost $500 SSD. Potential memory bandwidth means nothing if the thing is broken

[Reading Bug]

Thanks LS. Another question about data migration: if I have HDDs in storage, unpowered and put in use simply to add new files to them when I go there (the storage unit is in another town so it's only a few times a year), is it recommended I still observe the five year window? Or can I stretch it out a couple more years?

For that matter, it IS five years still, correct?

-- merged --

Bumpity bump bump?

[lordsmurf]

Stretch it longer. Just boot up several times per year, access, maybe even scan drive.

[Reading Bug]

Thanks LS. And congrats on post #10K.

[Hushpower]

Quote:

Originally Posted by Reading Bug

congrats on post #10K.

Yes, well done LS!

[lordsmurf]

Quote:

Originally Posted by Reading Bug

Thanks LS. And congrats on post #10K.

Quote:

Originally Posted by Hushpower

Yes, well done LS!

I'd never have seen that if you didn't point it out. I saw it that day. The big 10,000. Neat!

[OtakuSensei]

There's no perfect solution to the ages old archival question.

Quality of consumer grade HDDs has dropped in the past five years resulting in far more catastrophic failures than I have ever seen in the years prior. It use to be true that if your mechanical HDD failed there was a high probability of being able to recover most of your data from it. Now, the kinds of failures we're baring witness to are utterly absurd things that should never happen, platter damage! Platter damage is absolute and nothing can be retrieved from a damaged platter.

On the flip of that, SSD/memory chip data recovery technology has improved. It's still true that if a memory chip fails you lose the data that was on it and any overlapping files partially on that chip, but that's just the nature of how these things work sadly.

Best advice still holds true, back-up your important data to more than one device. I do not recommended those stupid consumer model pocket external HDDs to anyone though, they're junk. Use either desktop 3.5" HDDs or 2.5" SSDs either internally or via a USB HDD Docking station, and have a safe place to store the drive(s) after performing your back-up.

If you want to go the optical disc route, M-DISCs are the only real option. All other optical discs (DVD-R/+R/RAM and CD-R/RW) are extremely prone to a unique form of so-called "disc rot" from their chemical layers that the data is "burned" into degrading over time. In the worst of the worst I have had data discs suffer disc rot withing mere weeks of burning data onto them, while other discs have mysteriously managed to retain their integrity for over ten years now. Point is, it's very unpredictable how or when a writable disc will degrade. M-DISCS Are unique in that they were created for data preservation and are made differently, and also require a ODD that is marked as M-DISC ready as the writing laser uses a different intensity setting for writing data onto a M-DISC. M-DISCS Are not cheap however, when I was working at a electronics retailer just three years ago, the per disc cost averaged $5, but this may be a worthwhile price to the data you wish to preserve.

And your third option is data cassette back-up. Not the cheapest alternative anymore with falling SSD/Flash drive prices. Specifically for entry level DAT72 and DAT160 drives are the best choice as they can be purchased with a USB 2.0 interface, whereas the rest of the currently available stuff is SAS and Super SCSI interfaced, the sort of things for pro-grade servers. DAT72 Uses 72GB Data Cassettes, and DAT 160 uses 160GB Data Cassettes.

[always_summer]

As I have done for the last 5 years, I used a hard drive stored in a metal container to avoid problems with magnetic fields. A safety deposit box is the perfect place for such drives.
The short life expectancy of data readers is a major problem with digital storage.
For this reason, the Library of Congress does not consider any digital medium suitable for archival purposes. Physical negatives and physical prints will last (with the exception of inkjet prints which are more easily damaged).

[BarryTheCrab]

The more advanced the technology the more fragile it is.
Bury a clay tablet in the desert. Subject it to war, floods, cold, heat and break it in multiple fragments.
You can still get the data off that clay tablet.
Cut a VHS tape into pieces and give it environmental hell. With a little effort you can probably get the data from that tape.
Scratch a DVD and you’re screwed.

[lordsmurf]

Quote:

Originally Posted by OtakuSensei

Quality of consumer grade HDDs has dropped in the past five years

How so? The worst issues I've seen are how drives now require unique per-drive firmwares on attached chips. The portable 2.5" HDDs seem to have a slightly higher fail rate, mostly DOA immediately, or complete fail without any warning (no bad sector, nothing). However, I do find heat is the main fail point of the 2.5" HDDs, so keep those cool!

Quote:

Best advice still holds true, back-up your important data to more than one device.

Yep.

Quote:

I do not recommended those stupid consumer model pocket external HDDs to anyone though, they're junk.

Not really. As with anything else, brand and model matters.

Quote:

If you want to go the optical disc route, M-DISCs are the only real option. All other optical discs (DVD-R/+R/RAM and CD-R/RW) are extremely prone to a unique form of so-called "disc rot" from their chemical layers that the data is "burned" into degrading over time.

This is a mix of myth and marketing nonsense.

Their is no such thing as "disc rot" for CD/DVD/BD. That unofficial term (also "laser rot") was unique to the Laserdisc format, and involved oxidation and self-destruction of the foil layers. It had nothing to do with dye, dye-based media, or CD/DVD/BD. It cannot be extrapolated.

M-Disc is not a recommended DVD. Poor reflectivity, closer to phase-change RW.

Quote:

In the worst of the worst I have had data discs suffer disc rot withing mere weeks of burning data onto them, while other discs have mysteriously managed to retain their integrity for over ten years now.

Did you test the discs when burned? I don't mean sticking into a random player/reader, but PI/PO/PIE/PIF/jitter/etc testing (for DVD), BLER, etc. And are those datum available for analysis? Because, to date, every "my DVD died" claim is anecdotal, and lacks any degree of testing. You also cannot lump every DVD together. For example, a Princo DVD-R isn't a YUDEN DVD+R.

Quote:

Point is, it's very unpredictable how or when a writable disc will degrade.

Not really. If you take brand/model (and testing/verification of assumed quality of brand/model), adequate spot testing post-burn, and proper storage into account, as well as extrapolate current status of preceding formats (ie, CD for DVD, DVD for BD), then you can arrive at a decent forecast. I realize that takes testing and math, which most people won't do. But it can be done.

Quote:

M-DISCS Are unique in that they were created for data preservation

100% marketing horsepuckey.
The same BS existed for gold media, which also had lower reflectivity than standard silvery metals.

Quote:

and are made differently, and also require a ODD that is marked as M-DISC ready as the writing laser uses a different intensity setting for writing data onto a M-DISC.

So? DVD-R and DVD+R also need different lasers. LightScribe needed special something as well.

Quote:

M-DISCS Are not cheap however, when I was working at a electronics retailer just three years ago, the per disc cost averaged $5, but this may be a worthwhile price to the data you wish to preserve.

Overpriced. They rely on marketing to scare people into buying it. "archival" is a constant boogeyman term, and has been around for at least a century now. People are still as gullible as ever. I remember "archival" photo developer papers that were crap, while something like plain Ilford was still better. That translated heavily in the inkjet days, where "archival" was apparently just a term to describe how much you could fleece somebody at the office supply store. Tip: true "archival" products rarely carry that branding sticker. Competent archivists know to not be duped by BS.

Quote:

And your third option is data cassette back-up.

I have never liked backup tapes. Too fragile at the high speeds, and I saw far too many tapes ruinied (thus backups lost) in the 90s. Nope.

[OtakuSensei]

Quote:

Originally Posted by lordsmurf

How so? The worst issues I've seen are how drives now require unique per-drive firmwares on attached chips. The portable 2.5" HDDs seem to have a slightly higher fail rate, mostly DOA immediately, or complete fail without any warning (no bad sector, nothing). However, I do find heat is the main fail point of the 2.5" HDDs, so keep those cool!

Historically in context, 2.5" HDDs have always had a higher failure rate in comparison to 3.5" HDDs due to application of usage. WOrst thing you can do to a HDD is move it while it's operating, and these things were in mobile computers; laptops, early tablet PCs, vehicle mounted computers, etcetera. But moving beyond that, the manufacturers of the 2.5" HDDs in latter years literally got sloppy with the manufacturing of their 2.5" HDD offerings, excluding 2.5" highspeed (10k+ RPM) and server SCSI/SAS 2.5" HDDs because they acknowledged that they had a high fail rate due to "abuse", chiefly laptops getting bumped and dropped while running. I was in the PC business when all of the HDD manufacturers within a very short period of time cut the 2.5" HDD warranties to a mere one or two years from the previous 3 to 5 year warranties they'd all had prior. By no longer warranting their products for as long, it was severance from obligation to the consumer to put in the extra effort to ensure a quality product that could last as long was being delivered to the market.

The firmware chip you referred to on the modern drives actually simplifies data recovery for a data recovery professional should their services be needed as a physically identical board from a different HDD can be flashed with the firmware required by the drive that data needs to be retrieved from.

But going back to the catastrophic failure types being encountered in the modern drives, they're literally engineering and/or quality control failures. The R/W heads on the armature of a HDD should never separate from the armature and scrape the platter. The drive motor should never seize, overheat, and melt the platters. We should never open up a failed drive that in testing is mechanically sound but unable to read any data only to find aluminum flakes left over from the chasis machining process are still inside of the drive and have scratched the platters. Yes, finding DOA HDDs, and other components, has and will always be a issue, but I;d rather have DOA components that only create a temporary setback than poorly made functional equipment that suicide themselves and take a clients valuable data with them.

Quote:

Not really. As with anything else, brand and model matters.

For the 2.5" USB HDDs on market in present era, they're not worth buying. They have all of the above stated issues, plus custom interface boards with the USB directly on them. They're nightmares to do data recovery from as they were never made to interface via SATA. At least with the 2.5" USB HDD offerings of eight-plus years ago they were still off the shelf 2.5" SATA/IDE HDDs plugged into a USB adapter circuit board, but these modern drives are a whole other animal, and uniquely proprietary to each manufacturer. If you have your heart set on owning a 2.5" external USB drive go buy yourself a empty external USB HDD/SSD shell and put your own SATA drive, SSD preferably, in that so you know exactly what you've got.

Quote:

This is a mix of myth and marketing nonsense.

Their is no such thing as "disc rot" for CD/DVD/BD. That unofficial term (also "laser rot") was unique to the Laserdisc format, and involved oxidation and self-destruction of the foil layers. It had nothing to do with dye, dye-based media, or CD/DVD/BD. It cannot be extrapolated.

It is a problem with the chemical/dye-based media though. And the CD-R/RW DVD-R/+R/RW and even BD-R "burnable" media is all highly susceptible to data degradation as result. Is manner of care and storage a factor for this? Absolutely, but it's still a factor regardless. So yeah, I wasn't talking about the issues with LaserDiscs, that's a whole other kettle of fish.

Quote:

M-Disc is not a recommended DVD. Poor reflectivity, closer to phase-change RW.
Did you test the discs when burned? I don't mean sticking into a random player/reader, but PI/PO/PIE/PIF/jitter/etc testing (for DVD), BLER, etc. And are those datum available for analysis? Because, to date, every "my DVD died" claim is anecdotal, and lacks any degree of testing. You also cannot lump every DVD together. For example, a Princo DVD-R isn't a YUDEN DVD+R.

Maxell, Memorex, and store branded discs from Office Depot and Staples were the absolute worst I ever had the displeasure of using. Typically discs of those brands for me retained data no more than a year before they literally began to discolor and became unreadable. Verbatim, Imation, and TDK have yielded the best results for me, with some back-ups I made in the mid-2000's still readable presently. Philips I had mixed results with. most of the back-ups I did to Philips discs lasted for around five years before those discs began to fail.

Quote:

So? DVD-R and DVD+R also need different lasers. LightScribe needed special something as well.

LightScribe was a label side imaging technology for laser etching onto the disc a label. DVD-R/+R Don't use different lasers, just inverted reading/writing technology in the software side, with the +R technology being owned by a Sony-lead technology consortium that charged a licensing fee to any manufacture who wanted to use it, hence why it's uncommon.

Quote:

I have never liked backup tapes. Too fragile at the high speeds, and I saw far too many tapes ruined (thus backups lost) in the 90s. Nope.

I'm not a fan of tape simply because of the pricing in today's market makes it one of the most expensive, and slowest in terms R/W speed, options out there. It makes sense only in very select business environment applications, but is rather pointless for general use.

[lordsmurf]

Quote:

Originally Posted by OtakuSensei

Historically in context, 2.5" HDDs have always had a higher failure rate in comparison to 3.5" HDDs due to application of usage. WOrst thing you can do to a HDD is move it while it's operating, and these things were in mobile computers; laptops, early tablet PCs, vehicle mounted computers, etcetera. But moving beyond that, the manufacturers of the 2.5" HDDs in latter years literally got sloppy with the manufacturing of their 2.5" HDD offerings,

I agree with all of that.

Quote:

cut the 2.5" HDD warranties to a mere one or two years from the previous 3 to 5 year warranties they'd all had prior.
But going back to the catastrophic failure types being encountered in the modern drives, they're literally engineering and/or quality control failures.

This is often stated by users, but the real catalyst for shorter warranties was just cost savings. It had nothing to do with nefarious shortening the life of the HDDs. Do you invest? Dig into STX, WDC, or MU reports sometime, stuff like this in there, warranty costs. Cutting warranties is like cutting employees, just business. Now, I'd agree it was an anti-consumer change, it sucks for us users/buyers, but it wasn't nefarious, some plot to make more money by selling subpar HDDs that imploded earlier. That's just silly. The cheapening of components was just for the sake of cheapening components. Cuts were across the board, slash costs for more profits, appease investors (got to keep that STX dividend!).

Quote:

but I;d rather have DOA components that only create a temporary setback than poorly made functional equipment that suicide themselves and take a clients valuable data with them.

Yep.

Quote:

For the 2.5" USB HDDs on market in present era, they're not worth buying.

Again, cost cutting was across the board. 3.5" are affected as well.

Quote:

They're nightmares to do data recovery from as they were never made to interface via SATA.

Yep, that sucks, cannot do at-home/office recovery. Enclosure fails? You're just SOL. Been there, done that. (99%+ of it was backed up, so didn't freak out. Just annoyed, wasted my time.)

Quote:

It is a problem with the chemical/dye-based media though.

No. Did you know that some research has shown dye-based medias can actually outlast pressed media? But again, it really is based on exact materials. There is no "dye", but many formulas, from azoics to non-azoic inoganics to organics. Different blends of alloys. Details matter with blank media. Any media, optical or magnetic, or even a piece of paper (cotton rag vs. recycled tree, etc).

Quote:

highly susceptible to data degradation as result.

Again, all media.
Even a stone tablet isn't just a stone tablet. Limestone vs. clay.

Quote:

Maxell, Memorex, and store branded discs from Office Depot and Staples were the absolute worst I ever had the displeasure of using.

Maxell isn't necessarily MXL, but just a brand. The RITEKG05 Maxell were probably the most ghastly media ever under their label. While YUDEN was some of the best, even better than MXL. Most office brands, Memorex, was CMC, no shocker there, some of the worst discs ever made (but their were worse in Asia, also resold online in USA). Microcenter sold Umedisc, one of the crappiest discs ever made.

Quote:

Typically discs of those brands for me retained data no more than a year before they literally began to discolor and became unreadable.

There's really no truth to "disappearing" data. If bad, it was always bad or marginal. What more often change were the optical reader lasers, which have a high fail rate, and do go in 2-5 years. Not the discs, the lasers.

Quote:

Verbatim, Imation, and TDK have yielded the best results for me, with some back-ups I made in the mid-2000's still readable presently. Philips I had mixed results with. most of the back-ups I did to Philips discs lasted for around five years before those discs began to fail.

You really have to stop talking brands to understand optical media. All of those could be CMC, or all could be good discs like MCC/MKM, TDK, or RitekF.

Quote:

I'm not a fan of tape simply because of the pricing in today's market makes it one of the most expensive, and slowest in terms R/W speed, options out there. It makes sense only in very select business environment applications, but is rather pointless for general use.

I don't know of anybody that uses it anymore. Not governments, corporations, etc. I'm sure somebody does, because it apparently still sells, but no idea who it is. Slow and expensive, but also pro to damage. Wrinkled tape is not much different from using HDD as a basketball.