Recently, Western Digital announced its latest memory solution, OptiNAND, which combines solid-state drives with regular hard drives (HDD). What challenges do Flash and HDDs face, what does OptiNAND do, and how will it help with computing in the future?

What challenges do Flash and HDD memories face?

Hard drives are arguably the oldest form of memory storage that is still in widespread use today. Their origins date back to Christmas Eve 1954, when IBM first developed the hard disk, and their memory capacity has grown exponentially from a few hundred kilobytes to tens of terabytes. Not only have these memory storage units increased in capacity, but they have also reduced in size from large cabinets to small 2.5-inch drives.

Interestingly, hard disks and their internal components continue to decrease in size, which helps to pack more data on disks, but the physical size of hard drives has stopped shrinking.

Hard drives are excellent for long-term data storage as they are non-volatile (i.e. retain data when powered off) and can be written and read an unlimited number of times. Hard drives can still be found in ageing computer systems from the 90s whose data is still recoverable.

However, hard drives suffer from extremely slow read/write speeds compared to RAM, which means that disk operations are often a bottleneck for computers. This slow speed results from the heads needing to scan the disk and position themselves over the correct tracks to store whatever data is being read or written. This time is typically referred to as the “seek” time. Modern file systems may require reading tables and metadata before file data can be read, even when the head is correctly positioned.

Flash memory is a more recent technology dating back to the 90s and has been a major memory technology in microcontrollers, removable media, and mobile devices. Unlike hard drives, Flash memory is entirely electrical with no mechanical parts, which makes Flash memory significantly faster than HDDs. This high-speed is seeing Flash drives (Solid State Drives, or SSD) replace HDDs in high-performance computers to remove seek time and fasten file loading speeds.

However, Flash memory suffers from a significant drawback; write wear. Simply put, writing data to a flash memory cell is somewhat violent, with electrons being forced to jump to a floating gate, and this writing partly damages the semiconductor structure. Thus, writing too many times to the same memory cell will destroy its ability to work, reducing the amount of information that can be stored on the memory device.

Western Digital announce OptiNAND

Modern computer systems with performance in mind will often have two separate drives; an SSD to store OS and other program files and an HDD to store user files. This combination allows the OS and program files to load quickly, while the HDD provides ample storage space that is re-writable.

However, even this combination has its drawbacks, and files frequently read from the HDD may not operate at peak performance. Furthermore, file metadata can often be many gigabytes in size which may be required to read files from an HDD (such as track number etc.) or perform complex data analysis (such as search).

As such, Western Digital recently announced their new memory technology called OptiNAND, which combines an HDD and SSD into a single device. Instead of acting as two separate memory units, the NAND flash acts as a cache for frequently accessed files and storing metadata on files stored on the HDD. This allows access to files on the HDD at a much faster rate as metadata is faster to read while also allowing for faster data organisation operations.

Furthermore, their HDDs combine multiple technologies that help boost read speeds, such as triple-stage actuators (allowing for immediate track jumps by adjusting the position of the head tip) and provide better data protection (i.e. via a sealed helium atmosphere).

How will such memory technologies help drive the future of computing?

The amount of data stored on modern computers is incredible, but the amount of metadata produced around that data is even more staggering. Metadata is extremely useful when trying to organise data and understand what that data is without having to analyse it.

One example of where OptiNAND technology could be beneficial is big data and how it works with privacy. It is almost universal that people do not want their data being observed by either a computer or a human, but for AI to operate, they need access to some kind of data. However, this data doesn’t always have to be the exact data that is being analysed. This data can be metadata (such as data descriptors, fingerprints, and hashes).

Thus, a privacy-focused data processing centre may store raw data on an HDD but store resulting metadata on high-speed flash memory. Therefore, an algorithm that produces metadata could read through raw data found on the HDD and then store this resulting metadata on flash. From there, a high-speed AI algorithm (whose performance is dependent on how fast it can access data) could operate from metadata found on the flash.

Overall, separating files from metadata could be highly beneficial in reducing access time to key data, and combining NAND flash with HDDs could create highly responsive memory drives that are often the bottleneck in computer systems.