In the past few years, I made a lot of experiments, to find out is there any place for Storage hardware, or we were able to use only pure software defined storage solutions.
My feelings telling that, now we are on that evolution level, where we are able to run a complete datacenter on software defined storage.
The solutions can be chosen depending on the target area, which we want to serve. The main separate usage areas can be the following:
Low latency, high performance storage (32TB, 2.000.000 IOPS per node @ 0.7 ms)
General usage, storage for application servers (22,5TB, 500.000 IOPS per node @ 1ms)
Archive storage with high availability, and high capacity (432TB, 10.800 IOPS per node @ 5ms)
For low latency storage I would expect NVMe or PCIe SSD disks/cards, shared on Infiniband network, with minimal latency.
For general storage, I would prefer traditional SAS/SATA SSD drives, for homogenous environment also with Infiniband interconnect.
For archive we could use High capacity SAS/SATA rotational disks in 6-8TB size each.
On compute node level, we are able to transform the shared disks like we want. It is a similar process like JBOD drives. But we more easily able to swap drives in case of errors, from the shared disk pools.
What we are building is depending on the compute node side. For Hyper-V/MSSQL/File servers we can utilize the MS Scaleout File Server, for Oracle RDBMS we could use the ASM feature of Grid Infrastructure, for virtual environments, on Windows we could use the SOFS again, for VMware we can use VSAN product of them, for KVM Linux virtualization we can utilize CEPH, with the new generation disk backends.
This is the current situation. What I expect in the future?
Software defined layer will move to specialized purpose equipments, what will only serve the basic needs. In the future, I expect specialized nodes, which will do the same like Software Defined layer, but with specialized ASIC or FPGA modules. After the standardization of the basic elements complete, no need for emulation. This is the same question like Virtualize, or not.
Specialized ASIC can decrease environmental footprint. As the data which we are processing growth exponentially, this question will be much important in the future.