NAND flash solid state technology has been around for many years. However, it has only been in more recent years that NAND flash has seen a meteoric rise in popularity in enhancing and accelerating the performance of storage systems.

However, flash is not a panacea for all performance issues in a datacenter. In fact, Neuralytix argues that the simple introduction of flash memory into an environment is insufficient to result in a justifiable return on the investment (ROI). Flash memory needs to be complemented with hardware and, in particular, software. The software will aide in the proper distribution of I/O’s and proper placement of data to provide the desired return.

Over the last couple of decades, many independent studies have been performed to evaluate the amount of data within a datacenter that organizations would consider “hot”, or requiring ultra-high performance. These studies have indicated that on average roughly 5% of total active data (i.e. data that is not for backup, replication or archive) are considered “hot” and would benefit from the use of the highest performance persistent data storage technology possible.

Therefore, in theory, the long term available market for flash memory systems should be limited by roughly 5% of the capacity of traditional storage systems, as this number represents the proportion of hot or active data typically found in the datacenter. After the saturation of the market, the growth of the market should grow (or decline) in direct relation to the rise and fall of global business output since the business output will determine the growth of active datasets.

However, Neuralytix believes that this approach is too simplistic. The availability of all-flash arrays, plus the addition of server attached flash systems will extend the available market significantly[1]. Neuralytix does not believe that flash will become the primary medium for mass data storage.

Where does flash integrate?

Today, flash can be found in multiple integration points along the data path: from the host server to traditional storage systems. In fact, Neuralytix recognizes six distinct form factors for flash to be integrated into a storage network. These are:

• As a PCIe connected expansion board inside a server;
• As an array of solid state storage that does not offer advanced data management services;
• As an inline storage network cache;
• As an all-flash storage systems that includes advanced data management services;
• As cache extension for storage system controllers; and
• In a disk drive packaging as replacement for traditional magnetic rotating hard disk drives (HDDs).

It is critical to note that none of these form factors are unto itself a panacea to the performance or capacity needs for any given organization. Each form factor has specific benefits for a given environment.

The six form factors can be categorized into two distinctive groups:

• Solid state storage “north” of the storage network; and
• Solid state storage “south” of the storage network.

These two groups are pictorially depicted in Figures 1 and 2 respectively.

Neuralytix has determined that the use of “north” and “south” of the storage network as the most appropriate description of where flash can be integrated since traditionally, storage network diagrams have typically depicted servers above a storage network, and storage systems below the storage network.

Solid state storage “north” of the storage network are the PCIe and solid state storage devices without data management services, while solid state storage “south” of the storage network are the remaining four form factors listed above.

Typically, files that fall into this category have been noted as:

• Log files;
• Database indices;
• Cache files;
• Temporary files; and
• Frequently accessed static data (such as an HTML web page).

Not all vendors have the same set of recommendations. For example, EMC does not recommend the use of flash drives for storing log files as it is inconsistent with the recommendations from software vendors like Oracle.

In recent years, with the emergence of Big Data, a rise in the popularity of in-memory databases has seen flash storage used to significantly increase server main memory cost effectively.

It’s all about performance

Flash based storage is simply about performance. While there are some solid state storage systems (SSSS) vendors who promote their solutions as an alternative to tier-1 storage, at least in the short term, for most organizations, the use of flash based storage is to gain performance that would cost roughly an order or magnitude less, while providing performance benefits that would be an order of magnitude better than traditional solutions.

However, these costs are simply acquisition costs, and do not take into consideration maintenance and support. Neuralytix estimates that one extreme, the savings from replacing traditional high revolution HDDs with solid state storage to match a given performance level would reduce the total cost of ownership by a factor of over 80%.

Solid state storage is about performance; it is rarely about capacity. Too often, when calculating the business value of integrating solid state storage into an existing environment, the cost per unit of storage capacity (denoted as $/GB) is misused as the appropriate metric for justifying solid state storage. Instead, organizations need to look at the cost per unit of performance (denoted as $/IO).

From a capacity perspective, where $/GB is the de facto standard metric, the lowest cost primary storage is still Serial-ATA

(SATA) drives. SATA drives can cost 10 times less than a capacity comparable solid state solution. But, solid state solutions will typically deliver over 100 times the performance compared to SATA drives.

In many cases, the integration of some flash, married with a vast majority of SAS or SATA drives will achieve the optimal $/GB and $/IO balance.

“North” of the storage network

Solid state storage solutions that are “north” of the storage network is really about addressing the performance needs of the host application, and not necessarily the performance benefits of an entire storage network.

Generally, solid state storage “north” of the storage network does not have data management services such as advanced data protection features (e.g. RAID, replication, etc.).

PCIe Form Factor

The PCIe form factor provides a server with the opportunity to use significantly lower cost flash memory as opposed to DRAM to provide additional memory. It is the most common form factor for solid state storage “north” of the storage network.

The flash on the PCIe form factor are often presented to the operating system as a block device. A further device driver can present it as an extension of main memory. Other uses could include pinning specific files such as log files, indices, and other “hot” files to the available storage capacity offered on the PCI flash card.

The capacity on the PCI flash card could also be configured to page main memory, thereby bypassing the disk I/O system, and thus improve the efficacy of the server.

Other uses for PCIe flash cards include using it in conjunction with a traditional storage system and acting as a massive read cache for one physical server. Additional software is needed to make this possible, and in some cases virtual machines will also require additional drivers in the guest OS as well. When operating with traditional storage systems, the storage systems can be configured to be accepted as a write cache (assuming that all the servers connected to the storage system has a write cache). Therefore, each cache provides either read or write caching; eliminating the dynamic allocation of cache between reading and writing.

As specific application demands increase, and (resulting from virtualization technologies), the number of virtual machines concentrated on a single physical server increases, the need for server attached flash will increase. The size of the PCI based flash solutions will be limited to the availability of physical space inside servers.

Applications including Oracle RDBMS, SAP (especially HANA), Microsoft Exchange and SQL analytics platforms and virtualized environments (both server and desktop virtualization) look to improve performance by executing as much of the workload as possible in-memory. A prime example of this is found with VMware’s virtual desktop infrastructure (VDI). As a single physical server hosts more and more virtual desktops, the I/O needs of the server have a multiplicative impact. Serving as many of the I/O’s as possible from the server, as opposed to externally attached storage will help to improve response time, and reduce latency. This also frees up available bandwidth in the storage network for other processes and workloads that require data from disk subsystems.

Figure 1: Solid State Storage “north” of the Storage Network

FIGURE 1

There are two places “north” of the storage network in which solid state storage are deployed. Flash can be embedded in a PCIe card. It can also be implemented as a stand-alone, directly attached array of flash (SSSD)

Source: Neuralytix, 2012

Vendors[2] providing PCI based flash solutions

• EMC with its VFCache;
• Fusion-io with its ioDrive solution;
• Intel with its SSD 910-series solution;
• LSI with its WarpDrive solution;
• Micron with its P320h solution;
• OCZ with its RevoDrive and Z-Drive solutions;
• Oracle/Sun with its F20 PCIe card; and
• Texas Memory Systems with its RamSan 10, 20 70 & 80 solutions.

Of the vendors in this space, Fusion-io is probably the most recognized, as a result of their technology leadership, and successful initial public offering (IPO). Fusion-io solutions are sold directly as well as through leading server vendors Dell, HP and IBM. Most recently, NetApp announced that it will be reselling Fusion-io products.

Solid State Storage Devices (SSSD)

The solid state storage devices (SSSD) market is a relatively new one, even for the relatively nascent flash based storage market.

Essentially, it is the extension of the PCI-based flash concept outside of the physical server enclosure. This concept enables more flash to be directly attached to a given server (or limited number of servers).

A key differentiator between SSSDs and solid state storage systems (SSSS) is whether or not the solution integrates data services such as snapshots, deduplication, thin-provisioning, data protection technologies (such as RAID or a variant), replication, etc. This key differentiation is an important delineator as it separates whether the product is a large array of flash to extend capacities beyond what PCI-based flash can offer, or whether the product is designed with the possibility of replacing traditional storage systems.

Although SSSDs are direct connect devices, those SSSDs are likely to offer connections to multiple servers. This does not make them storage systems per se, only that the capacity can be shared, as storage systems require data management services.

Thus, this is the reason why Texas Memory Systems’ RamSan 630, 640, 710, 720, 810, 820, and 6300 products are SSSDs and not SSSSs. In our opinion, the inclusion of RAID technology is insufficient to be considered as advanced data management services.

FusionIO has recently taken an interest step towards a shared cache appliance. By combining its ION data accelerator software in an industry standard x86 server populated with FusionIO PCI cards, FusionIO is able to create a shared solid state storage array. This solution is analogous to a SSSD. However, since the solution requires an end-user to acquire and assemble the piece parts, Neuralytix does not recognize this solution as a SSSD.

Vendors providing SSSD

• EMC with its announced (but as yet unavailable) Thunder solution;
• Texas Memory Systems (TMS)[3] with its RamSan 630, 640, 710, 720, 810, 820, 6300; and
• Violin Memory with its 3100 series and 3200 series solutions.

“South” of the storage network

On the other side of the storage network, are solutions that are likely to be more recognizable. There are also significantly more vendors “south” of the storage network border. Neuralytix recognizes roughly 15 vendors, most of which are start-ups.

There are four points of integration “south” of the storage network. Each integration point has a varying degree of complexity and sophistication. These four integration points are depicted below in Figure 2:

Figure 2: Solid State Storage “south” of the Storage Network

FIGURE 2

There are four places “south” of the storage network where solid state can be deployed. As a cache along the network data path As all flash networked storage system (SSSS) NAND flash as cache for the storage controller Solid state disks (SSD) in a traditional storage system

Source: Neuralytix, 2012

Solid State Storage Systems (SSSS)

Solid State Storage Systems (SSSS) is arguably the most hotly contested segment of the solid state market. The vendors in this market look to usurp the traditional storage system with all-flash arrays. The argument made consistently by all vendors is that the performance of flash over HDDs is not questionable. It will perform significantly better.

They further argue that through the integration of cost effective multi layer cell (MLC) flash technology and intelligent data efficiency technologies (such as deduplication or compression), that the cost of a SSSS is comparable to traditional storage systems.

As noted earlier, SSSS differ from SSSD as they integrate a data management platform that mirror those found in traditional storage systems. These include (but are not limited to) snapshot technology, replication technology, high availability fail-over storage controllers. In some cases, vendors have also provided common network file systems protocols including NFS and CIFS.

All flash based storage vendors argue that the environmental benefits (that translate directly into the reduction in total cost of ownership (TCO) are critical factors to consider. Neuralytix concurs with this assertion; although, in many cases, the most notable reduction in TCO is physical footprint. One of the vendors in this space claims that it has helped a major financial services client reduce footprint by a factor of over 20 to one.

Vendors providing solid state storage systems (SSSS)[4]:

• EMC with its as yet unavailable XtremIO solution;
• GridIron Systems with its OneAppliance FlashCube solution;
• Kaminario with its K2 solution;
• Nimbus Data Systems with its S-series and Gemini solutions;
• Oracle/Sun with its F5100 solution;
• Pure Storage with its FlashArray solution;
• SolidFire with its SolidFire storage system;
• Skyera with its Skyhawk solution;
• Violin Memory with its 6000-series solution; and
• WhipTail with its Accela and Invicta solutions.

As can be seen, there are at least ten notable vendors in this space that Neuralytix recognizes for the purposes of this report.

Of all the flash solutions, the SSSS vendors are likely to produce the most disruption to the storage systems market. They will attempt to challenge traditional beliefs that HDD technology is required and is the most economical approach to mass data storage. These vendors have the best opportunities in industry-specific applications, and the mid-market.

Mid-market end-users are known for their cost-sensitivity. They are less known for their reluctance to do technology refreshes. Given that flash technologies represent a next generation in storage systems, many will be attracted by the relative cost differences between traditional storage systems and SSSS, as well as with the promised performance capabilities of SSSS. Intelligent Solid State Caching Appliances (SSCA)

Perhaps the singularly most under-recognized solution in the flash storage market is the solid state caching appliance (SSCA). These appliances sit in-band on the storage network and cache read and/or writes I/O’s at the network level. The appliances have specialized operating systems.

These solutions serve to provide minimal disruption to existing environments, while still providing the performance benefits of solid state storage technologies.

Vendors providing Solid State Caching Appliances (SSCA):

• Alacritech with its ANX1500 solution;
• Astute Networks with its ViSX solution;
• Avere Systems with its FXT solutions;
• CacheIQ with its RapidCache solution;
• Dataram with its XcelaSAN solution;
• GridIron with its OneAppliance TurboCharger family of solutions; and
• Violin Memory with its vCACHE solution.

These vendors differentiate themselves by the type of caching it provides. Some provide SAN caching, while others provide NAS caching.

Cache Extension for Storage Controllers

There are only two competitors in this category: IBM and NetApp. Each takes a completely different approach.

IBM, in its XIV storage system uniquely uses SSDs to extend the cache memory in the XIV controllers. The SSDs are not a tier, and not available as part of the general storage pool. This gives IBM a very flexible design since IBM can take advantage of higher capacity SSDs as they become available; while maintaining an excellent price:performance ratio as it exclusively uses serial-attached SCSI (SAS) nearly drives.

Using SSDs also give IBM the flexibility of altering and optimizing the “hot” active data to mass storage capacity ratio. This design also removes IBM’s ties to manufacturing or outdated engineering.

NetApp is unique in its use of flash memory to extend its storage controller with flash memory. A FlashCache card can be inserted into the NetApp storage controller that extends the available cache past the standard configuration.

In many instances, to gain an improvement in overall storage systems performance, this simple addition can be sufficient. The solution allows end-users to extend the life of NetApp storage by giving it a mid-life “shot in the arm” of computer adrenaline in the form of flash memory.

Some argue that EMC’s FASTcache is a competitor against NetApp’s FlashCache, although Neuralytix does not agree. EMC’s FASTcache, available on its VNX family of storage systems, integrates advanced tiering software (FAST) with SSDs that are integrated into the hybrid storage systems. NetApp’s FlashCache on the other hand, is a separate and distinct hardware modules for NetApp’s controllers

Vendor providing cache extension for storage controllers:

• IBM with its SSD caching in XIV
• NetApp with its FlashCache.

Solid State Disk Drives (SSDs)

The simplest way of introducing flash memory into storage is simply by adding or replacing HDDs with solid state disks (SSDs). Simply put, SSDs are flash memory that have been housed in a standard HDD physical casing and intelligence has been integrated to make flash memory act and be presented just like an HDD.

Although EMC must be recognized as the first to introduce SSDs into traditional storage system, essentially making its arrays a hybrid storage system, almost all traditional storage systems vendors now have options to integrate SSDs into their storage systems.

Vendors providing Solid State Disks (SSDs):

• Intel
• OCZ
• Samsung
• SanDisk
• Seagate
• STEC
• Western Digital

When it is all said and done, software is key. Whether it is a firmware upgrade for a storage controller, as in the case of NetApp’s FlashCache, or the integration of an entire storage operating system found in SSSS, software is ultimately what will make or break flash memory.

Even in traditional and hybrid storage systems, the need for auto-tiering software makes SSDs more than just another disk medium. The tiering software brings value through automation.

In the case of solid state solutions “north” of the storage network, software will help ensure that the right I/O’s land on the flash, and the inappropriate I/O’s write through to the backend traditional storage.

NetApp is noted above with its update to its Data ONTAP operating system on its storage controllers to take advantage of FlashCache. Commendation must also be extended to Fusion-io (through its acquisition of ioTurbine) and EMC with its combination of VFCache and FAST.

In the virtual environment, the virtual environment, accommodations must be made to override the hypervisor’s management of I/O’s that can ultimately be an interleaved set of I/O’s originating and destined for any number of virtual machines (VMs). Fusion-io had the foresight to see that, and through its acquisition of ioTurbine, was able to provide software that bypasses the hypervisor disk management, and allow VMs to write directly to its PCIe-based flash devices. Ultimately, this takes away additional layers of abstraction, complexity and latency. It also gives the end-user more control on how the resource is used and deployed.

EMC, with a more comprehensive set of flash offerings (both currently available and announced), saw the opportunity to leverage its PowerPath software in a similar manner to Fusion-io. Since EMC also had traditional and hybrid storage arrays the VFCache, it is able to intelligently manage those I/O’s that could and should be cached at the server level. Those that can be cached on solid-state only volumes, and those I/O’s that should be auto-tiered to lower cost storage.

IBM may have entered later than its competitors, but upon completion and integration of its announced acquisition of Texas Memory Systems (TMS), IBM’s portfolio of solid state solutions will measure up competitively against major rival EMC. IBM announced delivery of its next generation EasyTier software that will allow it to extend the functionality that integrates with PCIe and SSSD solutions (TMS’ RamSan solutions). To demonstrate its commitment to these solutions, it demonstrated the feature at its recent IBM Edge event in June.

Recognition should also be given to Violin Memory for its continued development of software solutions with its OEM and server systems partners.

With the majority of solutions that are marketed being hybrid solutions, either as a tier (automated or otherwise), or as additional performance enhancement point solutions for specific applications, management software will also be a critical element in terms of which vendors end-users will ultimately end up choosing.

For example, it is less likely for an end-user to integrate storage solutions from three different vendors (such as the incumbent storage systems vendor, plus a server based flash vendor, augmented and managed by a third automated tiering and data management vendor).

Neuralytix reiterates its position that vendors must invest in solutions that cover multiple integration points along the solid state spectrum – all flash, hybrid, server based and storage systems based.

Table 1: Competitive Matrix of Flash-Based Storage Solutions by Vendor

TABLE 1

Vendor PCIe SSSD SSSS Storage Network Cache Cache in Storage Controller SSD Alacritech X Astute Networkrs X Avere Systems X CacheIQ X Dataram X EMC X x* x* Fusion-io X GridIron X X IBM X Intel X X Kaminario X NetApp X Nimbus X OCZ X X Oracle/Sun X X Pure Storage X Samsung X SanDisk X Seagate X Skyera X SolidFire X STEC X Texas Memory Systems (TMS) X X Violin Memory X X X Western Digital X WhipTail X Total Vendors 7 3 10 6 2 7

Source: Neuralytix, 2012

Notes: Those denoted as x^* means that product is announced, but not yet shipping

Neuralytix has stated previously that it believes that flash storage is going to have a similar impact on the storage market as technologies such as RAID and SATA.

All end-users should integrate flash-based storage technologies into their environments, but only after careful consideration and consultation with their vendors and resellers regarding the most optimized placement and deployment of flash.

Start-up flash vendors are not going to become leading storage systems vendors overnight. There are many enhancements that will continue to evolve as it relates to solid state storage, in particular around the memory technology.

Do not expect solid state storage capacity pricing to match HDD pricing. The ultra high areal-density found in today’s 3.5” HDD technology (those HDDs that have a capacity at or above 2TB) will still make HDDs the most economical random access storage medium for a very long time.

For end-users and vendors alike, although this report specifically excludes those solutions that are capable of supporting hybrid SSD and HDD solutions, Neuralytix believes that these solutions will be the dominant go-to-market form factor at least for the next three to five years.

Companies including (but not exclusively) Dell, EMC, HDS, HP, IBM, NetApp have spent significant development funds in enabling their traditional storage systems to be able to support and benefit from the inclusion of solid state storage.

Some companies, such as HP have even gone to the extent of marketing variations of its traditional storage systems with solid state only configurations. This is a good way for these companies to enamor the installed base to adopt all-flash solutions.

Winners and Losers

The most obvious winners are the flash memory chip vendors – Intel, Micron, Samsung and Toshiba in particular.

The speculators who believe flash technology means the death of HDD are grossly incorrect to the point of demonstrating acute short-sightedness. In the short term, the HDD vendors will see a slowing in demand for high performance disk drives with a shift over the longer to towards high areal-density HDDs. Neuralytix believes that in the long term, the demand for HDD capacities may return close to historic levels as through the need for reliable data access resulting from data processing output, Big Data and compliance activities.

The biggest winners will be vendors who provide solutions at various integration points and complementary software solutions to take advantage of these solutions for the multitude of various workloads and I/O demands.

Amongst them are EMC, Violin Memory and IBM. Other vendors include NetApp through its partnership with Fusion-io, HP, through its partnership with Violin Memory and Oracle. Oracle is distinct in that although they have multiple solutions addressing multiple integration points, its solutions are purely designed to optimize the Oracle RDBMS. While no one is likely to argue over Oracle’s leadership position in the RDBMS and transaction processing workloads, there are many new and emerging workloads that demand very high I/O performance and leverage non-relational database technologies. These include analytical, Big Data and high-performance computing environments.

IBM is likely to be the dark horse in this race. Its decision to acquire TMS plus its leadership in the development of next generation non-volatile, commercially viable memory solutions including Phase Change Memory (PCM) is likely to put them in a different class altogether.

The biggest losers in this competition are those vendors that do not provide solutions in multiple solutions that address a variety of integration points and those that are unable to develop (or partner to develop) software solutions that can optimize their solutions for key applications and workloads.

This whole market is still very nascent. The market should expect continued merger and acquisition (M&A) activity over the next 36 months as major vendors take the decision to buy over build.

Neuralytix believes that the winning integration points will be the PCIe card, and a variation of the SSSD. Next generation SSSD’s are likely to be more aligned to SSSS. A combination of data management software, solid state storage management software, and capacity sharing using an industry standard x86 servers with PCIe based flash is most likely to be the next generation of SSSS, with SSSDs being a more exclusive use case.

In terms of integration points “south” of the storage network, expect to see a continued development and variation on the hybrid storage system and storage network caching.

Taking a very long term speculative view (say, towards 2020), Neuralytix believes that there will ultimately be only three integration points – the in-server integration point (“north” of the storage network); the hybrid storage system (“south” of the storage network); and the highly intelligent, highly cached network itself.

Neuralytix believes that technology should drive business value. The business value assessment of solid state storage is best viewed in terms of output and productivity.

From a revenue perspective, end-users should forecast and budget for top line stability or growth as the primary ROI of investing in solid state technologies. From a productivity perspective, end-users cannot mistake performance for capacity. Combining these two factors, organizations should take both the top line and bottom lines into account when investing in new technologies such as solid state storage.

Smaller organizations may be attracted by the simplicity and performance characteristics of SSSS; but if a SSSS does not decrease the required capacity investment with a corresponding improvement to the top line, then this would be a poor investment.

Organizations should also consider the long term opportunities for integrating solid state storage. Relying on a key tenet of Neuralytix’s beliefs as it relates to Big Data, “if you’re not doing it, your competitors are™”. Ask what you can do with your data, and how these and new processes can provide a competitive advantage for your organization.

The vendors and solutions mentioned in this report have been selected using a Neuralytix proprietary methodology. Factors including financial stability and significance were the primary factors taken into account. For those vendors that are pre-revenue or currently have insignificant relative revenue, factors such as marketing and communications prominence were evaluated.

In this report, Neuralytix takes the “branded view” approach, consistent with market research firms. This means that Neuralytix recognizes the last branding placed on a specific product. So if vendor “X” with product “A” is sold as vendor “Y” with product “B”, Neuralytix will only recognize vendor “Y” and product “B”. However, if the same vendor “X” and product “A” were simply resold, without any change in branding, then Neuralytix will recognize vendor “X” and product “A”.

Neuralytix recognizes only the brands at the date of publishing (August 31, 2012). For this reason, and this reason alone, IBM and TMS are recognized separately as the acquisition has not yet been completed.

Only solutions that are designed exclusively for solid state storage technologies were included in this report.