We once again return to discuss the latest Storage Performance Council (SPC) results*.
SPC-1*results
There have been only five new SPC-1 results these past quarter, one for IBM DS5300, one for 3PAR InServ F400, and three for HDS AMS2500, AMS2300, and AMS2100. None of these made the top 10 in IOPS™ or $/IOPS™ so these charts can be found in prior dispatches#. However, two of these products did make the top 10 for IOPS/$/GB (see Figure 1 below).
(SCISPC090527-001) (c) 2009 Silverton Consulting, All Rights Reserved
The two new subsystems here are the 3PAR InServ F400 and the HDS AMS2500, both of which would be considered mid-class storage subsystems. We calculate this metric as = IOPS / ($/GB) and created this metric as another way to factor in performance against cost and capacity. Although none of these metrics can tell the complete story.
(SCISPC090527-002) (c) 2009 Silverton Consulting, All Rights Reserved
As for response time, the latest IBM DS5300 managed to break into the top 10 at an average least response time (LRT) of 1.77 msec. I have been told this metric is not as important anymore but given all the interest in SSDs these days I find that hard to reconcile.
(SCISPC090527-003) (c) 2009 Silverton Consulting, All Rights Reserved
We have redone the IO operations/second/Drive (IOPS/Drv) chart so that it now only include drives over 140GB as the smaller drives held too great an advantage here. All of the latest benchmark results show up in this Top 10 IOPS/Drv with HDS holding the number 4, 7, and 10 spots, IBM DS5300 at number 8, and 3PAR F400 at number 9. Also, as mentioned in a prior report, both the J4400 and J4200 results had no RAID protection so may not be suitable comparisons for normal customer environments.
Previously this chart had an error for the ETERNUS2000 that caused us to report double its actual IOPS/Drv rate. We have fixed that error and it now shows the correct IOPS/Drive for the ETERNUS2000.
(SCISPC090527-004) (c) 2009 Silverton Consulting, All Rights Reserved
Figure 4 SPC-1 Bubble chart of IOPS against LRT, bubble size proportional to subsystem price
Always looking for an interesting cut on SPC-1 data we came up with this new bubble chart. It shows a scatter plot view of subsystem performance with the x-axis as LRT and the y-axis as IOPS. The one thing always missing from a pure performance analysis is subsystem cost, added here as bubble size. To keep this interesting we capped the subsystems shown on the chart to a $100/GB maximum to eliminate the higher priced subsystems.
From an end-user perspective it’s interesting to note that one can obtain a reasonably performing subsystem (~100K IOPS with LRT <2msec) for about 1/4th or less the price of truly high performance subsystem. Also, similarly priced or even more expensive subsystems can have much worse performance on an IOPS and/or a LRT basis indicating that pricing isn’t always the best factor in subsystem selection.
SPC-2, SPC-1C and SPC-2C results
There were no new SPC-2, SPC-1C and SPC-2C benchmarks released for this update and as such we stand with our last report SPC StorInt Dispatch&.
Significance
As we show above midrange subsystems can be high performers when configured properly. Although higher end subsystems dominate the Top 10 IOPS chart, the midrange subsystems discussed here have all managed to do well on most of the other performance metrics.
Over the years we have tried to come up with ways to compare performance to price and have used both $/IOPS and IOPS/$/GB (and others on occasion) as attempts to further this analysis. In the end we find that both of these metrics, although interesting on their own, leaves something out. The new bubble chart (see Figure 4 above) is our latest attempt to incorporate pricing information with subsystem performance. Hopefully, the reader will view this as a worthy addition to our SPC analysis. As always we welcome your feedback on how to do this better.
This performance dispatch was sent out to our newsletter subscribers in May of 2009. If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports. Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.
A PDF version of this can be found at
SCI 2009 May 27 Update to SPC benchmark resultsSilverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community
* All results from www.storageperformance.org as of 23 February 2009
# Prior SPC performance dispatches can be found at http://www.silvertonconsulting.com/page2/page2d/storage_int_dispatch.html
& Available at http://www.silvertonconsulting.com/page2/page2d/storage_int_dispatch.html