Been meaning to send this around to the group. This is a DIY system that is
currently in production at Johns Hopkins School of Public Health. It is
used primarily for archival storage. See url below for pdf of (mostly)
non-technical details.
Link: http://www.jhpce.jhu.edu/wp-content/uploads/2014/04/greenDirtCheapStorage.pdf
The salient characteristics are:
1PB raw (680TB raidz2)
vdevs striped across JBODs
4 x250TB pools.
360 x (3TB Western Digital Red Drives)
Only 3 failed disks in 9 months (i.e. failure rates comparable to
"enterprise" drives).
3.5kW average (~3.5W/raw-TB).
cost $117K to build + our time to design it.
ZFS-on-LInux
File systems on the device are NFS exported to a 60-node, 2000-core cluster.
This system could now be built for well under $100K or well under $100/TB.
There is a huge pent-up demand for this class of device., but no vendors
*currently* provide systems in this class. But maybe we can encourage them
to step up to the table.
Thanks,
-- Fernando
currently in production at Johns Hopkins School of Public Health. It is
used primarily for archival storage. See url below for pdf of (mostly)
non-technical details.
Link: http://www.jhpce.jhu.edu/wp-content/uploads/2014/04/greenDirtCheapStorage.pdf
The salient characteristics are:
1PB raw (680TB raidz2)
vdevs striped across JBODs
4 x250TB pools.
360 x (3TB Western Digital Red Drives)
Only 3 failed disks in 9 months (i.e. failure rates comparable to
"enterprise" drives).
3.5kW average (~3.5W/raw-TB).
cost $117K to build + our time to design it.
ZFS-on-LInux
File systems on the device are NFS exported to a 60-node, 2000-core cluster.
This system could now be built for well under $100K or well under $100/TB.
There is a huge pent-up demand for this class of device., but no vendors
*currently* provide systems in this class. But maybe we can encourage them
to step up to the table.
Thanks,
-- Fernando