The World's First High-speed Secret Sharing Engine for OpenStack Swift ～Developing durable/volumetric effective storage system, reducing risk of information leakage with highest speed～

1.Background

Solutions for security, data durability, and increasing data volume are more important than ever in IT business market. To achieve both high data durability and decreasing storage volume, "erasure code" technology has recently been applied to distributed storage systems as well as OpenStack Swift. In the erasure code technology, a storage system results in both volumetric efficiency and ensuring high durability by storing encoded data transformed from in-coming original data. This mechanism reduces the storing data volume by half compared with the traditional "replication" strategy.
The SHSS engine was developed as a pluggable engine for the OpenStack Swift erasure code framework, with the expectation of developing a secure distributed storage system.

2.Key Points of Technology

(1)OpenStack Swift Integration

The SHSS engine enables OpenStack Swift to both split in-coming plain data into secure fragments and reconstruct out-going plain data from the fragments. The reconstruction requires a number of fragments, and attackers obtain no information from fewer fragments. This mechanism decreases the information leakage risk of replacing broken physical drives by hard drive vendors.
Furthermore, SHSS requires no encryption key management, which is the problem in many cases, since secret sharing technology does not need keys for fragmentation and reconstruction.
The SHSS engine is compatible with the OpenStack Swift erasure code feature released at Kilo Integration in April 2015. Therefore, system constructors can build a secure storage system using OpenStack Swift and SHSS with high durability and volumetric efficiency as well as using OpenStack Swift and erasure code.

(2)World's Fastest Secret Sharing Engine

The main advantage of this technology is the highest speed for fragmentation and reconstruction, which allows OpenStack Swift to quickly store/retrieve files.
Previously, secret sharing processing for fragmentation and reconstruction was much slower than erasure code's encoding and decoding; therefore, it was difficult to apply secret sharing to storage systems. To improve performance, NTT has developed a new high-performance 64-bit processing, which is faster than the 8-bit processing used in previous mechanisms. It increases processing, so that SHSS can fragmentize/reconstruct by about 22 Gbps in the case of 24 fragments total and 20 fragments required for reconstruction. It is 50 times faster than the previous fastest firm's secret sharing engine, AONT-RS secure ^[2]. Combined with OpenStack Swift, SHSS's fragmentation and reconstruction perform about 10 Gbps, which is as almost the same as the performance of Jerasure ^[3], that is, OpenStack Swift's standard erasure code engine.

NTT is now able to construct highly secure and durable storage products due to the development of SHSS and applying it to OpenStack Swift.

3.Future Plans

This technology is expected to be used in storage systems later this year by our group businesses.

Glossary and References

^1:OpenStack (https://www.openstack.org/)

^2:AONT-RS: Blending Security and Performance in Dispersed Storage Systems, Json K. Resch, James S. Plank, FAST-2011, 9^th USENIX Conference on File and Storage Technologies, San Jose, CA, Febrary, 2011

^3:Jerasure (http://jerasure.org/)