Thursday, 05 April 2018 16:31

'Tape is the new tape,' says Azure CTO – for now


A lot of attention is paid to making data accessible as quickly as possible, but that comes at a price. There are situations — typically driven by regulations — where it is necessary to store data beyond the period during which you expect to use it.

After evaluating various technologies including Blu-ray and magnetic disk, Microsoft came to the conclusion that tape was still the way to go for Azure's archival storage tier, said Microsoft Azure CTO Mark Russinovich.

"We believe tape is the new tape", so the company builds automated tape libraries with up to 72 tape drives and 12,000 tape cartridges, plus one to two robotic arms to load the required cartridges.

The downside is the resulting high latency: the robotics have to load a cartridge into an idle drive, and then the drive has to seek the right place on the tape to start reading the data, he explained.

This arrangement is economical at a reasonably large scale, but the entry point is somewhat expensive for a new Azure region. Over the last five or six years, Microsoft Azure has been working with Microsoft Research to create a smaller-scale archival storage solution. The first generation of the resulting Pelican system contained 1100 hard disks with a raw capacity of over 11PB in one rack. Two servers connect to these drives via a PCIe bus.

The problem is that the power required by so many spinning drives would normally greatly exceed the budget for a single rack. Microsoft overcame that limitation by spinning up drives only when they are actually needed. Having a just a subset of the drives online at any time keeps the rack within the power budget, Russinovich explained.

The price is higher latency. If the maximum number of drives is already active, there is an initial delay while one becomes idle and is spun down. Then the required disk must spin up before the data can be read.

"But what this gives is this intermediate price point between tapes and standard hard disk storage," he said.

Microsoft Azure CTO Mark Russinovich

Microsoft isn't resting on its laurels: it is working on two technologies that have the potential to provide archival storage at prices lower than that of tape.

Project Silica — a collaboration between Microsoft Research, Azure and the University of Southampton (UK) — aims to store data on glass.

The advantages of such a system is that once written, the data is truly permanent. Where SSDs, disks and tapes require data to be rewritten every few years or every decade to avoid bit rot, "if you can store data in glass there is no decay at all. It will literally last for the rest of the lifetime of the planet Earth", Russinovich said.

Furthermore, glass is extremely cheap as the main raw material is sand.

The stumbling block as been that it is very hard to etch data into glass without compromising the integrity of the medium. Using standard lasers to do the etching results in microscopic cracks that eventually make the data impossible to read.

Project Silica gets around this by using lasers that can produce pulses as short as a femtosecond – one quadrillionth of a second. It also encodes three bits of data into one voxel (a point in three-dimensional space), and writes the data in multiple layers inside one piece of glass.

"Another really cool characteristic of glass is you can always create a reader for it," he said. "For reading glass all you need is a light. You read the reflections coming out of it."

"But that's not the only promising technology for storing data in an archival way very efficiently, very low cost," said Russinovich.

Project Palix — a collaboration between Microsoft Research, Azure and the University of Washington (US) — aims to encode data into DNA strands.

DNA lasts for around 2000 years if stored at around 10 degrees centigrade, and can be read using existing gene sequencing technology.

The real promise is in the remarkable density: as much as one zettabyte could be stored in one rack. A zettabyte is 1000 exabytes; an exabyte is 1000 petabytes; and a petabyte is 1000 terabytes. Russinovich explained it another way: "To give you an idea how big it is, people are estimating by the year 2020 there will be about 20 zettabytes of digital data on the entire planet. So [the proposition is] one-twentieth of the planet's data stored on a single rack.

"We believe we are close to making this commercially viable in the very near future," he said. "This has huge potential... [for] providing extremely low-cost archival storage."

Russinovich was in Australia for the opening of Azure's Australia Central 1 and Australia Central 2 (Canberra) regions.

WEBINAR event: IT Alerting Best Practices 27 MAY 2PM AEST

LogicMonitor, the cloud-based IT infrastructure monitoring and intelligence platform, is hosting an online event at 2PM on May 27th aimed at educating IT administrators, managers and leaders about IT and network alerts.

This free webinar will share best practices for setting network alerts, negating alert fatigue, optimising an alerting strategy and proactive monitoring.

The event will start at 2pm AEST. Topics will include:

- Setting alert routing and thresholds

- Avoiding alert and email overload

- Learning from missed alerts

- Managing downtime effectively

The webinar will run for approximately one hour. Recordings will be made available to anyone who registers but cannot make the live event.



Security requirements such as confidentiality, integrity and authentication have become mandatory in most industries.

Data encryption methods previously used only by military and intelligence services have become common practice in all data transfer networks across all platforms, in all industries where information is sensitive and vital (financial and government institutions, critical infrastructure, data centres, and service providers).

Get the full details on Layer-1 encryption solutions straight from PacketLight’s optical networks experts.

This white paper titled, “When 1% of the Light Equals 100% of the Information” is a must read for anyone within the fiber optics, cybersecurity or related industry sectors.

To access click Download here.


Stephen Withers

joomla visitors

Stephen Withers is one of Australia¹s most experienced IT journalists, having begun his career in the days of 8-bit 'microcomputers'. He covers the gamut from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences and a PhD in Industrial and Business Studies.



Recent Comments