Yahoo offers researchers supercomputer time

Stephen Withers
Tuesday, 13 November 2007 08:58

Business IT - Open Source

Yahoo will provide academic researchers with a 4000 processor supercomputer as a testbed for research into Internet-scale systems software.

The system in question is Yahoo's M45 cluster that also features three terabytes of memory and 1.5 petabytes of disks. Its peak performance of more than 27 teraflops puts it in the world's top 50 supercomputers.

"Yahoo! has been the primary contributor to Hadoop, an open source distributed file system and parallel execution environment that enables its users to process massive amounts of data. Hadoop has been adopted by many groups and is the software of choice for supporting university coursework in Internet-scale computing," company officials said.

The plan is for M45 to run the latest version of Hadoop along with other Yahoo-supported, open-source distributed computing software such as the Pig parallel programming language.

First off the rank will be Carnegie Mellon University. Faculty members will use M45 to tackle information retrieval, graph, graphics, natural language processing, and machine translation problems.

"Hadoop has become an important computing environment for data-intensive applications and Yahoo! is playing a leading role in its development. We are excited about collaborating with Yahoo! on systems software research, helping to advance the state-of-the-art, and creating new research possibilities in this critical area," said Randall Bryant, dean of the School of Computer Science at Carnegie Mellon.

Yahoo plans to make M45 available to researchers from other universities.

"Yahoo! is dedicated to working with leading universities to solve some of the most critical computing challenges facing our industry," said Ron Brachman, vice president and head of Yahoo! academic relations.