Technology news and Jobs 
Our Blogs Open Sauce Linux: for all things great and small
Our Blogs Open Sauce Linux: for all things great and small | Linux: for all things great and small |
|
| Open Sauce - Linux Blog | |
| by Sam Varghese | |
| Tuesday, 18 November 2008 | |
|
Page 4 of 6 VPAC is a non-profit but does deal with a fair bit of money. This kind of iron costs a cool million or two. Income is generated from the clients who use the cluster. Featured Whitepaper
5 Best Practices for Smartphone Support
"The universities get charged a fixed base per year and they are free to allocate it among their users," says Appleby. "We try to work with the users to find groups that are under-utilising the facility and groups that are over-using and kind of balance out the usage. It's very cyclical and that's the advantage of VPAC. We can be running at 80 to 90 percent utilisation 365 days a year. He says the fundamental advantage for science is two-fold. "On a large parallel cluster, a job that would take you a week to run can often be run in a matter of hours. The other advantage is that you can tackle a much larger problem. The interest these days is in tackling much larger simulations." VPAC does use some proprietary software too. "In some cases, the proprietary packages have features which people really need or for one reason or another. We have a budget for that. That does chew up a fair amount of cash each year," says Appleby. "We have to be pragmatic about it. Some people are in publishing and their journals will require them to replicate results with obscure pieces of software. We are here to help the users to do good science and if they are required to do it a particular way, then we have to help them," adds Samuel. Bannon points out that there is also software used at VPAC which is midway between the two genres. "We also have a range of so-called crossover software that is halfway between those extremes, software that is effectively free for scientists or academics to use but with some constraints. They are not open source as in GPL, but available free and we have some obligations to make sure that they are only used within those limitations." Samuel uses an open source queuing system called TORQUE to manage jobs on the VPAC cluster. "These kind of machines are always over-subscribed - there are always more people wanting to use them than they can take on," he says. "You need to keep that in an orderly manner so we use a queuing system called TORQUE which is open source." This means that he can contribute things back to the project. "At the last supercomputing conference, there was myself, a developer from USC in the states, and a guy from Cyro sitting down and nutting out the details of CPU support so that you can allocate processes to processors, bind them, so that they do not move. Out of that came an implementation of the same thing and that saved us a lot because users can sometimes do things which they do not intend to do. They may spark many more processes on a node than they think they are doing. Now with the CPU set times, it means that they cannot trash anybody else's jobs on the node, they can only trash their own job. And from the end user's point of view, that's an important step forward." VPAC also provides feedback and fixes to proprietary systems. "Commercial vendors are making applications to run on clusters but they are often not in a position where they can get to test them on big clusters," says Appleby. "People like Chris have to spend huge amounts of time fixing their scripts to get their applications to run; then he usually sends them back that material and they may or may not incorporate his changes in the next version. "These are proprietary binaries to which we have no access. But there are usually some scripts for actually doing the launching. The problem seems to be in the launching. Commercial vendors make a lot of assumptions about how their applications are going to run, they usually make assumptions on the lines that the cluster will only be used to run this particular application. Now that's not a bad bet for a small engineering house, for example, that buys a small cluster for running an application that's core to their business. It's not very good for very large machines that by their nature have to be for general use." |
|
| < Next story in category | Previous story in the category > |
|---|
