Debian's worst nightmare - and how it came about

Sam Varghese
Thursday, 22 May 2008 21:30

Opinion and Analysis

The implications of the bug are huge and systems administrators are not to be envied at this point in time. Craig Sanders, an extremely competent sysadmin based in Melbourne and a long-term Debian developer, said that he had all of his 15 hosts updated in half a morning - but this was because the non-signed certificates he had were generated on a non-Debian webserver.

For those looking after hundreds of servers, the process will be much more time-consuming and nerve-wracking; and depending on the number of certificates generated using vulnerable systems, they may be looking at months or even years of work to track down and change certificates.

"The process is essentially to upgrade to the latest OpenSSL and OpenSSH packages; regenerate any vulnerable keys; fix any known_hosts and/or authorised_keys files; replace any vulnerable SSL certificates (e.g. for SSL web server, TLS mail server, SSL IMAP and POP, etc) and repeat for every machine/service that is affected," Sanders said.

Peter Giorgilli, a longtime UNIX developer and systems engineer, said the whole process had shone a light on what had happened in an open source shop. "At least, you know how the sausages are made," he said.

"On the other side, there could be similar, or worse, things happening in a closed source shop. You never hear about them - and you still end up eating the sausages," he observed.

The Debian fix, issued on May 13, also included a link to a detector for known weak material and also a link to detailed instructions on how to roll over keys generated by the vulnerable versions of OpenSSL.

Russell, who is well known for his work on the security-enhanced Linux project, said the new Debian OpenSSH packages would check for and reject broken keys. "I am not aware of other distributions doing this, so if you run Linux servers that are kept up to date then running Debian may be regarded as being more secure because of this," he added. Upgraded servers also refuse log-ins from unsafe client keys.

However even in the darkest of moments, geeks do have a sense of humour - and this bears testimony to that.

That said, there are some lessons from this episode. The corporate world benefits enormously from the labours of a great many talented hackers, who churn out software at no cost and then give it away. It is high time that some of these leeches look at giving something back so that these hackers are able to devote some more time to their pet projects. After all, even the most motivated software developer has to eat. Projects like OpenSSL provide good quality software, as good or better than commercial alternatives. More time means the chance for more QA and more code audits; it means better quality and better security.

The same goes for the Debian project too. It could do with some more corporate involvement in the shape of jobs for developers, the same developers who keep the distribution going year after year.

And, finally, the OpenSSL fiasco is a reminder to all that a very simple thing can defeat the best of technology - remember, once a man with an ordinary hoe cost a goodly portion of the US its internet access. And it only took one anchor to cut a cable and keep most of South-East Asia off the net for more than a week.