Is Open Source software safe and secure?

David M Williams
Thursday, 22 May 2008 17:21

Opinion and Analysis

It’s a big question: how trustworthy is the software I use on my computer? When it comes to open source, can you trust the quality of programmers who work for free?  You can, according to a new report out this week – which also proves major open source offerings to be especially well written. It equally shows up the projects which are slow to respond to vulnerabilities.



One argument for open source has always been that you have inherently more security because anyone can examine the underlying program code and verify it does what it ought to be doing. But then, does this really mean anything for non-programmers? In one sense, you’re still depending on the word of others; open source does remove a major barrier by giving you the program code but that’s only part of the puzzle. You also need both the time and the expertise to analyse it.

Here’s where Coverity come in. They are a commercial code analysis company which began at Stanford University. Coverity have been running a project called Scan for two years with funding provided by the Department of Homeland Security in line with its own objectives to harden open source apps. This week Coverity released their Open Source Report for 2008. This report is interesting stuff; it draws on two years’ worth of data from over 250 significant C/C++ open source projects – like PHP, Perl, Python and Samba, all veritable household names (albeit in a fairly geek household.)

Coverity aren’t just operating on guesswork; in the past they uncovered a security flaw in the source code that makes up the X-Windows graphical user interface which could allow any user to gain superuser privileges. Coverity also claim that their data caused 32 open source projects to fix over 900 flaws within a single week – or, more than five bug fixes every hour. These projects include such fundamental s as Samba and Gcc as well as popular tools like Ethereal. The Samba project team said Coverity had found bugs in code which had been previously considered absolutely robust and tested.

So, Coverity’s report is an important one. While their name may not be uttered in the same tones as that of, say, Canonical or Red Hat, they have the demonstrated experience that they are not some fly-by-night company.

Over these two years, since March 2006, the Scan project has analysed a combined total of 55 million lines of program code (actually, the 250 projects were scanned multiple times giving a total of 14,238 individual scan runs over some 10 billion lines of code.) This gives a pretty wide sample of programming styles. Coverity draw six conclusions from their research.

Some of these are a bit dry and technical: research indicates static analysis defect density and function length are statistically uncorrelated, for example. This means that it doesn’t necessarily hold that the longer an unbroken individual routine is that it will have more bugs than one of lesser size.

By contrast, research did find a strong linear relationship between code base size and static analysis defect count. This means that the more complex, and the more overall code, a program has then the more bugs it will contain. This is fairly intuitive and it’s certainly the argument trotted out whenever any sizeable piece of software – like an operating system – has faults revealed.

However, other findings are especially interesting. One which I really want to draw to light is that Coverity found the overall quality and security of open source software is improving.

CONTINUED