I am probably biting off more than I can chew, within a brief blog post here. But, I wanted to capture a couple of points relative to the notion of the quality of open source software (OSS).
There are two simple principles which drive the potential for high quality in OSS. The first is the fact that open source software generally builds on the success of upstream projects. Second is the impact testing has on the lifecycle, not to be confused with the "many eyes" concept though it is orthogonal.
There are very few "root-level" open source software projects that do not consume other open source works. The benefit here is that the good ideas, and more importantly, the good implementations rise to the top and stick - validated by use in other projects. This generally holds true for all types of open source software, starting with tooling and libraries all the way up to end-user applications.
It is a Darwinian cycle; the weak simply won't survive. Indeed this in-community competition drives the quest for maximum performance and the highest quality. This collaboration between upstream/downstream developers is also a validation of the pragmatic practice of decoupling.
By separating "concerns" each project can focus on a singular domain, continually optimising within a given boundary. I'll go ahead and assert that this optimisation does result in higher quality software.
Based on this, I will additionally contend that a collection of higher quality software will result in a higher quality end product. Ultimately it is the genuine collaborative relationship that exists between the collections that allows for this to happen - technical and social dependencies of a true meritocracy in action.
A Google search for "open source software quality" will reveal the brewing desire to "prove" open source software quality in an academic (and thus political) sense, especially within the EU. Organisations like QualOSS and SQO-OSS want to measure and test open source projects to improve European organisations’ (specifically those building open software) ability to compete.
By virtue of open access to the source code (the "many eyes" concept) these organisations have an opportunity to qualify quality, in whatever terms they choose.
Sure, these scientific attempts to prove software qualities are beneficial, but they are not just focusing on the code: they analyse the projects' mailing lists and bug/issue repositories as well - adding the community itself as a factor in the quality/overall health of the projects, so have a wider scope than many might realise.
Testing is a fundamental part of any software development lifecycle. Let's take a quick look at ActiveMQ, an Apache-homed project providing a messaging and integration platform.
ActiveMQ is one of those projects that is highly dependent on the capabilities provided by other open source projects. Because of those dependencies it is imperative that ActiveMQ is run through numerous regression tests, tests that prove that ActiveMQ behaves as expected any time there is a change to its codebase, or any of the dependencies.
There are many different levels of testing that happen, starting with very low-level unit tests that prove out pieces of code functionality all the way through integration tests that ensure the external interfaces work as advertised.
ActiveMQ (as of this writing) has 2,299 tests that comprise its portfolio of project tests, ensuring its behaviour. Each time there is a new behaviour added to the project, there are associated tests that are also added. With these tests in place the community of developers can constantly re-factor and improve the code, while ensuring that tests continue to pass - software confidence.
These same tests that open source developers use to validate and verify their work are distributed with the source code - to anyone who wishes to build and test the project’s work - effectively making anyone an alpha/beta tester.
It is the same level of testing that "external" developers can use to solve their own problems, and collaborate with project developers, providing patches, insight, and feedback. From a system integration perspective the development community can take that software confidence, and relay it to our clients and customers. It is also this openness around testing that promotes good citizenship - all stakeholders can play an active role in the production of the software, even if not writing the project's code.
Quality is not a static value; it goes with the ebb and flow of the software development cycles, and the integration of the entire stack.
Open source projects do extremely well to ensure that quality is a rising element by using good, sound software development practices - because they have to.