The research explores the development practices of various OSS projects in order to identify the key elements contributing to SQA.
Project requirements often derive from different channels, such as direct feature requests by developers, user suggestions in mailings lists or from commercial partners. New feature requests are stored in public accessible lists or issue trackers to enable prioritization and status follow-up. The review process gains an enormous importance in a distributed user-driven development approach. It sets the foundations for development tasks and needs to reflect the evolving requirements during the entire lifecycle.
The documentation of development processes differs in its extent, however coding or style guidelines are frequently defined. The establishment of proper documentation has a high impact on quality, as its absence complicates the knowledge transfer to new participants.
We observed
OSS projects are able to handle the complexity of large development communities, as these projects largely benefit from user testing and parallel development, which scales up with increasing team size. However, the establishment of an effective organization becomes a vital issue to produce high quality software. Projects need to emphasize the organizational set-up to constitute a stable environment and enable efficient collaboration.
The participation and attraction of volunteers affects software quality. Appropriate structures are required to avoid frustration and keep people constantly motivated. This includes effective coordination of project tasks by the core development or management team, the implementation of issue tracking tools and the distribution of information via mailing lists.
Knowledge transfer processes gain an enormous importance to overcome a lack of know-how, as developers participate on a voluntary basis. We observed that some projects have strict rules and require mandatory documentation with code submission and often the information in the mailing list is used to capture knowledge.
The project infrastructure is very important, which directly affects the project coordination and communication. The establishment of appropriate tools for the infrastructure has a significant impact on process quality as well as the collaboration. Thus, the adjustment and integration of tools offers the basis for managing the complexity within a collaborative, distributed development approach.
QA processes under the OSSD model focus on design, coding and testing. Villa emphasized the need for an initial planning and a detailed documentation of design documents. An imperfect design phase which lacks of system analysis and the definition of the system architecture prior to coding might lead to severe quality issues when requirements evolve. Aberdour noted that
OSS projects neglect formal design and often go straight to programming.
Projects vary in their adherence to coding standards and guidelines, as most of them continually review changes, perform corrective actions and reject inappropriate code. A few rely on peer reviewing by their community. Sometimes style checkers are used for format corrections. We observed that large projects control the quality of the code more strictly than others. Many perform intensive reviews, require mandatory unit tests before code submission or con-duct automated testing.
Reviews and inspections processes assume a major importance in achieving software quality. These processes are conducted in parallel and scale up against system complexity. The more people look at the code the higher is the ability to detect defects early. The application of reviews and inspections in the
OSS becomes an integral element.
Larger
OSS projects take advantage of their community size and benefit from user testing. Michlmayr noted that defect-handling processes could suffer from an imperfect bug-reporting quality or security critical updates are not available on short call. Testing strategies vary in
OSS projects. Some projects require unit tests with every committed code, while others use automated testing and set up a test suite with test cases. A few projects do approach testing informally.
Improper release management procedures and strategies negatively influence software quality. The research has shown that a large number of
OSS projects follow a feature based release strategy, scheduled in larger intervals. However a time-based approach is often selected when closer to release. Quality is obtained by code freezes and intensified integration or regression testing towards the delivery of release candidates.
OSS projects apply the definition of quality targets and their assurance is mostly informal. We noticed the existence of minor goals, such as some benchmarks or code coverage targets. More surprising is the fact that no strict quality management approach could be observed with the aim of defining quality goals and their assurance.
OSS projects seems to neglect quality management processes or do not make a clear distinction of software quality assurance processes. The definition and applicability of QA processes across
OSS projects is not prevalent.
SQA processes are more frequently established when projects evolve in size and face an increasing complexity of their development processes. Some larger projects define dedicated responsibilities to achieve the execution of SQA tasks. Often projects perform SQA tasks more informally without any strategy or planning.
In summary, we suggest the following key processes to support QA practices under the OSSD: Product and Process Documentation, Coordination and Team Communication, Infrastructure and Tools, Knowledge Transfer, Requirements Evaluation and Design, Software Engineering Control, Peer Reviews and Inspections, Verification and Validation, Release Management, Quality Management, Software Quality Assurance and Continuous Process Improvement.