It's Tuesday - what a day to visit jfoobar!

Continuous Integration in PHP; the building blocks

Written by Wilco Jansen Monday, 07 September 2009 21:56

Maintain a code repository and use version control

Karn Fogel already describes that a code repository and version control tools are a key piece of the infrastructure of your open source project. The code repository of course is a no brainer, without it there is nothing to maintain ;-) Version control or sometimes also referred as a revision control system is a combination of technologies and practices for tracking and controlling changes to a project's files, in particular to source code, documentation, and web pages. If you have never used version control before, the first thing you should do is go find someone who has, and get them to join your project. These days, everyone will expect at least your project's source code to be under version control, and probably will not take the project seriously if it doesn't use version control with at least minimal competence. An overview of available version control tools can be found on wikipedia.

There are a wide variety of version control systems out there, most common used are Concurrent Version System (CVS) and Subversion, while GIT - with Linus Torvalds as original author - is gaining some serious attention. CVS is used by Drupal for example, Joomla uses Subversion. A wide variety of client tools are available, in our examples we will focus on Subversion. Beside the wide variety of client (and server tools) available on virtually any platform out there. You will also find tons of documentation on the Internet, and if this is not enough you also have a lot of books describing these tools in detail.

Version control tools are not difficult at all. Just grasp the basic concepts of updating, committing, check out, and committing and you can do the most basic tasks incredible fast. You do not need programming skills to use these tools.

Use build tools

The source code that is stored in the code version control repository must be buildable when you want to release the software package. Professional projects automate the building process for obvious reasons. There are a lot of tools used in various environments like make on Linux or Ant for Java development. It is common practice that the build automation takes care of the integration (automatically) which often includes deployment into test and production-like environment. In many cases, the build script not only compiles binaries, but also generates documentation, website pages, statistics and distribution media (such as Windows MSI files or RPM or DEB files).

Within PHP Phing is a tool used very often to take care of the automated build process. GNU stands for "GNU's Not Unix" and with Phing they came up with the name in a similar way. PHing Is Not GNU make...well that explains it all does it not? It is a project build system based on Apache Ant, but then re-written in PHP (and also extensible using PHP). You can do anything with it that you could do with a traditional build system like GNU make. It uses simple XML build files and extensible PHP "task" classes that make it an easy-to-use and highly flexible build framework. Features include running PHPUnit and SimpleTest unit tests (more on that further on), file transformations (e.g. token replacement, XSLT transformation, Smarty template transformations), file system operations, interactive build support, SQL execution, CVS/SVN operations, tools for creating PEAR packages, and much more.

An example Phing script can be downloaded from the JFoobar project area on Joomlacode. This phing script could be used as daily build script and once has been used by me when building the first stable version of Joomla 1.5. It will give you a rough idea on how it works (also check out the Phing site for more information).

Self testing code

Code that tests itself? That does sound like some black magic voodoo, does it not? Why should you even consider writing code to test code? To answer this question, let us first answer the question on what we consider to be a successful build. When implementing a continuous testing strategy, one of the elements that determines a successful build is finishing all automated tests. A test driven development approach is one of the key elements for continuous building. As soon as one of the tests fails, the build is considered to be a failure also. There are more parameters that determine the success of a build, finishing the tests successfully is certainly one of them.

Zend for example uses PHPUnit and this with unit test-tool there has been some initiatives within the Joomla project. First visible work was done by Enno Klassing during the Summer of Code 2007, and his work was later on picked up by Alan Langford who assembled an article on the Joomla documentation wiki about Joomla unit testing. The unit test code for Joomla seems to be moving since quite some time and can be found in one of the Joomla repositores. Drupal uses Simpletest, another PHP unit testing tool that does the job and they have an impressive amount of tests there.

Some project also work on implementing web-gui automated test tools like Selenium. If you integrate this into the continuous build logic, you not only test API level logic, but also program flow. The challenge with automated testing, beside writing tests that really implement the goals you have, is platform dependency and PHP version dependency. PHP 5.1.0 can give different results then PHP 5.2.6 and PHP 5.3 can give different results with the same tests also. And with GUI tests you face browser hell :-D So far I have not seen an integrated solution to overcome this problem...

Validate your code against the coding standards

There are a lot of PHP coding standards in various projects. PEAR, Typo and Zend to just name a few. These coding standards are there for a reason. The use of coding standards helps in a way that everyone feels they are on the same playing field, and in my personal experience it help you with the readability of the produced code.

Good style is a subjective matter, and is difficult to define, but the use of coding standards makes it simpler to judge if main rules have been followed by developers. There are several elements common to a large number of programming styles. The issues usually considered as part of programming style include the layout of the source code, including indentation; the use of white space around operators and keywords; the capitalization or otherwise of keywords and variable names; the style and spelling of user-defined identifiers, such as function, procedure and variable names; the use and style of comments; and the use or avoidance of particular programming constructs (such as GOTO statements).

The Joomla coding standards are documented on the Joomla documentation wiki. If you look closer you will see the Joomla coding standards are a subset of the PEAR coding standards, which make it an interesting case for the blog series where we can validate how well the coding standards have been followed using PHP codesniffer (more on that later in a separate blog article).

Beside coding standard validation there are also additional tools available that analyse how complex your code is, Manual Richter has done some interesting work where he implements PHP Unit as a Project Mess Detector, combined with xdebug, Phing and the code sniffer you have some very valuable tools that help you improve code quality and with that maintainability of you code.

Code coverage analyses

Xdebug not only allows you to use an interactive debugger within for example PDT, is also can help you with profiling your code. With those features Xdebug also is able to analyse your code at run-time, and see what code is covered by the unit tests. If you dive into the PHP Unit manual, you see that chapter 15 describes the integration of code coverage analyses using PHP Unit In the ideal situation you have 100% coverage, but that is extremely difficult in larger projects since not all situations can be tested automatically (yet), but it is a very good indicator on how well your tests have been written, it gave me insight on missing parts during development at numerous occasions. The project mess detector for example handles code coverage below 70% as medium...

Generation of (API) documentation

Good code comes with documentation, and within continues builds this is no exception. Writing good documentation is essential to the success of any software project. The quality of documentation can be even more important than the quality of the code itself, as a good first impression will prompt developers to look further into your code. phpDocumentor is designed to make it easier to create documentation. phpDocumentor even makes it possible to generate separate sets of documentation from the same source!

Within PHP projects I see often that phpDocumentor is used as one of the tools to generate automated documentation based upon the DocBook markups in the code. The tool does an automated scan of the code, and is able to generate the output in various outputs (mostly HTML). Within continuous building the documentation is also generated on every commit, this way you always have your API documentation up to date (make sure the output of course can be accessed by your development audience).

Every commit triggers a full build

All previous building blocks should be triggered after every commit. The developer who performs the commit instantaneously will be warned if his commit breaks anything, varying from breaking the code guidelines till breaking some logic he might even not be working on. Of course you can use a less strict building policy, like nightly builds but this could introduce some additional coordination problems in environments where multiple teams are working on the same code-base.

To make sure that the continuous build approach does not stall development because developers need to wait to long I advise to keep the building process as short as possible, and also consider a local (manual) option for the developers to validate some basic aspects of the code they want to commit. In shared development environments it is important that everyone can see the results, once implemented you will find out that this way of working is incredible addictive, and boosts code quality and cuts development time considerable.

In the next articles I will go into a little more detail on some of these tools mentioned here, before I describe how you can actually bind them all together.

Like it? Share it!

• bookmark.it
• del.icio.us
• digg
• furl
• blinkllist
• reddit
• feedmelinks
• technorati
• yahoo
• rawsugar
• netvouz
• rojo
• shadows
• gabbr
• dzone
• newsvine
• ma.gnolia
• stumbleupon
• google
• squidoo
• spurl
• blinkbits
• blogmarks
• bloglines
• co.mments
• scuttle
• ask
• slashdot