Measuring contribution by counting commits

Written by Wilco Jansen Monday, 03 November 2008 17:32

Working together in an open source community There is no open source project without code, but there is also not an open source project without a community. So saying the one is more important then the other will result in a endless debate, for me personally it's a fact of life that these two need to co-exist. Measuring contribution is a very tricky subject. As Arno already described in his article here on jfoobar people who are the public faces for an open source project tend to be recognized as the biggest contributors. The bloggers on this site are amongst that group, and I personally don't see me as the biggest contributor to the project, despite the fact that I amaze people when I tell how much time and effort we put into the project as volunteers. A project like Joomla! is a result of the work of thousands of volunteers, and with the lack of a benevolent dictator it is getting pretty difficult to say Joomla! is made by a small group of individuals. A project reward is more of a recognition of what has been achieved, although it is good that we sometime put some people into the spotlights as happened in the past with the member of the month initiative.

This blog got triggered by this years announcement of Packt; the Open Source CMS Most Valued People. For Joomla! Johan Janssens was mentioned by Packt. On Johan his business site I spotted an article stating that the amount of commits are a measurement for contributing to open source also referring to OhLoh that keeps track on the commits of other open source projects to support. The above inspired me to de-compile Joomla! Subversion and see what the actual (and latest statistics) are. My analyses gives an interesting view on what code committing means.

To perform the analyses I run the StatSVN tool for an in depth analysis of the code. The amount of data collected is impressive and too much to share online on this site. I am looking for a proper location on the web to host these statistics, and if possible auto-update the statistics (working to get them on developer.joomla.org).

For now I will just share my findings in this blog. Keep in minds that the tool does a full scan (including a full diff on every commit) to collect the statistics, this can easily overload the subversion servers of Joomla! I double checked with site and infrastructure before I ran these analysis.

Let's start with an introduction of the subversion structure of Joomla! before we dive into the statistics. Subversion for Joomla! initially was set up for Joomla! using the trunk as main commit path. Initially the trunk only held 1.0.x code. On 28 February 2006 the 1.0.x trunk was moved to a separate branch (the 1.0.x branch ), and work continued from there. From that date on the trunk was used for the total re-factory to Joomla! 1.5. Just before the release of Joomla! 1.5 stable, to be specific it was on 16th of January 2008, we moved the trunk to a separate 1.5.x branch.

A small summary of the subversion locations used in this blog:

Trunk location : http://joomlacode.org/svn/joomla/development/trunk/
1.0.x branch : http://joomlacode.org/svn/joomla/development/releases/1.0/
1.5.x branch : http://joomlacode.org/svn/joomla/development/releases/1.5/

The 1.0 branch

This branch was created on 28 February 2006. Last commit is from 5 March 2008 that supports the impression that Joomla! 1.0 is close to be end of life. The branch holds 121.315 lines of code. A total of 19 committers have been working on Joomla! 1.0.x. Let's throw some statistics from the top-5 committers here:

Rey Gigataras (stingrey); 1.995 changes (48.2%), comitting 40.351 (58,8%) lines of code.
Wilco Jansen (willebil, that's me); 613 changes (14.8%), comitting 8181 (11.9%) lines of code.
Robin Muilwijk (rmuilwijk); 20 changes (0.5%), committing 4.481 (6.5%) lines of code.
Enno Klasing (friesengeist); 328 changes (7.9%), committing 4.481 (6.0%) lines of code.
Alex Kempkens (akede); 33 changes (0.8%), committing 3.488 (5.1%) lines of code.

There is no doubt Rey Gigataras was most actively involved in maintaining Joomla! 1.0. The top-5 do cover around 85% of the total commits done.

The 1.5 branch

During the development of 1.5 the trunk was the main location where code was committed. Shortly before the final release of Joomla! 1.5 this branch was created on 16 January 2008. The code of the final version of Joomla! 1.5 and all maintenance releases have been committed into this trunk, giving a perfect overview on the committers who have been responsible for the maintenance of Joomla! 1.5. This branch holds 319.137 lines of code and a total of 17 committers have been working on this branch. You will notice a huge increase of the number of lines compared to Joomla! 1.0, almost 3 times as much.

This has partly to do with the legacy support that is included and also the dual support for PHP 4 and 5, so I think Joomla! 1.6 will show us a decrease of the amount of code. Let take a look at the top-5 for this branch:

Samuel Mofatt (pasamio); 2.745 changes (44.7%), committing 175.667 (66.6%) lines of code.
Wilco Jansen (willebil - hey that's me again); 913 changes (15.2%), committing 39.022 (14.8%) lines of code.
Mati Kochen (mtk); 65 (1.1%) changes, committing 22.140 (8.4%) lines of code.
Andrew Eddie (eddieajau); 903 (14.7%) changes, committing 11.220 lines of code.
Ian Maclennan (ian); 461 (7.5%) changes, committing 9.113 (3.5%) lines of code.

That's interesting is it not? You will ask where Johan is in this overview, you probably miss some other people also that you would expect in this overview, don't you? Let's put some things into perspective here. First of all the top ranking of Sam is perfectly explainable; in May 2008 Sam did a lot of work replacing comment fields in almost all the core files, not many actual code got changed here, but the stats show he is the top committers there.

I also show up (again) as second committer, some perspective to that would also be welcome. I have been the core maintainer during maintenance, processing all patches that have been applied by the bug squad since we moved to a "test the patch before we commit" procedure. I am marked as the committer, but the people in the bug squad really should get all the credits for that.

The trunk

The trunk is a mixed environment. All versions we know, even the new upcoming version of 1.6 are maintained here. When you look at the statistics you will see how mixed the statistics are here. A total of 42 developers have been registered as committer. The trunk was created on 8 October 2006 and holds a total of 323.078 lines of code. Let us share the top-10 of committers:

Johan Janssens (Jinx); 20.224 (33.1%) changes, committed 521.407 (46.5%) lines of code.
Louis Landry (webImagery); 6.808 (33.1%) changes, committed 111.737 (10.0%) lines of code.
Andrew Eddie (eddieajeu); 7.443 (12.1%) changes, committed 77.727 (6.9%) lines of code.
Louis Landry (louis); 2.218 (3.6%) changes, committed 66.322 (5.9%) lines of code.
Rey Gigataras (stingrey); 3.151 (5.1%) changes, committed 55.865 (5.0%) lines of code.
Anthony Ferrera (ircmaxell); 3.634 (5.9%) changes, committed 42.737 (3.8%) lines of code.
David Gal (davidgal); 1.630 (2.7%) changes, committed 33.169 (3.0%) lines of code.
Andy Miller (rhuk); 1.156 (1.9%) changes, committed 28.688 (2.6%) lines of code.
Hannes Papenberg (Hackwar); 1.766 (2.9%) changes, committed 19.036 (1.7%) lines of code.
Andy Wallace (humvee); 1.208 (2.0%) changes, committed 18.291 (5.9%) lines of code.

This overview gives a good impression on the people who have committed to the Joomla! project in terms of changes and lines of code committed. If we include the sandbox branches the overview really is totally different. The data taken from the top-level structure of the development tree gives no specific information, so I leave it out of this overview.

What do these statistics tell us?

I conclude that looking at the commit statistics gives a false impression on the actual contribution to the Joomla! project. No doubt a lot of time and effort goes into committing the thousands lines of code. The amount of work is not recorded anywhere for the amount of work the bug-squad members put into testing the code produced by the developers. Without community members supporting the coders, we would not have a stable and secure product that could be used, and these volunteers are often forgotten when we talk about contribution!

In the end it comes down to the recognition of individuals in open source projects, and that sometimes is only given to those who have a visible role within the open source project. We cannot thank the army of invisible volunteers enough, no open source project can live without them. Let us not ever forget that!

About the author Wilco Jansen

Wilco was born in 1967 in the Netherlands where he still lives. After years of being a programmer Wilco has worked as project manager and IT manager. Discovered Joomla! when he was creating his own content management system, and never lost focus after then. Joined core team as development coordinator in May 2006 just helping to make Joomla! even better then it is already. Wilco has been deeply involved in the Joomla project as Google summer of code program manager 2006, 2007 and 2008 editions, co-organizer of the Google Highly Participation contest in 2008, first ever development coordinator, creator of the Joomla bug squad, member of the board of Open source matters, regular speaker on world wide conference advocating Joomla and much, much more. Wilco has a bachelor degree in business and information engineering and studied Master of Science knowledge and information engineering at the Middlesex University in London.

More about Wilco Jansen

There are 7 comments posted.

Re: thanks for the detail

# 1 - Posted by: Brian Teeman on 2008-11-03 08:44:25

Thanks for the detailed analysis, it certainly makes an interesting read.

I've been playing with an exciting tool for providing a visual interpretation to the commits in a project over time. If you can share these joomla stats I can translate the stats into something more interesting.

(BTW it should be Andrew Eddie, not Eddie Andrew)

# 2 - Posted by: Wilco Jansen on 2008-11-03 10:01:02

The statsvn tool already provides a historical overview, including a mapping tool (most changed areas are red, and you can drill down from there).

As mention, I will try to get the stats hosted on developer.joomla.org.

Re: Childish Behaviour

# 3 - Posted by: Phil on 2008-11-03 14:14:16

All this tit for tat blog posting is getting Joomla no where - we all know Johan and the core are not best of friends any longer - but really is there any need for this childish behaviour - from either party?

We all know commits are not the best measurement of contribution. Contribution cannot be "measured" it just "exists".

Surely that's what TEAM WORK is about?

# 4 - Posted by: Phil on 2008-11-03 14:17:04

Furthermore, although Johans blog post implies the award was for commits - please read the Pack definition:

"The following list of names were put forward by members of the Content Management System's development team and community and represent the exceptional support, guidance, and sheer amount of time that the MVPs have given up to support the development and growth of the respective CMS."

Clear enough for any discerning reader.

# 5 - Posted by: Wilco Jansen on 2008-11-03 15:01:33

Hi Phil,

This post is not about flaming or blaming people from our community, maybe you are not aware, but this is not about the core team versus Johan since I am not part of the core team any more from August this year.

We all know commits are not the best measurement of contribution. Contribution cannot be "measured" it just "exists".

The fact that this was taken as a staring point in some discussion triggered me to de-compile subversion and take a look myself (OhLoh points to the wrong part of subversion for example). With the article I tried to make that visible, and we both agree here . The statistics give us a good insight on the huge amount of effort that a lot of developers have put into this project. No more, no less.

Regards, Wilco

# 6 - Posted by: Richard on 2009-12-05 00:00:49

I don't know the intricacies of this particular situation, but what I do know is that those who are "measuring contribution by counting commits" are idiots.

# 7 - Posted by: Wilco Jansen on 2009-12-05 00:11:07

Exactly my point when I wrote this article!

Help for creating beautiful comments.

It's Friday - what a day to visit jfoobar!