ETD is now configured to be tested & monitored in Hudson: http://waterhouse.library.emory.edu:8080/hudson/job/ETD/
It took a bit of work to get PHP SimpleTest results into a format that Hudson can process and give detailed, granular results like it does for the Django apps or anything else that natively produces jUnit-style testsuite xml, but now that I’ve figured it out for ETD it should be pretty easy to set up for any other existing PHP code-bases using simpletest for unit testing.
Here’s what I’m using for the build command in the ETD Hudson configuration:
Then, in the post-build options I enable Publish JUnit test result report with this path:
trunk/tests/suites/test-results/TEST-*.xml
A few notes on the what/why/how.
Simpletest has an xml reporter that can be extended, but it’s rather limited– it outputs the xml as the tests run, so you don’t have information on total pass/failure until the end. I extended the default XmlReporter to make one minor modification– timing each test method as the test runs. To use this, include simpletest/xmltime.php and use XmlTimeReporter to run the report you wish to pass off to Hudson. For ETD, I created a new test suite script that includes all of the other unit test groups and uses the XmlTimeReporter.
I also discovered that if there are any errors, warnings, etc. before the test actually starts, those warnings are displayed before the xml output– making the resulting file invalid xml. I wrote a very minimal php script to clean up anything before the xml prolog and put it into a system-err tag. Then I wrote an xslt to convert the xml generated by simpletest XmlReporter (with my test duration addition) into the testsuite xml format used by jUnit and understood by Hudson. I couldn’t find any actual documentation of this format, just examples, so the output is an approximation and may miss some things, but Hudson is now correctly picking up failed tests and displaying test results for those failures, which I’m happy with (for now at least).
The additions to simpletest that make this conversion possible are now in our local simpletest subversion repository, so any project that is using simpletest should have access to them.
Update January 29, 2010: for anyone who’s interested but doesn’t have access to our subversion, here’s the content of the relevant two files.