java work

Continuous Integration woes

It's three months into a hectic projects and we still can't get a fully automatic build using AntHill featuring : CVS checkout, full build, EAR generation, non-regression tests compile (I don't dare call them unit tests!), BEA startup, EAR deployement, test running, JUnitReport, BEA shutdown, email to everyone if the tests didn't run 100% and I'm frustrated! I haven't had the time to really solve a nagging problem with this... Here it is.

We have a database that's setup just for the non-regression tests (which delete all data in the tables and sometimes insert data using DbUnit). We also have a development database for each developer (actually, just a different schema since we're using oracle). Obviously, when the tests run we need to tell DbUnit which database to connect to and the EJB Business Delegates the URL to the J2EE server... This is specified through a properties file using properties, one of them is jdbc.instanceName. Here is the catch : this properties file needs to be in the CVS for AntHill to check it out and it needs to contain the correct data. But if it's in the CVS, any developer may commit the database instance name he's been using by accident. Worst of all, he may commit the name of the database that actually has real data in it (used for sanity checks on the code)...

So I had this great idea, I'll refactor the config module so it looks for a config property in the SystemProperties first and I can tell AntHill to add a -D property when building the project...

That was a great idea until I noticed that when you fork your junit task in ANT, it doesn't propagate SystemProperties (which does make sense, I agree). But if I don't fork it, I get ClassNotFoundException errors and LinkageErrors because of XML and ANT's classloader

Now, you technically can set system properties when launching your JUnit task using (unsurprisingly...) the systemproperty tag so I did something like this: . But it fails when tests are run through the command line by developers since the system property is not set when launching ANT that way...

So I guess, I will have to tell every developer to add a system property named jdbc.instanceName with the name of the database to use :(

Version control strategies. A must read for anyone technically managing projects

I just found Streamed Lines: Branching Patterns for Parallel Software Development, it's a great repository of common mistakes and their consequences as well as best practices. concerning version control, branching and merging.

We had some pretty efficient version control strategies at my previous company and I learned a lot from talking with the people in charge of it company-wide. I was glad to see a lot of them in that document. I have been able to bring some best practices to my current company which has helped prevent code freezing and just make sure we always know what's in a release. Here is what we're now doing (bearing in mind, we develop our own plateform and we run it in ASP mode)

• all developement takes place on the MAIN branch


• whenever we have finished a set of features which form a release (whether minor or major), we create a branch and tag its root


• We use the branch to build the EAR and the rest and "acceptance-test" it on our mock production environment


• Any bug corrections during testing takes place on the branch, once everything is running ok, we upgrade the production environment and tag the branch with the version number


• We merge the branch into MAIN


• During this time, development of new features went on on the MAIN branch...


• If we need to correct a bug on the current release running on the produciton environment, we go to that branch, make the changes, create a new EAR, test it, put it up on the production environment and tag the branch with a different version number


• Whenever that happens, we merge the changes back into MAIN. Since we're using CVS, merging is easy if we use the previous production tag as the root tag...

Apparently, we're doing Parallel Releasing/Development Lines (a.k.a Anti-Freeze Line) with Deferred Branching, It works pretty well and I'm pleased to see the name they give to it explains exactly why I put that strategy in place :) Sometimes things just fall into place !

Testing with SQL persistance & DbUnit

glen.blog-city.com - Ant, JUnit and SQL seems to be doing the same thing I am lately : setting up the build process, using ANT, doing Db setup for them etc...

Here is how I did it : our project is built using ANT. I have a target that creates the whole EAR (we're using EJB). I have a target that compiles the testsuite and another that starts weblogic, deploys the EAR, runs the tests and stops the server.

Every tests derives from DatabaseTestCase which extends TestCase. This class uses DbUnit (http://dbunit.sf.net) which is great tool...
This class has two methods of interest : emptyAllTables() which empties all the tables from the database and insertFileIntoDb() . This method takes the name of an XML file that is on the test's classpath. This is a flat xml file of what the contents of the database should be. We have one such file for each table. Whenever we test a method in a class, we load the necessary tables. We created these files by extracting the contents of a test DB using DbUnit itself
This allows us to prevent side effects and to control the test environment when mocking objects is not a possibility.

A few great things about DbUnit : it can handle blobs, it can handle DBs with multiple schemas such as Orcale, it has lots of other nice features such as SQL queries through a simple interface, etc...

Prevayler in java & the dangers of object serialization

Today we've been hit by something which I thought wasn't possible : our project is build around EJBs. We use eclipse as an IDE. Our EJBs are built using ANT and javac while Eclipse has its own compiler.
We have tests which we run from Eclipse. Two of our value objects create marshallingexceptions when they are exchanged between client (in eclipse) and server (javac-compiled EJBs). This got solved by setting the serial UID manually. I'm surprised the JLS doesen't specify that the same code compiled by different compilers must give the same serial version ! In fact different versions of the same compiler can give different results and JBuilder 5 seemed to be snagged by that, there are FAQs about it and someone even created a bug in Eclipse which wasn WONTFIXed... see #10104.

So here's what I've been thinking... I've been reading about prevayler... it takes consistent snapshots of the system. It's currently implemented using object serialization... ok, let's imagine I'm running JDK1.3 and I want to update to 1.4. I shutdown the system... everything gets stored to a persistant store. I recompile the system without changing any code and start it again... given the problems we ran into, I'm pretty sure someone would run into incompatibilities... I think that's a problem... perhaps all serializable objects in Prevayler should have there serialVersionUID set manually...

my 2 cents...