Andy Pols CTO, Entrepreneur

Some random thoughts...

The Perfect Customer

I enjoyed reading Keith Braithwaite’s post on tests and gauges. I like the way Keith uses the the metaphor of gauges in metalwork to describe Fit style tests:

You don’t work anything out from a gauge. What you do is apply it to see if the workpiece is within tolerance or not.  And then you trim off, or build a bit up, or bend it a bit more, or whatever, a re-apply the gauge. And repeat.

I agree with Keith.

The fit tests I write are business facing</a> and for the benefit of the business customer.

On a recent project our customer got really enthusiastic about our fit tests and got extremely upset when I implemented a story without a fit test. He refused to let the system go live until we had the fit test in place.

The story in question was very technical and involved sending a particular xml message to an external system. We just couldn’t work out what a fit test would look like for this type of requirement. Placing the expected xml message, with all it’s gory detail, in the fit test wouldn’t have been helpful as it’s a technical artefact and of no interest to the business. We could not work out what to do. The customer was not around to discuss it, so I just went ahead and implemented the story (very naughty!).

The customer was using the fit tests to show him what the system was doing. In this case he wanted to be sure that we were sending the correct product information in the xml message. To resolve the issue, I suggested that we have a Fit test that shows how the product attributes get mapped onto the xml message using xpath; although I still thought this was too technical for a business user.

I gave the customer a couple of links to explain what xpath was so he could explore and decide if this was a good solution for him. To my amazement, he was delighted with xpath (I now know who to turn to when I have an problem with xpath) and filled in the fit test.

The interesting bit for me (and why I’m posting it on the blog) is that as soon as he knew what the message looked like and how it was structured, he realised that it did not really support the business - we were sending information that was stale and should have been supplied by another system. He was also sceptical about the speed the external team could add new products due to the complex nature of the xml.

Most agile people I tell this story to think we have the “perfect customer”!

I agree.

Speeding up a slow build

I though it was worth sharing some experience we are having speeding up our very slow build from 1 hour plus to around 10 minutes.

In the past we have spent a lot of time exploring why our tests are slow and have never really come up with a good solution. We’re continually extending the test suite, so any savings we make don’t last that long.

The unit tests are quick (~40 seconds), but it’s the acceptance tests that start up and run the application that are painfully slow.

We decided to group the acceptance tests into small logically grouped test suites that can run in 10 minutes or less. Then we added build agents (each running on a separate machine), so we can run as many of these small test suites in parallel as we can.

Every time the build gets too slow we break them down into smaller chunks and add another build agent. Hardware is cheap so it’s easy to sale this solution.

So far this is working really well.

Experience with FIT

I have just discovered an old post by Jim Shore ( about how he started out using FIT. He started out by switching from a unit test driven approach to a more story driven development approach.

I encountered a big problem with this approach, though: I stopped doing good TDD. “Storytest-driven development” blurred the line between Fit and unit tests. The “STDD” cycle largely replaced the TDD cycle and, as a result, I had far fewer unit tests than normal.

That was a problem because Fit examples are not unit tests. Fit examples are just that: examples of a business rule in action, from a business perspective. In TDD, I write tests for every line of code that can possibly break. In my Fit examples, I try to talk about every possible business case. That’s a bigger net than TDD, and without TDD, my code wasn’t well tested.

I had the exact same experience. I also found that the design suffered due to the lack of tests (it did not force me to refactor as often as I should) and it had a detrimental impact on my productivity. Story tests are larger in scope and take longer to run. Unit tests provide a much faster feedback loop when you break something.

I wish I had read this before I started using FIT.

Using puzzles to find good developers

As a follow up to my post on pair programming based interviews, I noticed this on a recent job advert:

In chess it is possible to place eight queens on a board so that no one queen can be taken by any other. Write a program to determine all such possible arrangements for all eight. I am not looking for a Chess Games Developer. I am looking for Java Developers who have the OOA and OOD skills to solve this problem….

Would you prefer a test that’s abstract and has nothing to do with the work involved, or one that that involves doing the job you are interviewing for?

I wonder if they have run this test on all of their existing staff to see if their is any useful correlation?

Pair Interviewing

I noticed an interesting discussion on the IXP mailing list about pair programming with developers during an interview. I was surprised by the number of people who thought it was a bad idea.

I have done a lot interviewing for various companies and one of the best ways to find out the beliefs and values of a software developer is to look at their code. The best way to know what someone is like to work with is to work with them! This is why I really like pair programming interviews. It’s one of the best interview techniques I know.

Another nice feature of a pair-programming interview is the continual conversation you have with the candidate. It provides you with ample opportunities to ask sensible questions about the code you are writing. This gives the candidate a better understanding of why you are asking the question. There’s nothing worse than crazy interview questions that make no sense!

There are certain things you have to be aware of. You have to be very good at putting people at ease. You need to explain why you are running the interview like this. Some people get very nervous at an interview. You have to treat it as if you are pairing with a colleague. If people get stuck, you should help them out.

An interview is a two way process. The candidate also gets a much more realistic idea of what it would be like being in the team. They see what it’s like working with the team and experience the development environment first hand.