22/10/09, 8:59
I’ve posted a lot lately, but I think this post is well deserved. Tomer has recently finished an open source project he’s been working on for quite some time – ant-intellij-tasks. It is an ant task that takes an intellij-format project structures, builds and tests it. We have been using it for the past two months or so, and I must say it has brought great joy to the CI. Before that, we used to maintain two separate sets of files – IntelliJ project files, because that’s our IDE of choice, and Eclipse files, because … that’s how the build system worked. Many a times the build broke because one did not apply a change to the eclipse file, and “it all worked on his machine“. No more (we still have broken builds, but usually not because of compilation errors).
In related news, JetBrains recently open-sourced IntelliJ itself. Tomer’s work is still worth while, simply because there is no other tool we know of that does what it does – however the code there might use some refactoring now that he can simply use portions of the IntelliJ codebase. If you want to help, go on to the project homepage.
25/12/08, 13:57
We’ve all had these. Annoying tests, that work perfectly 85% of the time, but fail mysteriously when the moon is half full and someone is using decaff. We (myself most certainly included) usually tend to ignore it, rerun the test and pray the problem will just go away.
This is simply not the way! It’s an acceptable solution for tests that work 99% of the times, but as soon as a test starts failing sporadically, you’d better do one of these:
1. Analyze the test, understand the source of the indeterminacy, and make it deterministic! This is not always practical because it’s usually the most time-consuming solution.
2. If the test is really fast, you can consider rerunning it automatically on failure / introducing more sleep() to eliminate fuzinness. Not a real solution, but will get the test green some times.
3. If all else fails, just comment out the problematic part of the test or even the entire test. This is not what you really want, but it’s better than just leaving the test failing sporadically.
If you do nothing, you’ll quickly experience CI degradation – your test suite will become meaningless. People will no longer care about it, not even enough to fix tests that are easy – because “The CI is broken anyway”. An all green CI is a wonderful productivity tool, and it is reachable and worth the ROI.
