I was working on some example code for one of my writing projects recently when I ran into a failing test. "Ouch", I thought, "those tests were passing last week - what happened?" Rather than trying to find the bug in the code in front of me, I used what I think I'd like to call diff debugging.

First went to some clear space on my machine and checked out the code as of when I'd last thought it worked - May 14. I compiled and tested, and yes the test that was failing now was fine then. I looked at the repository logs - the only changes since then were the next day when I marked the code up with some comments for my auto-import. So I checked out the project as of May 15 into another directory. This time the test failed. So now I knew that I did something bad on May 15.

To find the mistake I then threw a diff tool on the two directories to see what had changed. I started with the most likely file and ran my eyes down the changes. There were some comments added, a couple of renames (which I've never had fail with IntelliJ), and ooo - a line removed. A line whose name clearly indicated it was relevant to the failing test. Although a background process kicked off in my brain to wonder why I'd removed a line of code while marking up, my main task was to restore the line to my head copy and re-run the tests - green bar :-)

Now I moved that background task to the foreground (it's always good to do a mini-retrospective in your head after a bug hunt). Why had I randomly deleted a line? The answer, I'm sure, was an awkward interaction between IntelliJ, Emacs, and my fingers. IntelliJ has this feature where if you type CTRL-X with nothing selected, it cuts the current line. In the past this never caused me any problem (although I don't think I ever used it.) However I've been using emacs a lot in the last year or so and, as anyone who uses the editor-that-wants-to-be-an-operating-system knows, to save a buffer in emacs you use the key sequence CTRL-X, CTRL-S. That double key sequence quickly gets wired into your fingers, so if you do that in IntelliJ you lose a line just before you save. I stopped this from biting me again with a plugin.

The most important lesson here is to be aware of diff debugging. It only works when you have a regression problem and you need to have a test and a reasonable build process so you can locate the problem reasonably quickly. Indeed most of the time I see diff debugging is during an programming session when you have local version control. A lot of people don't think to do this because their build process makes it too difficult to do, but if you have a good build process this can be a very powerful alternative to other forms of bug hunting