January 11, 2011 06:19
Posted by Jeremy Durham
Regardless of what Monty Python or Indiana Jones might have us believe, or not believe, there is no Holy Grail. In our case, the Holy Grail is the ability to run browser based Cucumber tests expediently. We’ve been chasing the Holy Grail for some time, and it ends today.
We started using Cucumber as we got more involved in the agile process. Unfortunately, like most other people, we were unable to catch our unicorn, and we never got our “customers” to write acceptance tests. Considering that we mainly do work for the government, I suppose we couldn’t expect to really have a lot of success getting them to write acceptance tests, but we subscribed to the process. The road was paved with good intentions.
Recently, something snapped. Ben mentioned that his friend had rejected Cucumber and exclusively written view specs with graceful degradation. I respect the merits of doing that, but it can be difficult to sell the idea of building functionality in such a way that it will never be used, or specifically for testing purposes.
Ultimately, we’d love to increase our test coverage and validity, decrease our test running time, and as a by-product increase developer happiness. There’s nothing more demoralizing than finishing a feature, only to have to wait an hour or so to be able to push it and start on your next one. Ideally we’ll return to the days where running tests 50-100 times a day is not unrealistic. Right now, we’re lucky to run the full suite 5-6 times a day.
Cucumber, you’ve been good to us, and although you probably won’t be going away completely, it feels like the end of an era. Hopefully the next phase will bring faster builds and better tests. I’m happy we’re ending our quest for the Holy Grail, and although we’re not holding the Holy Grail in our hands, we’re more knowledgeable and ready to move on.