Jeremy Durham

A new design and now a new post. I’m on fire!

I’ve been spending a little more time lately optimizing our build times, and ran across a useful snippet I’d written a few months ago.

We have a few projects with over 4000 Selenium features (we love that Javascript!), and every once in a while we run into a few false positives. Features will run slow on the build machine and cause the build to fail, even though it shouldn’t.

Cucumber supports a “rerun” format, where it will output a rerun.txt file of all the files that failed. You can push that rerun.txt back through Cucumber to just rerun those features.

Here’s the code from ci.rake:

namespace :ci do

  task :all do

    rerun = "#{Rails.root}/rerun.txt"

    RAILS_ENV = 'test'

    Rake::Task['log:clear'].invoke

    Rake::Task['setup:app'].invoke

    Rake::Task['spec'].invoke

    FileUtils.rm_f rerun if File.exist?(rerun)

    begin

      Rake::Task['ci:features'].invoke

    rescue Exception => e

      if File.exist?(rerun)

        rerun_features = IO.read(rerun)

        Cucumber::Rake::Task.new(:rerun_selenium_features) do |t|

          t.cucumber_opts = "--format progress -q #{IO.read("#{Rails.root}/rerun.txt")}"

        end

        Rake::Task['rerun_selenium_features'].invoke unless rerun_features.to_s.strip.empty?

      end

    end

  end

  Cucumber::Rake::Task.new(:features) do |t|

    t.cucumber_opts = "-q --format rerun --out #{Rails.root}/rerun.txt"

  end

end

It’s important not to blindly use this task; if you continue to run into the same feature failing over and over again, it could be another issue. We’ve seen instances where stories will set some persistent data in the browser’s local storage, and later stories will fail unless run in isolation. Make sure you’re looking at your build output regularly!