Making Your Test Suite Scream with Specjour

You know all those other Macs you have lying around the house sitting idle all the time? Well I have some great news. You can finally put those freeloaders to work! With specjour ( http://github.com/sandro/specjour ), you can distribute your test suite across all the cores on your network!

It uses bonjour to communicate and can support multiple projects simultaneously. As long as the machine has the dependencies required for the test suite (gems and unix packages), it can participate in the testing goodness. The current contents of your project get rsync’d to the managers, so they will always match your local changes.

You get started by installing it into your Rails project. First, setup the dependency in the test group of your Gemfile.

group :test do
  gem ‘specjour’, ‘0.2.5’
end

Now make a small update to your config/database.yml. A number needs to be appended to the test database name to support multiple simultaneous tests for the same project. It matches that used by parallel_test, so if you used that previously you should be all set.

test:
  database: blog_test<%=ENV['TEST_ENV_NUMBER']%>

To fire up a local manager, open up a new terminal tab and simply run:

specjour

To execute your test suite, run:

rake specjour

It will automatically determine the number of cores on your machine and utilize them all. With it working on your main dev machine, you are all ready to get the real magic started!

SSH to another machine on your network that you want to run the tests for you. Make sure you install the dependencies your project needs (mysql, imagemagick, mongodb, etc) and all the gems it needs. Run specjour just like you did on the first machine.

Back on your dev machine, run rake specjour again. This time you will see more workers in the list and your suite should run considerably faster.

There are additional options you can pass to specify project names to listen for and how many cores to use, but for most people the defaults should work fine. Those options are passed to the manager and would look something like this:

specjour --workers 4 --projects blinq,workbeast

We have seen drastic improvements in the execution times of long running test suites using specjour. But even if your suite only takes a couple of minutes to run and you only have one machine, you should still seen a nice performance gain.

That Pesky Git Error Message

If you are using git on OSX, then you have undoubtedly run into this error message at one point or another:

”.git/COMMIT_EDITMSG” 69L, 2767C written
error: There was a problem with the editor 'vim'.
Please supply the message using either -m or -F option.

This happens when you do a git commit but accidentally type :Wq when you attempt to write the file. It then bitches that you didn’t type a valid command and so you type the correct command, :wq, only to be greeted by this message in the terminal.

You do have MacVim installed though, right? If not, install it through homebrew (brew install macvim) or go here and grab it from the website: http://code.google.com/p/macvim/. Be sure to put mvim in your path so you can launch it from the command line.

With MacVim all set up, correcting this issue is actually very simple. All you need to do is symlink vim to mvim and you will never see the error again.
I put mvim in /usr/local/bin, so to create the symlink I typed:

ln -s /usr/local/bin/mvim /usr/local/bin/vim

You can put it anywhere you want as long as it is before /usr/bin/vim in your path.

 

Integration Testing with Capybara and Selenium

You may not realize this, but your javascript is full of bugs. Because of the ghetto nature of most applications’ javascript, it is easy to break things that seem unrelated to what you are currently working on. It can be tough to know what exactly is acting on what parts of your site, depending on who did it, when it was written, what libraries were used, etc.

A few options exist for javascript-enabled integration testing, but the easiest to set up and most reliable is still selenium. With capybara, it is simply a matter of tweaking some settings and turning it on where you need it.

To set up selenium, you can start by adding database_cleaner to the test group in your Gemfile.

gem 'database_cleaner', ‘0.5.2’

Now you need to add the following code to spec/support/database_cleaner.rb.

require 'database_cleaner'

Spec::Runner.configure do |config|

  DatabaseCleaner.strategy = :truncation

  config.before :each do
    Capybara.reset_sessions!
    DatabaseCleaner.clean
  end

end

Then, you need to turn off transactional fixtures in your spec_helper.rb.

config.use_transactional_fixtures = false

To set up a specific test to use the selenium driver, you just need to put before and after blocks at the start of your test.

before(:all) { Capybara.current_driver = :selenium }
after(:all) { Capybara.use_default_driver }

Now when you run this test, you will see a Firefox window pop and the browser will dance for you. You shouldn’t need to change anything else in your tests for it to work thanks to the capybara dsl.

Integration Testing Setup with RSpec and Capybara

Also see updated post using RSpec 2

Integration testing is critical for modern web applications. It provides a chance to exercise all the features of your site in a real or mock browser and make sure that everything still work correctly with any changes made. It plays a key role in insuring the quality of your application and that your users have as bug-free an environment as possible.

Integrating it into your existing rspec setup is a breeze, too. I am a big fan of Jonas Nicklas’ Capybara (http://github.com/jnicklas/capybara), which is a great gem that lets you mix and match web drivers over a common interface and allows you to test specific functions of your site with the appropriate platform. You can easily use it directly with RSpec to write a comprehensive integration test suite for your application.
Let’s get started by running rake. If you don’t have a passing test suite right now, then shame on you! You need to fix that before doing anything else. If you don’t have a test suite at all, then...well...thanks for reading this and hopefully you will get off to a good start when you’re done.

Once you are all green, setup the latest capybara and rspec gems. Update your Gemfile with the necessary includes.

source :gemcutter

...

group :test do
  gem 'capybara', '0.3.7'
  gem 'rspec-rails', '1.3.2'
  gem ‘factory_girl’, ‘1.2.4’

  ...
end

Run bundle install --relock. If you are updating or installing RSpec for the first time, be sure to run the generator to get your configuration up to date:

script/generate rspec

One last piece of setup, we need to update the spec_helper.rb with the settings for Capybara.

require 'capybara/rails'
require 'capybara/dsl'

Spec::Runner.configure do |config|
  config.use_transactional_fixtures = true
  config.use_instantiated_fixtures = false
  config.fixture_path = RAILS_ROOT + '/spec/fixtures/'

  config.include(Capybara, :type => :integration)
end

Just to make sure all this is working properly, go ahead and run rake again. If this doesn’t work, you need to fix whatever the problem is before moving on.

Now on to Capybara! By default it uses the rack-test driver, which is similar to webrat in that it is headless and does not run javascript. Unless you have a very javascript intensive app, the rack-test driver should handle the majority of your test cases. The really cool thing about Capybara is that you can use another driver like selenium when you need javascript and rack-test when you don’t.

Now let’s test a simple sign in process. Create a file called spec/integration/guest_signs_in_spec.rb and fill it in like so. Be sure to change the field, button, and link labels to match your actual screens.

require ‘spec_helper’

context ‘as a guest on the sign in page’ do

  #Make sure your factory generates a valid user for your authentication system
  let(:user) { Factory(:user) }

  #Browse to the homepage and click the Sign In link
  before do
    visit root_path
    click ‘Sign In’
  end

  context ‘with valid credentials’ do

    #Fill in the form with the user’s credentials and submit it.
    before do
      fill_in ‘Email’, :with => user.email
      fill_in ‘Password’, :with => ‘password’
      click ‘Submit’
    end

    it ‘has a sign out link’ do
      page.should have_xpath(‘//a’, :text => ‘Sign Out’)
    end

    it ‘knows who I am’ do
      page.should have_content(“Welcome, #{user.email}!”)
    end

  end

  context ‘with invalid credentials’ do

    #No form entry should produce an error
    before do
      click ‘Submit’
    end

    it ‘has errors’ do
      page.should have_xpath(“//div[@id=‘errorExplanation’]”)
    end

  end

end

That is a trivial example and you would typically want to make more numerous and comprehensive asserts than that, but it should get you going with your first test.

Now that you have a modern integration test setup, I expect to see the quality of your app improve dramatically! Happy TDD’ing!