Events in acceptance tests

I've been recently experimenting with the idea of events in the acceptance tests.

With the current approach based on Cucumber-like tools I see the following problems:

• Short scenarios share the same state but they prepare the state every time.
• Long scenarios are hard to follow
• Fixtures/Factories are in use, which makes the tests dependent on the implementation
• Slow builds

The idea is to solve some of the problems by using events.

As "events" I mean the following concept:

• Split tests into base scenarios and side scenarios
• Base scenarios can trigger certain events
• Side scenarios can declare which events they're interested in
• When the base scenario triggers an event all the side scenarios are executed

In practice it means that some tests depend on other tests. However scary it sounds I believe the reward is worth the risk.

This is going to require more discipline than "normal" tests, you need to find out what are the best base scenarios and which are fine as side scenarios. In my opinion it comes down to better understanding the business domain - which is always critical to the success of the project.

It's important to note that the side scenarios have to leave the application state unchanged, they may create some new content, but it needs to be destroyed at the end of the tests or it must be a content that doesn't influence other tests.

Let's look at an example. In an e-commerce app I think this would be a base scenario.

admin.create_new_product("a book")
user. visit_main_page
user. add_to_cart("a book")
user. confirm_order
admin.should_see_new_order("a book")
admin.confirm_the_last_order

This is the core scenario without looking at any side effects/side scenarios. The shop exists so that admin can sell products.
What are the possible side scenarios here? Several of them come to my mind:

• user can find the product using the search box
• google bot can find the product page
• admin can edit the product data
• user can recommend the product to his friend
• the product is in the "recently added" box

As you see, all of the above require that the product is already created.

Here are some tests that require an order to be created by the user:

• admins receive an email
• the user data exist in the admin panel
• the user receives an email

One way to look at it is that the requirements are implemented as events, so that:

admin.create_new_product("a book")
trigger("product:created", "a book")

user. visit_main_page
user. add_to_cart("a book")
user. confirm_order
trigger("order:created")

admin.should_see_new_order("a book")
admin.confirm_the_last_order

How can we implement an example side scenario?

With my current implementation it looks like this.

class SearchProductScenario

  def initialize(test_framework)
    test_framework.subscribe("product:created", self)
  end

  def execute(options)
    product_name = options[:product_name]
    user         = options[:user]

    user.visit_main_page
    user.fills_in "query", :with => product_name
    user.clicks("Search")
    user.should_see(product_name)
  end
end

I've got a simple TestFramework class that is responsible for instantiating all of the scenarios. It's not fully automated yet. It works with RSpec now, but it's framework independent.

We use this approach in one of our projects. It's too early to say if it works well in development. In my opinion it will help with most of the problems stated at the beginning of this post. Scenarios should be shorter and focused on the core of the feature in test. Tests should run quicker as they can share the state with their base scenarios. Fixtures or factories are no longer needed. It's all at the cost of being disciplined in writing scenarios in a certain way.

What do you think about this approach?

Some good news for people interested in this concept! We've been discussing the ideas with other DRUG (Wroclaw Ruby User Group) people and we're going to release a gem which will include this idea. The gem will contain the best things from Cucumber and Steak, so the name is natural: bbq :)

Thanks for reading!

If you read this far you should and subscribe to my RSS