In my last blog post I highlighted the two problems that I have with Cucumber for acceptance testing:
- It's difficult to get the customer to write/read Cucumber scenarios
- Cucumber scenarios most of the time are procedural programming.
The story
Being aware of the problems mentioned above I started experimenting with my side project to find a better way. I prefer trying out new ways on my side project instead of on the client work. The project is a social network app for board game geeks. As with every social app the interaction between users is quite high. People can vote for good games, comment on other people votes, write game reviews etc. I started writing tests in the following format:
user = TestUser.new
user.visit_main_page
user.click "Login"
user.fills_in :login, :with ="andrzej"
As you see it's a fairly normal Ruby code with API similar to what Capybara/Webrat offers out of the box. In fact, I use Steak with Capybara.
Ruby code
When you compare it to the Cucumber style, the main difference is that it's Ruby code. If it's so hard to convince the customers to write scenarios then I see no point in inventing and using a new language. We all love Ruby.
Programmer-friendly
If it's mostly programmers who read the scenarios, then Ruby is the best choice. It's programmer-friendly and I think many programmers would parse the Ruby code above more easily than the Cucumber way:
Given a new user
When he visits the main page
And he follows "Login"
And he fills in "Login" with "Andrzej"
..
User object
In most apps the concept of User is very important. Why not making it an object? It simplifies a lot and you can hide some implementation details in the (Test)User class. You can create methods and reuse the implementation in an easier (IMO) way than reusing step definitions in Cucumber.
Everything is an object
In fact when you start working this way, you quickly realize that there are more objects appearing in your test world. You will probably have a Website object which can keep some implementation details, like the path to admin panel or methods responsible for setting up the application state.
You can also have objects encapsulating certain pages, if it's useful, or objects for some of your website widgets - sidebar, search_box etc.
Object oriented programming
As you see it's all about the good old OOP that we use in our production code. This solves my problem with Cucumber being too procedural. I can write code like:
user.should_see("Popular games", :inside => sidebar)
Whatever OOP style you like, you can use in your tests. Sounds good to have all pages being subclasses of Page? Write it so.
Human readable
I complain about the fact that it's hard to get customers read/write the scenarios. Cucumber is human readable, no doubts about it. However, if you look at the OOP code - is it not human readable? It takes discipline to have all your methods well named, but it's so worth it.
I claim that this kind of code is still readable to our non-programmers stakeholders.
Documentation
If you agree that it's readable, then you can actually think of your scenarios as the documentation. Nothing different here, as compared to Cucumber. If you organize your test objects nicely then you not only have a scenario view but also you can see what are the widgets responsible for or what is possible for the Guest user and what is possible for the logged-in User.
Interaction
I found it hard to implement interaction between users in a Cucumber style of testing. I mean things like this:
andrzej = TestUser("andrzej", "password")
john = TestUser("john", "password")
andrzej.logs_in
andrzej.writes_a_review("Agricola", "content")
andrzej.logs_out
john.logs_in
john.visits_reviews_page
john.should_see "content"
john.comments("nice review!")
john.logs_out
andrzej.should_receive_email("nice review!", "john")
I think it's readable and if you hide the details in methods then the scenario reads very well. You can easily introduce new users into the interaction.
Personas
Personas is a concept in which you find the different types of users who interact with your app. You can give them names, describe what they do, what matters to them etc. This concept can help you think about new features in terms of which persona would use the feature in what way.
In your scenarios you can then use the persona to highlight which aspect of the feature you test. If you worry about security of the new feature then have a user in mind who can hack into your website and write scenarios to ensure it's not possible.
little_bobby_tables.visits_main_page
little_bobby_tables.fills_in :login, :with => "Robert'); DROP TABLE Users;--")
db.should_have_table("users")
Common.rb
When I started with this kind of acceptance testing I didn't think it was so useful at first. When I showed it to my colleagues they quickly started using it as well. Special thanks to Paweł Pacana, who used it in our Ruby User Group website and published it to Github. Since then we started using this approach in our client projects and it works really well. I think it brings back the joy of writing acceptance tests.
I call this approach "object oriented acceptance testing" but my friends started calling it "common.rb" as the main test file that I used. Here is an excerpt from the first version of my common.rb file:
class TestUser
attr_accessor :steak
attr_accessor :email
attr_accessor :password
attr_accessor :login
def initialize(steak)
@steak = steak
yield if block_given?
self
end
def click(*args); steak.click_link_or_button(*args) end
def choose(*args); steak.choose(*args) end
def select(*args); steak.select(*args) end
def fill_in(*args); steak.fill_in(*args) end
def visit(path); steak.visit(path) end
def attach_file(*args); steak.attach_file(*args) end
def should_see(text); steak.page.body.include?(text).should == true end
def should_not_see(text); steak.page.should_not(steak.have_content(text)) end
def should_receive_email(text)
steak.find_email(@email, :with_text => text).should_not == nil
end
If you want to see more, look here:
All of the scenarios are kept in spec/acceptance, so look around how it works.
Not a gem, an approach
I was asked to release it as a gem. After some thinking I decided not to do that, as this would make it limiting. I prefer to think about it as an approach, not a specific technology. It all comes down to:
Use object in your acceptance tests
The technology I choose is Capybara + Steak. Steak is a very simple DSL around RSpec. Since I started Capybara already provided a replacement for Steak, so it's no longer needed.
This approach is not web specific. In fact, we have a project which consists of a server written in Ruby with EventMachine which has no views but just exposes an API. We write test scenarios using objects that simulate clients connecting to the API. It works very well with typical acceptance tests but also with load testing.
Summary
As you see the approach is not revolutionary and I know that I'm not the first developer who uses it. There are not so many articles about it and I hope it can become more popular. Obviously, it doesn't solve all of the problems that exist in acceptance testing.
We still need to think how to make acceptance tests run quickly or solve the problem of preparing the application state for testing. I think we can do better than using Factories or Fixtures in acceptance tests and I have some ideas for simplifying it. I will present how it works in my next blog posts. Stay tuned. Thanks for reading.
If you read this far you should
and
subscribe to my RSS
