Thursday, May 31, 2007

How to programmatically submit a POST form using IronPython

Sometimes you may want to download a web page that is a result of submitting a form that uses a POST method. The code below shows how to do that using IronPython.

  1. prepare a request object, that is created for a given URI.
  2. write the PARAMETERS string to the request stream.
  3. retrieve the response and read from its stream.


URI = 'http://www.example.com'
PARAMETERS="lang=en&field1=1"

from System.Net import WebRequest
request = WebRequest.Create(URI)
request.ContentType = "application/x-www-form-urlencoded"
request.Method = "POST"

from System.Text import Encoding
bytes = Encoding.ASCII.GetBytes(PARAMETERS)
request.ContentLength = bytes.Length
reqStream = request.GetRequestStream()
reqStream.Write(bytes, 0, bytes.Length)
reqStream.Close()

response = request.GetResponse()
from System.IO import StreamReader
result = StreamReader(response.GetResponseStream()).ReadToEnd()
print result

Wednesday, May 30, 2007

How to download a web page using IronPython

Downloading a web page is a common programming task. Here are two snippets of code that show how to do it with IronPython. The code was tested on Windows and Mac (with Mono).

Using the WebClient class

from System.Net import WebClient
dataStream = WebClient().OpenRead('http://google.com')
from System.IO import StreamReader
reader = StreamReader(dataStream)
result = reader.ReadToEnd()
print result

Using the WebRequest and WebResponse classes

from System.Net import WebRequest
request = WebRequest.Create("http://google.com")
response = request.GetResponse()
responseStream = response.GetResponseStream()
from System.IO import StreamReader
result = StreamReader(responseStream).ReadToEnd()
print result

Monday, May 28, 2007

Selenium on Rails in 5 minutes

Introduction
Selenium is a very good framework for testing web applications. It's an ideal tool to use when you want to test your Rails apps from a user perspective. There is a Selenium on Rails plugin that simplifies creating and running Selenium tests.

Step 1. Create a Rails app and install the Selenium on Rails plugin

rails selenium_rocks
cd selenium_rocks
script/plugin install http://svn.openqa.org/svn/selenium-on-rails/selenium-on-rails

(it may take a while)

Step 2. Configuration
Create vendor/plugins/selenium-on-rails/config.yml file and paste the following:

environments:
- test
browsers:
safari: '/Applications/Safari.app/Contents/MacOS/Safari'
#firefox: 'c:\Program Files\Mozilla Firefox\firefox.exe'
#ie: 'c:\Program Files\Internet Explorer\iexplore.exe'

Change the browser path to point to your browser.

Step 3. Create a Selenium test

script/generate selenium welcome_page_test

Edit test/selenium/welcome_page_test.sel. Replace the existing content with the following:

setup
open '/'
assert_title 'Ruby on Rails: Welcome aboard'

Step 4. Start the server in a test environment
run in a new terminal:

script/server -e test

Step 5. Run the Selenium test

rake test:acceptance

You should see the following output:

1 tests passed, 0 tests failed

Congratulations!

More information:
Selenium On Rails website
Full-stack Web App Testing with Selenium and Rails (RailsConf 2007)

Friday, May 11, 2007

rcov for rails and how to analyze its reports

rcov is a code coverage tool for Ruby.

There are at least two reasons why it's worth using code coverage tools in your application.
1. It shows you which area of your code is untested.

This information can give you some clue, where it might be worth to cover the code with tests.

2. Gives you information about the code that is never executed.

You run your tests and the code coverage tool reveals a code that was not executed. It either means that this part was not tested or that the code is actually never used by your application. If it's the second case then you may simplify your codebase by removing the code. Don't worry, if you think you will later need it you can always find it your Subversion history. You use version control system, don't you?

rcov for Rails - installation

First, you install rcov gem.

gem install rcov

Then, you go to your application directory and install rcov for rails plugin:

./script/plugin install http://svn.codahale.com/rails_rcov

The plugin extends your Rake by adding new tasks like:

rake test:units:rcov
rake test:functionals:rcov

When you run 'rake test:units:rcov', it first runs all of your unit tests and then creates a coverage report. You can find it in the coverage/units directory.
It also outputs a simple text report:

5 tests, 13 assertions, 0 failures, 0 errors
+----------------------------------+-------+-------+--------+
| File | Lines | LOC | COV |
+----------------------------------+-------+-------+--------+
|app/controllers/application.rb | 7 | 3 | 100.0% |
|app/helpers/application_helper.rb | 3 | 2 | 100.0% |
|app/models/word.rb | 21 | 18 | 100.0% |
+----------------------------------+-------+-------+--------+
|Total | 31 | 23 | 100.0% |
+----------------------------------+-------+-------+--------+
100.0% 3 file(s) 31 Lines 23 LOC

How to analyze rcov results?

That's a result for the current code of our Words application. The only line that is interesting here is the one for word.rb. It says that the tests fully cover the Word class, which is a good thing.
When I run 'rake test:functionals:rcov' I get the following:

1 tests, 2 assertions, 0 failures, 0 errors
+-----------------------------------+-------+-------+--------+
| File | Lines | LOC | COV |
+-----------------------------------+-------+-------+--------+
|app/controllers/application.rb | 7 | 3 | 100.0% |
|app/controllers/words_controller.rb| 6 | 5 | 100.0% |
|app/helpers/application_helper.rb | 3 | 2 | 100.0% |
|app/helpers/words_helper.rb | 2 | 2 | 100.0% |
|app/models/word.rb | 21 | 18 | 22.2% |
+-----------------------------------+-------+-------+--------+
|Total | 39 | 30 | 53.3% |
+-----------------------------------+-------+-------+--------+

First, let's have a look at the second line, which says that we have 100% test coverage for the word_controller. That's nice. Have a look at the last line, related to the word.rb file. Why do we have only 22.2% here? Is that a problem? Does it mean that we have untested code? The answer is 'no'. We shouldn't be bothered about this line. Remember that test:functionals are tests for the controllers and this what we focus our tests on. We don't want to care about model classes here, they are already tested in test:units suite. In our case the reason for a low coverage is that we mock the Word.random method in our controller's tests. The real Word class is never called.

Summarizing, all we should care about in my opinion are results of coverage analysis for model classes when we run test:units and the results for controllers when we run test:functionals.
I suggest you give rcov a try. It should take you only 5 minutes to discover how your tests cover the code of the application. Who knows, maybe you can find some code that was never executed?

Thursday, May 10, 2007

... and some more TDD steps with Rails

We were working together with Marcin on the Words application. It was a lot of fun. We were pair programming for about two hours. First, we started by going through the last TDD steps again. That was good as an exercise and also it was a good introduction for Marcin (he was driving the keyboard during this part) to Mac, TextMate, Rake and Autotest. After we repeated all the steps, we decided that we want to have a way of feeding our database with more words. The idea was to have a form with a text box where a user could paste any kind of text. The text would then be parsed into words which are put into the database.
It was obvious that we need some kind of Word.add_content method. So we started with tests for that method. The final result of those tests is as following:

def assert_count_after_add(count, content)
Word.add_content(content)
assert_equal count, Word.count
end

def test_add_content
assert_count_after_add 2, ""
assert_count_after_add 3, "single"
assert_count_after_add 5, "two words"
assert_count_after_add 9, "four wordzz are funny"
assert_count_after_add 10, "duplicate duplicate"
end

As you can see there is a custom assertion. We also test that if there is already a word in the database we don't want it to be added again.
The implementation for the add_content:

def self.add_content content
if not content.empty?
content.split(" ").each {|eng_word|
if Word.find(:first,
:conditions => "eng = '#{eng_word}'") == nil
Word.new(:eng=>eng_word).save
end
}
end
end

It's not the prettiest piece of code but it should be fine for now.
Oops, it looks like we don't have a test for the fact that words are saved with their english translation only...
After implementing the add_content method we realized that now our database could be filled with words that are not translated (pl field is empty). It means we want to change the implementation of the 'Word.random' method so that it only displays words that are already translated. There is no point in displaying the english version only...
It sounds like we need word.translated? method. Let's write some tests:

def test_translated
assert !Word.new.translated?
assert (Word.new :pl=>"tak", :eng=>"yes").translated?
assert !(Word.new :pl=>"tak").translated?
assert !(Word.new :eng=>"yes").translated?
end

The implementation is simple:

def translated?
pl and eng
end

Now, it's time for adding a test for the fact that Word.random only returns translated words. Again, we call the test several times so that we can assume it works. We can probably refactor it later to some nicer way. We add a word to the database and then assert that it wasn't chosen even after 100 calls. Any ideas how to test it better?

def test_use_words_with_translation
Word.new( :pl=>'tak', :eng=>'yes').save
Word.add_content("hello")
randoms = []
100.times {randoms << Word.random.eng}
assert (randoms.include? "hello") == false
end

The new implementation looks like that:

def self.random
(Word.find(:all).select {|word|word.translated?}.sort_by {rand})[0]
end

Unfortunately, there was no time to create a user interface for adding a content. Sounds, like a nice topic for the next session.

Wednesday, May 9, 2007

15 TDD steps to create a Rails application




Hi,

Testing Rails applications is my passion for over 7 years now. If you sign up to this newsletter you will receive exclusive information about everything related to Ruby unit testing, Rails acceptance testing, JavaScript testing, testable architectures, TDD, BDD and good OO design. 


  Subscribe to the Testing Rails mailing list


Introduction

Several times recently, I have been asked how to develop a Rails application using the Test Driven Development approach. I'm not an expert here, but I've put together some notes on how to start working on a Rails application whilst being test-driven all the time.
As an example I will use a word-learning web application. The simplest use case is to display a random word object (with its Polish translation) from the database.
Every time we refresh we want to see a different word.

1. Create a new Rails application
rails my_app
cd my_app

Run tests with 'rake test'. It fails due to missing database configuration.

2. Set up the databases - config/database.yml
The code below assumes sqlite3 databases.
development:
adapter: sqlite3
database: db/my_app_development.sqlite

test:
adapter: sqlite3
database: db/my_app_test.sqlite

'rake test' now runs fine.

3. Create a Word class with a corresponding unit test
script/generate model Word

4. Write a unit test for the Word class. Edit the test/unit/word_test.rb.
def test_word_is_english_and_polish
word = Word.new :eng=>'never', :pl=>'nigdy'
assert_equal 'never', word.eng
assert_equal 'nigdy', word.pl
end

'rake test' now fails due to missing words table.

5. Edit db/migrate/001_create_words.rb
We are using a migration here in order to create a table. It's a recommended way of dealing with database changes.

def self.up
create_table :words do |t|
t.column :eng, :string
t.column :pl, :string
end

Word.new(:eng=>'yes', :pl=>'tak').save
Word.new(:eng=>'no', :pl=>'nie').save
Word.new(:eng=>'everything', :pl=>'wszystko').save
end

def self.down
drop_table :words
end

The sample words that we are adding use Word.new .. lines, will be added to the development database. It's important to distinguish the 'test' and 'development' database. The first one is only used during tests. The latter is used by default when you start the application.

Apply the migration with 'rake db:migrate'.

'rake test' now succeeds with the following:
'1 tests, 2 assertions, 0 failures, 0 errors'


6. Fixtures and test for word.random. Edit word_test again.
It's not easy to test a method which behaves randomly. Let's assume that it's enough to test that if we have only two words in our database then one of them should be called at least once per 10 calls.
fixtures :words
def test_random
results = []
10.times {results << Word.random.eng}
assert results.include?("yes")
end
Note the 'fixtures :words' line. Edit the 'words.yml' file.
yes:
id: 1
pl: 'tak'
eng: 'yes'
no:
id: 2
pl: 'nie'
eng: 'no'
This will be loaded to the test database before every run of tests. 7. Implement the Word.random method
def self.random
all = Word.find :all
all[rand(all.size)]
end
Warning: The code above could be slow for many words in a database (we retrieve all words only for selecting a random element). It's good enough for our needs. 8. Generate the Words controller with a 'learn' action
script/generate controller Words learn
9. Write a test for the learn method Just as there is a one-to-one ratio between unit tests and models, so there is between functional tests and controllers. The Controller's responsibility is to retrieve objects from the Model layer and pass them to the View. Let's test the View part first. We use the 'assigns' collection which contains all the objects passed to the View.
def test_learn_passes_a_random_word
get 'learn'
assert_kind_of Word, assigns('word')
end
10. Make the Test Pass
def learn
@word = Word.new
end
11. Write more tests in the words_controller_test How can we test that controller uses the Word.random method? We don't want to duplicate the tests for the Word.random method. Mocks to the rescue! We will only test that the controller calls the Word.random method. The returned value will be faked with a prepared word. Let's install the mocha framework:
gem install mocha
Now we can use 'expects' and 'returns' methods. 'expects' is used for setting an expectation on an object or a class. In this case we expect that the 'random' method will be called. We also set a return value by using 'returns' method. Setting a return value means faking (stubbing) the real method. The real Word.random won't be called. If an expectation isn't met the test fails.
require 'mocha'

def test_learn_passes_a_random_word
random_word = Word.new
Word.expects(:random).returns(random_word)
get 'learn'
assert_equal random_word, assigns('word')
end
'rake test' now fails. The Word.method wasn't called. 12. Rewrite the implementation
def learn
@word = Word.random
end
'rake test' now passes. 13. Test that a word is displayed: Extend the existing test with assert_tag calls.
def test_learn_passes_a_random_word
random_word = Word.new(:pl=>'czesc', :eng=>'hello')
Word.expects(:random).returns(random_word)
get 'learn'
assert_equal random_word, assigns('word')
assert_tag :tag=>'div', :child => /czesc/
assert_tag :tag=>'div', :child => /hello/
end
14. Implement the view - learn.rhtml
<div>
<%= @word.eng %>
<%= @word.pl %>
</div>
15. Manual testing
script/server
Go to 'http://localhost:3000/words/learn'. Refresh several times. Related articles ... and some more TDD steps with Rails Testing Rails controllers with mock objects If you want to read more about testing in Rails go to the Guide To Testing The Rails.

If you read this far you should Follow andrzejkrzywda on Twitter and subscribe to my RSS