Testing Evidence with Selenium IDE

Recently I ran into a fan. My fan! I still can’t contain my surprise! I never expected that I would have fans. (OK, just one so far :-))

He asked me if there was a way to save the test case in a way that shows which commands have passed and which commands have failed. I asked him why on earth would he want to do such a thing and not use the -htmlsuite option of the Selenium remote control. To cut a lengthy discussion short, there is a real use case for this. There are companies that require testers to keep track of the test results for process or legal purposes. Some of the smart ones have started using Selenium IDE to reduce their effort. While these testers are not programmers, Selenium IDE helps them to save tons of time and effort. There are times when they cannot use the -htmlsuite and would benefit from test results exported from the Selenium IDE itself.

While my File Logger plug-in really helps to provide a log file with all the details of executed commands. Screen shots are another option, but quickly become a pain when your test case exceeds the screen size. A HTML version of the test case showing which tests have passed and which have failed would be a great help to easily digest the results. Being the pragmatic type, I told him that the answer to his question was simple and it was easily possible in Selenium IDE — if you know the right internals. It is as simple as copy the HTML format, throw a splash of colours with a style sheet and mix in the results.

Here is a step by step guide to do it.

1. Create a new format based on the built-in HTML formatter:-

Open the Options dialog and Formats tab then click on HTML and finally click the Source button. From the source dialog, copy all the source text and click the Cancel button to close the source dialog. Back in the Options dialog, click the Add button to add a new format and paste the source to replace the shown source. Before clicking the Save button, give the name of the format as Evidence.

2. Add a set of styles for providing colours:-

You will need to close the Options dialog by clicking the OK button and open it again to see the Evidence format. Click on the Evidence format, in the formats tab. Now in the Template for new test html file textbox just before the </head> tag, insert the following:-

<style>
.failed {color: red; background-color: #ffcccc;}
.passed {color: black; background-color: #ccffcc;}
.done {color: black; background-color: #eeffee;}
</style>

3. Add the results to the command:-

Next, in the Template for command entries in the test html file textbox, replace the <tr> with the following:-

<tr class="${command.result}">

And click on the OK button to save the changes.

From now on, after you have run a test case, you can simply export it (File menu – Export Test Case As menu item) using the Evidence formatter we just created. This will save a test case that will show nicely coloured rows when you open it in a browser similar to the way you see it in the Selenium IDE. Just don’t forget to give the extension as .htm.

I would also like to point out the genius of the people who designed the Selenium IDE. I think that providing this kind of flexibility is an application is sheer genius. Great work guys!

I think the above process is really simple and works reasonably well for single test cases. If all this is sounds too complicated or you are efficient (or lazy :-P), worry not! I will soon release a Test Results plug-in for Selenium IDE that will do all this and also allow you to export the test results for an entire suite in a single file without having to do any changes mentioned above.

And if you are my fan, don’t forget to let me know