Just Behave! Part 2 of 2

In the previous post, we built the steps necessary to convert our coded HelloWorld test into a NBehave test.

In this post, we will build the NBehave test runner and use the steps we built in the previous post to run our HelloWorld test. To accomplish this we need 2 things.

  • Extensions class that contains methods for kicking off and validating results of the tests.
  • Feature files.

Extensions Class

Create a new class in the HelloWorld project named Extensions.cs.

The first method we will add is the ExecuteTest method.

public static FeatureResults ExecuteTest(this string path)
{
    return NBehaveConfiguration
        .New
        .DontIsolateInAppDomain()
        .SetEventListener(new OutputEventListener(new ConsoleWriter()))
        .SetAssemblies(new[] { typeof(HelloWorld.Google).Assembly.Location })
        .SetScenarioFiles(new[] { path })
        .Build()
        .Run();
}

This code sets up and kicks off a NBehave test and returns the results.

Next we need a method to validate the results.

public static void AssertTestResults(this FeatureResults results)
{
    if (results.NumberOfScenariosFound == 0)
        Assert.Inconslusive("No scenarios have been run");
    else
        Assert.AreEqual(results.NumberOfScenariosFound, results.NumberOfPassingScenarios);
}

This code returns an inconclusive result if no scenarios have been run. If scenarios have been run, we compare the number of scenarios ran to the number of scenarios that passed to determine if our tests passed or failed.

Next we need to build our .feature files. Feature files are the files that contain the NBehave tests. They are just regular old text files with a .feature extension.

Using Windows explorer, create a file named Google.feature in the same folder as the HelloWorld project. Then in the VS solution explorer right click on the HelloWorld project and select Add -> Existing Item… and add the new file to your project.

Add the following text to Google.feature:

Feature: Google
Validate Google functionality
	Scenario: Search for "Hello World!"
		Given a Firefox browser
		When I visit http://www.google.com/
		And I search for "Hello World!"
		Then the help button exists

This text defines a feature, “Google”, whose purpose is to “Validate Google functionality”. The scenario were testing is “Search for ‘Hello World!'” and the rest of the file contains references to the steps we built in part 1 of this tutorial.

The last thing to do is to change the SearchForHelloWorld method in the Google class from a coded test to an NBehave test runner. Replace the SearchForHelloWorld method with this method.

[TestMethod]
public void SearchForHelloWorld()
{
    string test = Environment.CurrentDirectory + @"\Google.feature";
    test.ExecuteTest().AssertTestResults();
}

When you kick off the SearchForHelloWorld test, its now driven by the feature file and executed with the NBehave steps!

Just Behave! Part 1 of 2

The advantages of using Gherkin with Cucumber are well documented. This is the first of two posts that describe how to integrate NBehave (the .NET port of Cucumber) into your test project, specifically our Hello World! test project.

There are frameworks that support Gherkin for many languages, such as JBehave for java, Behave.js for javascript, or Behave for python. This post will describe how to implement NBehave, which is a C# solution.

Get the NuGet Package

The first step in implementing NBehave is to get its NuGet package. The name of the package is NBehave and it can be added to your project using the process described in the Hello World! post.

Create the NBehave steps

The biggest advantage NBehave offers is the ability to write your tests in a language that non-technical users may be able to understand.

The individual units that make up NBehave tests are called steps. To create these steps, we will add a class named Steps to our Hello World! project. Steps allow sections of code, written in C#, to be executed via strings, thus allowing tests to be written by non-technical stakeholders, such as product owners or QA engineers w/o programming experience.

We will need the NBehave library as well as the Selenium libraries to build the steps.

using NBehave.Narrator.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Support.UI;

Directly before the Steps class declaration we need to add the ActionSteps tag. This tells the architecture that this class contains steps.

[ActionSteps]
public class Steps
{
}

The first step we’ll add to the Steps class is a browser step.

[Given("a Firefox browser")]
public void LaunchFirefox()
{
    FeatureContext context = FeatureContext.Current;
    IWebDriver webDriver = new OpenQA.Selenium.Firefox.FirefoxDriver();
    context.Add("browser", webDriver);
}

This method introduces a new concept, the context. NBehave consists of three contexts, a Feature context, a Scenario context, and a Step context. A context is essentially a dictionary whose scope is limited to the element it describes (a step context’s scope is a single step).

This provides two advantages.

  1. A place to store your variables.
  2. If something within your context implements the IDisposable interface, it’s automatically disposed of once the end of the context’s scope is reached.

Next, lets add a step for navigation.

[When("I visit $url")]
public void NavigateTo(string url)
{
    FeatureContext context = FeatureContext.Current;
    IWebDriver webDriver = context.Get<IWebDriver>("browser");
    webDriver.Navigate().GoToUrl(url);
}

The first thing we do in this step, is get the driver from the context. Then we can use the .Navigate().GoToUrl() syntax to navigate the browser to the URL described in the url variable. In NBehave ‘$” annotates that the text following the ‘$’ is a variable name.

Next, the SearchFor step.

[When("I search for \"$text\"")]
public void SearchFor(string text)
{
    FeatureContext context = FeatureContext.Current;
    IWebDriver webDriver = context.Get<IWebDriver>("browser");
    IWebElement textBox = webDriver.FindElement(By.Id("gbqfq"));
    textBox.SendKeys("Hello World!" + Keys.Enter);
}

Finally, the validation step.

[Then("the help button exists")]
public void HelpExists()
{
    FeatureContext context = FeatureContext.Current;
    IWebDriver webDriver = context.Get<IWebDriver>("browser");
    webDriver.WhenThisFails(dr =>
    {
        new WebDriverWait(dr, TimeSpan.FromSeconds(5)).Until<bool>(d =>
        {
            try
            {
                IWebElement helpButton = d.FindElement(By.XPath("//span[@id='fsl']/a"));
                Assert.AreEqual("Help", helpButton.Text);
                return true;
            }
            catch
            {
                return false;
            }
        });
    }).TakeScreenShot();
}

Ninja Level

  • Create a method that gets the WebDriver from the context to avoid redundant code.

Up Next

  • We will implement the NBehave test runner and replace the code in the Google class with a .feature file.

Taking Screenshots with Selenium

The Problem:

You have some automated Selenium tests that you run periodically. Maybe these tests run as part of a continuous deployment process. Occasionally these tests will fail and you’ll find yourself analyzing error messages or stack traces that may or may not be very helpful. Wouldn’t it be nice to have some other tool available to help you with your debugging?

A Solution:

This solution provides a way of capturing screenshots of your browser, but only when your test fails. This way you don’t have a mess of screenshots to go through when its time to see why your test failed.

We are going to continue adding functionality to the project we created in the Selenium Hello World! post. We will be adding a new class FailureTasks. Place the class before the last squiggly bracket in Google.cs.

public class FailureTasks
{

}

Failure tasks will have two global variables, a webdriver and a function.

private IWebDriver _webDriver;
private Action<IWebDriver> _function;

Next, lets add the mechanism which actually takes the screenshot. In addition to the OpenQA.Selenium library. We will also need to import the System.IO and System.Drawing.Imaging libraries.

using System.IO;
using System.Drawing.Imaging;
public void SaveScreenshot() {
    Screenshot ss = 1)ITakesScreenshot)_webDriver).GetScreenshot();

    ss.SaveAsFile(
        Path.Combine(
            Environment.CurrentDirectory,
            string.Format("{0} - {1}.png",
                "Hello World",
                "Google",
            ImageFormat.Png);
}

Now, lets build the mechanism that takes the screenshot, but only when something in the _function global variable fails. This is done easily with a try-catch.

public void TakeScreenShot() {
    try {
        _function(_webDriver);
    }
    catch {
        SaveScreenshot();

        throw;
    }
}

The last thing we need for the FailureTasks class is a constructor.

public FailureTasks(IWebDriver webDriver, Action<IWebDriver> function)
{
    _webDriver = webDriver;
    _function = function;
}

We now have everything we need to take our screenshot. We just need to wrap the code in SearchForHelloWorld in a FailureTask object by using a lambda expression and tack on the TakeScreenShot method.

public void SearchForHelloWorld()
{
    IWebDriver webDriver = new OpenQA.Selenium.Firefox.FirefoxDriver();
    new FailureTasks(webDriver, f =>
    {
        webDriver.Navigate().GoToUrl("http://www.google.com/");
        IWebElement textBox = webDriver.FindElement(By.Id("gbqfq"));
        textBox.SendKeys("Hello World!" + Keys.Enter);
        new WebDriverWait(webDriver, TimeSpan.FromSeconds(5)).Until<bool>(dr =>
        {
            try
            {
                IWebElement helpButton = dr.FindElement(By.XPath("//span[@id='fsl']/a"));
                Assert.AreEqual("Help", helpButton.Text);
                return true;
            }
            catch
            {
                return false;
            }
        });
    }).TakeScreenShot();
    webDriver.Dispose();
}

The test will now execute as it did before, but now, if the test fails, we will be able to find a screenshot of the browser at the time of failure in the HelloWorld project’s bin/Debug folder. You can make the test fail by removing its WebDriverWait.

Ninja Level

  • One of the things you can do to clean this up is to create a method that creates the FailureTasks object for you. If you do this, you can then use the screenshot mechanism like this:
webDriver.WhenThisFails(dr => {/*some function*/}).TakeScreenshot();

References   [ + ]

1. ITakesScreenshot)_webDriver).GetScreenshot(); ss.SaveAsFile( Path.Combine( Environment.CurrentDirectory, string.Format("{0} - {1}.png", "Hello World", "Google"

The WebDriverWait

It won’t take very long for an automated tester to realize that their test become brittle when dealing with dynamic web pages. Elements the tester is trying to manipulate will change in the middle of your tests causing exceptions.

Take, for example, the test we created in the Hello World! post. The Help button is one of the last things to load on the Google search page.

Lets test that the text of the Help button is indeed “Help”. Add the following to the HelloWorld test right before the WebDriver disposal.

IWebElement imagesLink = webDriver.FindElement(By.XPath("//a[[text()='Images for Hello World!'\]]"));
Assert.AreEqual(true, imagesLink.Displayed);

Run the test and you’ll find that you receive a StaleElementReferenceException.

You may be tempted to use something like c# threading functionality to “pause” the test and wait for a dynamic website to settle. This should be avoided at all cost. “Pauses” of a constant length will waste valuable resource time and you’ll find that your tests still fail occasionally.

Instead, perform a “pause” of dynamic length. The WebDriverWait can be used for this purpose.

The WebDriverWait class is contained in the OpenQA.Selenium.Support.UI library, so import it in addition to the traditional Selenium library. The Support.UI library is contained in the Selenium WebDriver Support Classes nuget package.

using OpenQA.Selenium;
using OpenQA.Selenium.Support.UI;

Now, we can wrap the code we added to HelloWorld in a WebDriverWait.

new WebDriverWait(webDriver, TimeSpan.FromSeconds(5)).Until<bool>(dr =>
    {
        try
        {
            IWebElement imagesLink= dr.FindElement(By.XPath("//a[[text()='Images for Hello World!']]"));
            Assert.AreEqual(true, imagesLink.Displayed);
            return true;
        }
        catch
        {
            return false;
        }
    });

The WebDriverWait constructor takes two parameters.

  • IWebDriver webDriver – this is your WebDriver object.
  • TimeSpan Timeout – this is a time span which describes the maximum amount of time we will wait for the test to pass.

The Until method takes one parameter, a function. This syntax is referred to as a lambda expression.

The variable ‘dr’ becomes a reference to the calling object. In our case, the WebDriverWait.

What’s going to happen is code between the squiggly brackets is going to loop until either it returns true or the timeout is exceeded. We want to avoid throwing exceptions during this time so we catch all exceptions and return false instead.

With this technique, you can “wait” for an element to appear without wasting resource.

Ninja Level:
  • Make your timeout a global variable. This way if you need to increase/decrease your timeout, you only need to change it in one place.
  • You can change the object between the brackets to that of another object type. In this case, the function loops until a non-null value is returned.

Selenium Hello World

This is the first of a series of posts regarding Selenium.

This tutorial will be a step by step description of how to build a simple automated Selenium test.

Context:

  • C#

Prerequisites:

  • Anyone should be able to follow along with this tutorial.
  • Visual Studio (VS)

The Tutorial

Create the project

  • Open VS and on the menu bar: File -> New -> Project…
  • On the LHS select Visual C#.
More info
Selenium contains support for Java, C#, Ruby, Python, and Javascript. This tutorial deals specifically with C#.

 

  • Select “Class Library”.
  • Name the project “HelloWorld”, specify a location, and click OK.

Nuget

To avoid having to manually add the Selenium libraries to our project, we’ll use Nuget to automatically download them.

More info
Leveraging Nuget has several advantages. It allows you to not have to keep 3rd party libraries in source control and it allows for automatic updates when 3rd parties release new versions.

 

  • In the Solution Explorer, right-click References and select Manage Nuget Packages… from the right-click menu.
  • In the upper right corner of the new screen, search for Selenium.
  • Select Selenium WebDriver and click Install.

We’ll also need the VS quality tools library. Its library is downloaded by default, but you still need to add a reference to it.

  • In the Solution Explorer, right-click References and select Add Reference…
  • In the upper right corner of the new screen, search for UnitTest.
  • Check the check box for one of the Microsoft.VisualStudio.QualityTools.UnitTestFramework and click OK.

Hello World!

  • In the Solution Explorer, right-click Class1.cs and select Rename from the right-click menu.
    • Lets give the class a more descriptive name, Google.cs.
    • Upon changing the name, you will receive a popup asking if you would like to update all references to Class1. Click Yes.
More info
A class is a computer programming term that describes a file which contains code. The code contained within could serve many purposes.

 

  • ┬áIn the Solution Explorer, double-click Google.cs to view it.
  • The Google.cs file contains 3 parts: imports, namespace, and class.
Imports
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
  • Imports consist of the keyword “using” and then the library you would like to use. To leverage Selenium we need to import it, as well as the unit testing library.
using OpenQA.Selenium;
using Microsoft.VisualStudio.TestTools.UnitTesting;
Namespace

The namespace (in this case HelloWorld) is used to describe a library. If someone wanted to leverage the code contained in HelloWorld in their own code, they would import it by:

using HelloWorld;
Class

The class (in this case Google) is used to describe what the code in this file is for. Also, if another developer wanted to reference our code, he would use the class name.

HelloWorld.Google

Above the line which declares the Google class, add the following line so that it looks like this:

[TestClass]
public class Google

This signifies to VS this class contains tests.

Method

We will now create a method that navigates to http://www.google.com/, and searches for “Hello World!”.

Between the squiggly brackets under the Google class enter:

[TestMethod]
public void SearchForHelloWorld()
{
    IWebDriver webDriver = new OpenQA.Selenium.Firefox.FirefoxDriver();
    webDriver.Navigate().GoToUrl("http://www.google.com/");
    IWebElement textBox = webDriver.FindElement(By.Id("lst-ib"));
    textBox.SendKeys("Hello World!" + Keys.Enter);
    webDriver.Dispose();
}
[TestMethod]

This signifies to VS this method contains a test.

public void SearchForHelloWorld()

This is the method declaration.

  • public
    • This controls access. Public means anyone can use it.
  • void
    • This is the method’s return value. Void means it returns nothing.
  • SearchForHelloWorld
    • This is the name of the method.
  • ()
      If the method required any parameters, they would be contained between the parenthesis.
IWebDriver

IWebDriver is an object which contains all the accessors and methods you need to read and manipulate a web browser.

This line opens a new Firefox window.

IWebDriver webDriver = new OpenQA.Selenium.Firefox.FirefoxDriver();

This line navigates the browser to http://www.google.com/.

webDriver.Navigate().GoToUrl("http://www.google.com/");

This line gets the element from the browser who’s id is “gbqfg”. You can obtain an element’s id by right clicking on it in your browser and selecting Inspect Element from the right-click menu.

IWebElement textBox = webDriver.FindElement(By.Id("gbqfq"));

This line sends the text “Hello World!” to the text box and clicks Enter.

textBox.SendKeys("Hello World!" + Keys.Enter);

This line closed the Firefox browser and frees up its resources to other processes.

webDriver.Dispose();

Build and Run

You can now right-click on HelloWorld in the Solution Explorer and select Build from the right-click menu.

To view the Test Explorer, on the menu bar: Test -> Windows -> Test Explorer

After the project is done building, the test should be available in the test explorer. Right-click it and select run or debug and enjoy watching your test run.

Up next:

  • Building your Selenium test architecture.