The Beginning, Data Analytics, and the Future

I have just published the first release of a new data analytics tool. It is named River, http://riverui.azurewebsites.net/. It’s features are limited, but I intend for this to be the first step of many on a journey of discovery and learning.

Let’s step back a little bit. A few months ago, after attending a conference with speakers from organizations like Facebook and IBM, I became interested in data analytics. I knew I wanted to learn more, but I wasn’t sure how to get started. After some thinking, I settled on building a data analytics tool that will analyze stock and foreign exchange prices. The reason I chose stock and forex is because of the abundant availability of data.

While my focus is currently on stock and forex prices, my goal is to build a tool that can analyze many types of data. I’m not sure how exactly this is going to happen, but I have three tenants that will guide me on my quest. Every decision will be made with these three tenants in mind.

  • Follow the data
  • Make no assumptions
  • KISS

Follow the data

Success in data analytics depends on the availability and the quality of data. I will store raw data as well as processed data. I will process historical data and real-time data. Data will be processed at the request of on-demand clients as well as continuously running processes.

The goal is to have an abundance of quality data that can be used to find trends and drive decision making.

Make no assumptions

My greatest ally during this adventure will be the scientific method. I intend to look at data with fresh eyes, making no assumptions. I will run experiments on my data and from those experiments I will reach conclusions.

KISS

Keep it simple stupid. If I build a behemoth of spaghetti code that is difficult to maintain and to extend, I will have failed. The code should be easy to read. The data should be easy to read and manipulate.

I hope you will join me on my journey. Feel free to follow along here and post questions and comments. As well as checkout the River homepage as well as the AP Invent home page.

Web Deploy and Windows

I’m working a new app (more information on what and why in the near future), but part of the process for deploying the app is configuring a Windows server for Web Deploy. The instructions I’ve found on other sites haven’t been completely helpful so I thought I would provide some end-to-end instructions on how to configure Web Deploy on a Windows server. The context of these instructions is specifically for configuring a Windows server hosted by Azure, however the same instructions may be applied to a server hosted elsewhere.

What is Web Deploy?

Web Deploy is a tool provided by Microsoft for publishing .NET applications directly from Visual Studio or as part of an automated continuous deployment system.

Virtual machine creation

The first step is to create a virtual machine. The steps for creating the VM is typical. The only thing I will say is about selecting a VM size. While you may be able to get away with using an A0 sized VM for your application, it will be painful to configure the VM with so few cores and so little memory.

The strategy I used is to keep the VM size set to A0 during day-to-day activities, but when I need to configure the VM, I resize it to A2. This helps keep the cost of the VM down and keeps you from pulling out your hair during configuration.

Connecting to the VM

Once the VM is created, you will need to connect to it in order to begin configuring it. This task may be as simple as clicking the connect button on the VM dashboard page.

However, more than once, I’ve had an issue where the connect button is grayed out. The reason the connect button is grayed out is usually because the VM’s network security group is not configured to allow inbound RDP connections. If you have this issue, open the network security group for your VM and add an inbound security rule for RDP.

Add server features

Once you are connected to the VM, it’s time to install the features you will need. From Server Manager, click ‘Manage’ in the top right corner and select ‘Add Roles and Features’. This wizard will help you get the necessary features installed for your server.

Click ‘Next’ until you get to the ‘Select server roles’ screen and select ‘Web Server (IIS)’ from the provided list.

On the next screen, select ‘ASP.NET 4.6’, underneath ‘.NET Framework 4.6 Features’, to also be installed.

On the next screen, select the Application Development features you will need. The features you will need will vary, but ‘.NET Extensibility 4.6’ and ‘ASP.NET 4.6’ are typical. Also, select ‘Management Service’ to be installed.

Install Web Deploy

The next step is to install Web Deploy. To do this, download and install Web Deploy from, https://www.iis.net/downloads/microsoft/web-deploy. To make this possible, you will need to turn off IE Enhanced Security Configuration.

When installing Web Deploy, make sure to select the ‘Complete’ installation option.

Open the Web Deploy port

To leverage Web Deploy, you will need to allow connections to the Web Deploy port, 8172, both in the Azure network security group and the Windows server’s firewall.

Configure IIS

Open the IIS manager and select your web server from the Connections pane on the left hand side.  In the center pane, beneath ‘Management’, double click ‘Management Service’. In sure that the Management Services is started and that remote connections are enabled.

Test

That should do it for Web Deploy configuration. All that is left is to test it out by attempting to publish a .NET application via Visual Studio.

Hope you find this helpful and if you have any questions, please feel free to leave it in the comment section.

Unit Testing Part 2

In the last post, I started covering unit tests. Specifically, I talked about how only testing one unit at a time can help us with debugging. I also covered the basic usage of xUnit.

Understanding basic unit testing concepts can get you a long way, but before too long, you’ll find that you need some more advanced techniques. This post will cover mocking interfaces in order to test code that interacts with outside applications (such as databases).

When it comes to writing testable code, interfaces are your friends. Interfaces make your code cleaner and easier to test. Here is the classic barn yard example (adapted from https://spin.atomicobject.com/2014/04/29/code-untestable-part-2-developers/).

public class Pig {
    public void Oink() {
        Console.WriteLine("Oink!");
    }
}

public class Cow {
    public void Moo() {
        Console.WriteLine("Moo!");
    }
}

public class Rooster {
    public void Crow() {
        Console.WriteLine("Crow!");
    }
}

public class BarnYard {
    public void Noises() {
        var pig = new Pig();
        var cow = new Cow();
        var rooster = new Rooster();

        pig.Oink();
        cow.Moo();
        rooster.Crow();
    }
}

There are a couple problems with this. First of all, it can be made simpler. Wouldn’t it be easier if instead of having to call a different method for each animal, we could instead just call one for all the animals? Also, if we needed to mock this code, we would have one mock per animal. Instead, let’s implement an interface.

public interface Animal {
    public void Vocalize();
}

public class Pig : Animal {
    public void Vocalize() {
        Console.WriteLine("Oink!");
    }
}

public class Cow : Animal {
    public void Vocalize() {
        Console.WriteLine("Moo!");
    }
}

public class Rooster : Animal {
    public void Vocalize() {
        Console.WriteLine("Crow!");
    }
}

public class BarnYard {
    public void Noises() {
        var animals = new List<Animal>()
        animals.Add(new Pig());
        animals.Add(new Cow());
        animals.Add(new Rooster());

        foreach (var animal in animals) {
            animal.Vocalize();
        }
    }
}

Now the BaryYard will work with any type of animal and we only have one mock to build if we want to test it.

Let’s move to a more concrete example and use our FlightSim application and a .NET mocking framework named ‘Moq‘.

In the FlightSim.Service.FlightController class there is this method:

public bool IsDuplicateFlight(Flight flight)
{
    var flights = Get(flight.Number);

    if (flights.Any())
    {
        return false;
    }
    return true;
}

When the user creates or updates a flight, the system checks to make sure there are no other flights with the new flight’s flight number. To do this, the database is queried and duplicate flights are returned.

When we test this method, we don’t want the database to be queried because the machine running the unit test may not have access to a database. Also, testing with a database would go against our principle of only testing one thing at a time. So, instead, let’s test it using a mock.

using FlightSim.Data;
using FlightSim.Entity;
using Moq;
using System;
using System.Collections.Generic;
using Xunit;

namespace FlightSim.Service.Tests
{
    [Trait("Category", "FlightController")]
    public class WhenICreateADuplicateFlight
    {
        public bool _isDuplicate;

        public WhenICreateADuplicateFlight()
        {
            var mockContext = new Mock<IContext>();
            mockContext.Setup(c => c.Get(It.IsAny<Func<Flight, bool>>()))
                .Returns(new List<Flight>()
                {
                    new Flight("123", "Denver", "Pittsburgh", DateTime.UtcNow, DateTime.UtcNow, new Airplane("Cessna", 15, 5, 3))
                });

            var controller = new FlightController(mockContext.Object);

            var flight = new Flight("123", "Pittsburgh", "Denver", DateTime.UtcNow, DateTime.UtcNow, new Airplane("Cessna", 15, 5, 3));
            _isDuplicate = controller.IsDuplicateFlight(flight);
        }

        [Fact]
        public void IsDuplicateFlightShouldReturnTrue()
        {
            Assert.True(_isDuplicate);
        }
    }
}

You can see here, that we create an instance of the FlightController with a mocked instance of IContext. We use Moq’s ‘Setup’ method to mock the IContext’s ‘Get’ method. Now instead of connecting to and querying a database, a static list is returned to the application.

An especially powerful syntax is It.IsAny<>(). You can use this when your mocks are parameterized. It tells the mock to return the default value for parameter value. You can include a data type between the brackets to limit the parameter type.

That’s all for this post. I hope you enjoy unit testing and building mocks for your unit test.

Unit Testing the Right Way (or at least the less wrong way) Part 1

On this blog, I’ve covered automated UI testing and automated HTTP testing. This post will cover the bottom of the agile automated test pyramid, unit testing.

AATP

I will use a demo application named FlightSim to help us learn some basic unit testing techniques.

Each unit test should test one and only one unit.

In order to keep tests simple and to avoid ambiguity each test should test one and only one unit.

For example, in our FlightSim entities, we have IsValid() methods that determine if the entity is valid or not. Here is the Airplane entity.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace FlightSim.Entity
{
    public class Airplane
    {
        public Airplane() { }
        public Airplane(String name, Int32 numberOfSeats, Int32 rows, Int32 seatsPerRow)
        {
            Name = name;
            NumberOfSeats = numberOfSeats;
            Rows = rows;
            SeatsPerRow = seatsPerRow;

            var errorMessages = this.IsValid();
            if (errorMessages.Any())
            {
                throw new Exception("Invalid airplane." + Environment.NewLine + String.Join(Environment.NewLine, errorMessages));
            }
        }
        public String Name { get; set; }
        public Int32 NumberOfSeats { get; set; }
        public Int32 Rows { get; set; }
        public Int32 SeatsPerRow { get; set; }

        public IEnumerable<String> IsValid()
        {
            var errorMessages = new List<String>();

            if (String.IsNullOrWhiteSpace(Name))
            {
                errorMessages.Add("'Name' is required.");
            }
            
            if (NumberOfSeats <= 0)
            {
                errorMessages.Add("'Number of seats' must be greater than 0.");
            }
            else
            {
                if (Rows * SeatsPerRow != NumberOfSeats)
                {
                    errorMessages.Add("'Rows' and 'seats per row' does not match 'number of seats'.");
                }
            }

            return errorMessages;
        }
    }
}

It is best to write a single test for each of the validations so that when one of those tests fails, we know exactly which validation needs looking at just by looking at the name of the test.

using FlightSim.Entity;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Xunit;

namespace FlightSim.Entity.Tests
{
    [Trait("Category", "Airplane")]
    public class WhenICreateAnAirplaneWithNoName
    {
        private IEnumerable<String> _messages;

        public WhenICreateAnAirplaneWithNoName()
        {
            var airplane = new Airplane("", 1, 1, 1);
            _messages = airplane.IsValid();
        }

        [Fact]
        public void ItShouldReturnOneErrorMessage()
        {
            Assert.Equal(1, _messages.Count());
        }

        [Fact]
        public void ItShouldReturnTheCorrectErrorMessage()
        {
            Assert.Equal("'Name' is required.", _messages.First());
        }
    }
}

If we test more than one validation per test, we would have to run the test in order to determine why it failed.

xUnit

There are many frameworks out there for performing unit tests and feel free to choose the one you like the best. For this post, we will use xUnit. I believe you will find this information helpful regardless of the framework you use.

I want to take look at the test above and explain what its doing, but first I need to let you know which NuGet packages you need to build this test.

  • xunit – this contains all the binaries you need for running tests and asserting values.
  • xunit.runner.visualstudio – you need this package so that your xUnit tests display in the test explorer. You only need to add this package to one of the projects in your solution.

Traits

You will notice this attribute added to the test class.

[Trait("Category", "Airplane")]

This attribute helps you organize your unit tests. If you organize the tests in the Test Explorer by ‘trait’, you will see the tests above listed under the heading of “Category [Airplane]”.

Constructor

The constructor for this class “WhenICreateAnAirplaneWithNoName” is executed before each of the tests run. It creates the test data and then executes the IsValid() method, saving its result in a global variable to be asserted later.

Fact

You will notice this attribute added to the two methods in the class.

[Fact]

This specifies the method as a test and adds it to the Test Explorer. When you select this test to be ran, it executes the constructor then the Fact method.

The nice thing about these tests is that if you read the constructor name followed by the name of the Fact method, you can understand what the test is checking, “When I create an airplane with no name, it should return one error message”.

Up next

In the next post I will show you how implementing interfaces can make your application code more testable. Then I’ll show you how to mock those interfaces so that we can test code that interacts with other applications.

Automated HTTP Testing! Part 2 of 2

In part 1:

  • We created the data structure to hold the Person entity and also  a method to compare two people together.
  • Wrote a method for executing a GET request for a HTTP serivice.
  • Wrote a simple GetPerson test.

In this, the second part, we add an additional GET test as well as add functionality for creating data.

Get All

Sometimes, you may want to test a collection of entities. Json.NET provides the ability to serialize/deserialize collections as well. Let’s add a method to the Person entity class for getting a collection.

        public static IEnumerable<Person> GetPeople()
        {
            return JsonConvert.DeserializeObject<List<Person>>(Api.Get(new Uri(Config.Url + "api/person")));
        }

Next, we’ll use this method in our PersonTests and make some assertions.

        [TestMethod]
        public void GetPeople()
        {
            var people = Person.GetPeople();

            Assert.AreEqual(2, people.Count());
            Assert.IsTrue(people.Any(p => p.Equals(new Person
            {
                FirstName = "Joe",
                LastName = "Schmoe",
                Age = 25,
                Married = false,
                BirthDate = new DateTime(1980, 1, 1)
            })));
            Assert.IsTrue(people.Any(p => p.Equals(new Person
            {
                FirstName = "Jon",
                LastName = "Smith",
                Age = 35,
                Married = true,
                BirthDate = new DateTime(1990, 2, 1)
            })));
        }

Post

The only difference between posting and getting data is, instead of deserializing data that we’ve received from the server, we serialize the data we would like to send to the server.

For example, let’s say we would like to add this person to the system:

{
    "firstName":"Jane",
    "lastName": "Smith",
    "age": 35,
    "married": true,
    "birthDate": "03/01/1990"
}

The first thing we need to add is a Post method to the Api class.

        public static String Post(Uri uri, String payload)
        {
            using (var client = new HttpClient())
            {
                HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, uri);
                request.Content = new StringContent(payload.ToString(), Encoding.UTF8, "application/json");

                HttpResponseMessage response = client.SendAsync(request).Result;
                if (response.IsSuccessStatusCode)
                {
                    return response.Content.ReadAsStringAsync().Result;
                }
                else
                {
                    throw new System.Net.WebException(response.Content.ReadAsStringAsync().Result);
                }
            }
        }

This looks similar to the Get method except it takes two parameters. The second parameter is a string payload. This is what gets sent over the wire and is data the application under test will use to create a Person.

Next, let’s add PostPerson to the Person entity class.

        public static String PostPerson(Person person)
        {
            return Api.Post(new Uri(Config.Url + "api/person"), JsonConvert.SerializeObject(person));
        }

This looks similar to the GetPerson method, but instead of deserializing a person returned from the server, this serializes a person to be sent to the server.

With the pieces in place, we can write a test.

        [TestMethod]
        public void PostPerson()
        {
            var person = new Person
            {
                FirstName = "Jane",
                LastName = "Smith",
                Age = 35,
                Married = true,
                BirthDate = new DateTime(1990, 3, 1)
            };

            var response = Person.PostPerson(person);

            //If we were testing a real application, we would
            //execute a GET here in order to validate the POST
            //was successful.
            //Assert.IsTrue(person.Equals(Person.GetPerson("jane")));

            Assert.AreEqual("Person creation success!", response);
        }

You’ll notice the commented lines above. If we were testing a real application (not a mocked one), we would request the newly created ‘Jane’ by executing GET against http://demo3192753.mockable.io/api/person/jane and comparing the actual result with the expected.

Sad path

What if we wanted to test the error conditions of our application? We can do that too. Check out this test for requesting a person that doesn’t exist.

        [TestMethod]
        public void InvalidPerson()
        {
            try
            {
                var response = Person.GetPerson("sally");
                Assert.Fail("Expected 404 error message.");
            }
            catch (System.Net.WebException e)
            {
                Assert.AreEqual("No person exists with specified name.", e.Message);
            }
        }

We expect the Person request to fail, so if the test makes it past that line of code, we should fail the test. Once the Person request fails, the catch asserts that the error message is equal to the error message we expect.

That’s it for automated HTTP testing (for now). Want to see some different tests or have any issues? Just ask!

Automated HTTP Testing! Part 1 of 2

When it comes to testing web applications, I’ve talked a lot about running automated tests against the UI via Selenium. While automate UI tests are fun and important, they should be just part of your automated test suite.

AATP

The picture above is an example of the agile automated test pyramid. The agile automated test pyramid describes the quantities of different types of tests in your test suite.

The specific numbers are unimportant. What’s important to understand is that automated UI tests are brittle and difficult to maintain. It’s best to just use automated UI tests to test the happy path of your application. Automated HTTP tests can be used to test more complex scenarios.

To demo how automated HTTP testing works, I’ve created a new C# project, https://github.com/asphaltpanthers/HelloHttp.

There are 5 steps in writing an automated HTTP test:

  1. Identify and build data structures.
  2. Serialize data structures (POST & PUT).
  3. Execute request.
  4. Deserialize response (GET & GETALL).
  5. Execute assertions.

This post deals with executing a test that GETs data from a HTTP service.

Identify and build data structures

HTTP payloads may consist of either JSON or XML. We will use JSON because it is more widely used and easier to serialize and deserialize.

I’ve created some mock HTTP clients via mockable.io. The first one is a GET request that returns a single person.

http://demo3192753.mockable.io/api/person/joe

returns

{
	"firstName":"Joe",
	"lastName": "Schmoe",
	"age": 25,
	"married": false,
	"birthDate": "01/01/1980"
}

From this we can build our C# data structure.

using System;

namespace HelloHttp.Entities
{
    public class Person
    {
        public String FirstName;
        public String LastName;
        public Int32 Age;
        public Boolean Married;
        public DateTime BirthDate;
    }
}

I created this class in a folder named ‘Entities’ in the HelloHttp project.

Execute request and deserialize response

Since we are executing a GET we can skip the serialize step and go directly to executing the request and then deserializing the response.

To send the request we need a new class. I call this one Api and place it at the root of the project.

using System;
using System.Net.Http;

namespace HelloHttp
{
    public static class Api
    {
        public static String Get(Uri uri)
        {
            using (var client = new HttpClient())
            {
                HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, uri);
                HttpResponseMessage response = client.SendAsync(request).Result;
                if (response.IsSuccessStatusCode)
                {
                    return response.Content.ReadAsStringAsync().Result;
                }
                else
                {
                    throw new System.Net.WebException(response.Content.ReadAsStringAsync().Result);
                }
            }
        }
    }
}

This method accepts a URI and sends it over the wire via a HttpClient. Upon success, it returns the result payload as a string. Upon failure, it throws a WebException with the error text as the exception text.

To parameterize the tests so that they can run on different environments, I added an App.config.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="Url" value="http://demo3192753.mockable.io/"/>
  </appSettings>
</configuration>

And I reference this key in a class named Config.cs.

using System;
using System.Configuration;

namespace HelloHttp
{
    public static class Config
    {
        public static readonly String Url = ConfigurationManager.AppSettings["Url"];
    }
}

Next, we’ll add a method to the Person class that executes the GET and deserializes it. This code requires you include the Newtonsoft.Json NuGet package (Json.NET). Json.NET is a third party library that provides JSON serialization and deserialization.

        public static Person GetPerson(string name)
        {
            return JsonConvert.DeserializeObject<Person>(Api.Get(new Uri(Config.Url + "api/person/" + name)));
        }

This method executes a request via the Api class. Remember, the Get method returns a string, so we deserialize it into a Person object.

Execute assertions

There’s one more thing to add to the Person class before finally writing a test.

        public Boolean Equals(Person person)
        {
            return String.Equals(FirstName, person.FirstName) &&
                String.Equals(LastName, person.LastName) &&
                Int32.Equals(Age, person.Age) &&
                Boolean.Equals(Married, person.Married) &&
                DateTime.Equals(BirthDate, person.BirthDate);
        }

Although, not required, this method helps when comparing the actual Person to the expected Person.

We’re now ready to write a test. I created a folder named PredefinedScenarios and added this test.

using HelloHttp.Entities;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
using System.Linq;

namespace HelloHttp.PredefinedScenarios
{
    [TestClass]
    public class PersonTests
    {
        [TestMethod]
        public void GetPerson()
        {
            var person = Person.GetPerson("joe");

            Assert.IsTrue(person.Equals(new Person
            {
                FirstName = "Joe",
                LastName = "Schmoe",
                Age = 25,
                Married = false,
                BirthDate = new DateTime(1980, 1, 1)
            }));
        }
    }
}

We first request the person “joe” and then assert that the actual “joe” is equal to the expected “joe”.

Congratulations! You’ve written an automated HTTP test!

History of Software Testing

For my first post of 2016, I’d like to try a non-technical article. I’ve pulled together several sources and compiled this brief history of software testing.

The main source is an article by D. Gelperin and B. Hetzel entitled “The growth of software testing”.1)Gelperin, D.; B. Hetzel (1988). “The Growth of Software Testing”. CACM 31 (6): 687–695. doi:10.1145/62959.62965. ISSN 0001-0782.

Enjoy!

The Debugging-Oriented Period (pre 1956)

In the beginning, testing was focused on the hardware. On September 9, 1947, Grace Hopper (an early computing pioneer known for inventing COBOL) found the first “bug”. A moth was found in the relay of the Harvard Mark II machine. The moth was removed and taped to the page in the log book with the description “1535 Relay #70 Panel F (moth) in relay. First actual case of bug being found”; the first bug report.2)“Bug”, The Jargon File, ver. 4.4.7. Retrieved June 3, 2010.

Testing was not considered a separate part of the SDLC. A developer would “write” a program and then would “check it out” in order to get the bugs out.

In 1950, Alan Turing writes the first article that discusses software testing. The Turing test addresses the question “How would we know that a program exhibits intelligence?” The test consists of an interrogator given the task of interrogating a human player and a computer player and determining which player is which. If the interrogator cannot reliably tell the machine from the human, the machine is said to have passed the test.3)Turing, Alan (October 1950), “Computing Machinery and Intelligence”, Mind LIX (236): 433–460, doi:10.1093/mind/LIX.236.433, ISSN 0026-4423, retrieved 2008-08-18

The Demonstration-Oriented Period (1957 – 1978)

In 1957, Charles Baker distinguishes debugging from testing in a review of Dan McCracken’s book Digital Computer Programming. Program checkout composed of two goals:

  • Debugging “Make sure the program runs”
  • Testing “Make sure the program solves the problem”

Testing during this time would only focus on the happy path.

During this time, employers begin posting job listings explicitly requesting testing skills and software test engineering becomes a career path in a few companies.

The first conference on software testing is held at the University of North Carolina in June of 1972.

The Destruction-Oriented Period (1979-1982)

In 1979, Glenford J. Myers publishes the first software testing book, The Art of Software Testing and defined testing as “the process of executing a program with the intent of finding errors.” and introduces sad path testing.

By using test data that has a high probability of causing program failures, we increase the probability of detecting issues in the program.

During this time the term “testing” became associated with activities other than checkout, such as software analysis and review techniques.

The Evaluation-Oriented Period (1983 – 1987)

In 1983, the Institute for Computer Sciences and Technology of the National Bureau of Standards published Guideline for Lifecycle Validation, Verification, and Testing of Computer Software. This publication describes testing as a methodology which includes analysis, review, and testing activities.

Two national standards are published, one on test documentation (ANSI/IEEE STD 829) in 1983 and one on unit testing (ANSI /IEEE 1008) in 1987.

The Prevention-Oriented Period (1988-present)

Starting in in the late 1980s and continuing to the present, testing becomes focused on preventing program failures before they occur.

The Systematic Test and Evaluation Process (STEP) methodology introduces testing activities being performed in parallel to development activities.

Testing activities begin to include planning, analysis, design, implementation, execution, and maintenance.

It is realized that defect detection during the design phase is significantly cheaper than defect detection at the implementation phase.

Test driven development (TDD) is introduced, raising the veil between tester and programmer. Writing tests first and sharing them with programmers results in code with less bugs.

References   [ + ]

1. Gelperin, D.; B. Hetzel (1988). “The Growth of Software Testing”. CACM 31 (6): 687–695. doi:10.1145/62959.62965. ISSN 0001-0782.
2. “Bug”, The Jargon File, ver. 4.4.7. Retrieved June 3, 2010.
3. Turing, Alan (October 1950), “Computing Machinery and Intelligence”, Mind LIX (236): 433–460, doi:10.1093/mind/LIX.236.433, ISSN 0026-4423, retrieved 2008-08-18

From Dev to Production: Managing Continuous Deployment with Octopus Deploy

When it comes to continuous deployment, automation is key. Octopus deploy provides an automated solution for deploying your application, as well as performing post-deployment operations such as testing.

The purpose of this article is not to detail the features Octopus provides, but here are some highlights. For more information, check out the great documentation provided by the folks at Octopus, https://octopusdeploy.com.

  • Deployment solution for .NET applications.
  • Deploys on-premises or to the cloud.
  • Built in integration with TeamCity and TFS.

Other things I won’t cover in this article is server and tentacle installation. I will say that it is ridiculously easy.  Even a software tester can do it! ;).

Now, on to what I will cover. The goal of this article is to provide the reader with enough information to get a continuous deployment process built and running using Octopus deploy.

I’ve already setup a demo solution to showcase Octopus’ functionality. You can do the same easily. It’s simply the default .NET web API project with a cloud service, a UI test project and a performance test project.

Setup OctoPack

OctoPack is a tool that packages your application. It produces an Octopus compatible NuGet package of your application that can then be sent to the Octopus server.

Add it to your test projects via NuGet.

Configure .targets

Adding the OctoPack NuGet package creates some files in the packages folder of your solution. Navigate to packages/OctoPack.#.#.##/tools and open the OctoPack.targets file for editing.

We need to give OctoPack some information in order for it to know where to send the packages. Make the following changes:

<OctoPackPublishPackageToHttp Condition="'$(OctoPackPublishPackageToHttp)' == ''">http://youroctopusserver.com/nuget/packages</OctoPackPublishPackageToHttp>
<OctoPackPublishApiKey Condition="'$(OctoPackPublishApiKey)' == ''">API-YourAPIKey</OctoPackPublishApiKey>
<OctoPackPackageVersion Condition="'$(OctoPackPackageVersion)' == ''">$([System.DateTime]::UtcNow.ToString(`yyyy.MM.dd.HHmm`))</OctoPackPackageVersion>

To prevent OctoPack from hooking into the AfterBuild target, we should comment out the following lines.

<!--<PropertyGroup>
<BuildDependsOn>
  $(BuildDependsOn);
  OctoPack
</BuildDependsOn>
</PropertyGroup>-->

Next, we need to instruct the cloud service and test projects to execute OctoPack at the appropriate times.

Add the following to the end of the cloud service project. This instructs the cloud service project to package after the CorePublish task.

<Import Project="..\packages\OctoPack.3.0.42\tools\OctoPack.targets" Condition="Exists('..\packages\OctoPack.3.0.42\tools\OctoPack.targets')" />
<Target Name="EnsureOctoPackImported" BeforeTargets="BeforeBuild" Condition="'$(OctoPackImported)' == ''">
  <Error Condition="!Exists('..\packages\OctoPack.3.0.42\tools\OctoPack.targets') And ('$(RunOctoPack)' != '' And $(RunOctoPack))" Text="You are trying to build with OctoPack, but the NuGet targets file that OctoPack depends on is not available on this computer. This is probably because the OctoPack package has not been committed to source control, or NuGet Package Restore is not enabled. Please enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=317567." HelpKeyword="BCLBUILD2001" />
  <Error Condition="Exists('..\packages\OctoPack.3.0.42\tools\OctoPack.targets') And ('$(RunOctoPack)' != '' And $(RunOctoPack))" Text="OctoPack cannot be run because NuGet packages were restored prior to the build running, and the targets file was unavailable when the build started. Please build the project again to include these packages in the build. You may also need to make sure that your build server does not delete packages prior to each build. For more information, see http://go.microsoft.com/fwlink/?LinkID=317568." HelpKeyword="BCLBUILD2002" />
</Target>
<Target Name="ExecuteOctoPack" Condition="$(RunOctoPack)" AfterTargets="CorePublish">
  <CallTarget Targets="OctoPack" />
</Target>

Add the following to the end of the test projects. This instructs the test projects to package after the Build task.

<Import Project="..\packages\OctoPack.3.0.42\tools\OctoPack.targets" Condition="Exists('..\packages\OctoPack.3.0.42\tools\OctoPack.targets')" />
<Target Name="EnsureOctoPackImported" BeforeTargets="BeforeBuild" Condition="'$(OctoPackImported)' == ''">
  <Error Condition="!Exists('..\packages\OctoPack.3.0.42\tools\OctoPack.targets') And ('$(RunOctoPack)' != '' And $(RunOctoPack))" Text="You are trying to build with OctoPack, but the NuGet targets file that OctoPack depends on is not available on this computer. This is probably because the OctoPack package has not been committed to source control, or NuGet Package Restore is not enabled. Please enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=317567." HelpKeyword="BCLBUILD2001" />
  <Error Condition="Exists('..\packages\OctoPack.3.0.42\tools\OctoPack.targets') And ('$(RunOctoPack)' != '' And $(RunOctoPack))" Text="OctoPack cannot be run because NuGet packages were restored prior to the build running, and the targets file was unavailable when the build started. Please build the project again to include these packages in the build. You may also need to make sure that your build server does not delete packages prior to each build. For more information, see http://go.microsoft.com/fwlink/?LinkID=317568." HelpKeyword="BCLBUILD2002" />
</Target>
<Target Name="AfterBuild" Condition="$(RunOctoPack)">
  <CallTarget Targets="OctoPack" />
</Target>

We also need a NuSpec file to tell OctoPack what files to package in the cloud service Project. Create a file named OctopusDemo.nuspec in the root folder of the cloud service project and add the following text.

<?xml version="1.0"?>
<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
  <metadata>
    <id>OctopusDemo</id>
    <version>1.0</version>
    <authors>Dustin Iser</authors>
    <description>Cloud Service Package</description>
  </metadata>
  <files>
    <file src="bin\Debug\app.publish\*.*" target="" />
  </files>
</package>

We can now execute the following command from the command line and our application and it’s tests will be packaged and copied to the Octopus server.

msbuild OctopusDemo.sln /t:Publish;Build /p:RunOctoPack=true
Build a Deployment Process

Now we need to create a deployment process to utilize our newly created packages.

  • If you haven’t already, create a project named OctopusDemo on the Octopus server.
  • Once in the project, click Process on the left hand side.
  • Click “Add step” to create the deployment step.
    • Click Deploy to Windows Azure.
    • Enter the following information:
      • Step name
      • Machine roles – The role that will perform the step.
      • NuGet package – “OctopusDemo”
      • Subscription Id – Your Azure subsription id.
      • Cloud server – The name of the cloud service you would like to deploy to.
      • Storage account – The name of the storage account to use.
      • Slot – “Production” or “Staging”
      • Environments – The names of the environments you would like to deploy to.
    • Click Save.

Next we should create a release and deploy it.

  • Click “Create release” in the upper right corner.
  • Give the release a version number and insure the latest version of the OctopusDemo package is selected for the deployment step.
  • Click Save.
  • Click “Deploy to Development”.
  • Click “Deploy now”.

We can now use Octopus to deploy our application manually, but continuous deployment this is not. Let’s make the deployment process completely automated and triggered on the build process.

Implement OctopusTools

Octopus tools is a NuGet package that provides the commands necessary for automatically creating and deploying Octopus packages.

Right click the OctopusDemo solution and add OctopusTools to the solution via the Manage NuGet Packages dialog.

Add the following target to the OctoPack.targets file.

<Target Name="AfterOctoPack">
  <PropertyGroup>
    <OctopusTool>..\packages\OctopusTools.2.6.3.59\octo.exe</OctopusTool>
    <OctopusAPICommand>create-release</OctopusAPICommand>
  </PropertyGroup>

  <Exec Command="$(OctopusTool) $(OctopusAPICommand) --debug -apiKey=$(OctoPackPublishApiKey) --server=$(OctoPackPublishPackageToHttp)api --project=$(OctoPackProjectName) --deployto=Development --version=$(OctoPackPackageVersion)" />
</Target>

Also adjust the cloud service project to also run AfterOctoPack post-publish.

<Target Name="ExecuteOctoPack" Condition="$(RunOctoPack)" AfterTargets="CorePublish">
  <CallTarget Targets="OctoPack" />
  <CallTarget Targets="AfterOctoPack" />
</Target>

Now when we execute a build, a release is created and deployed automatically. Next, lets setup some testing.

Automated Tests

From the Octopus UI, navigate to the OctopusDemo Process page. Here we will add a step to automatically test the application post-deployment.

  • Click Add step.
  • Select “Deploy a NuGet package”.
    • Enter the following information.
      • A step name.
      • A machine role.
      • The NuGet package id, OctopusDemoUITests.
      • Environments.
    • Click Save.
  • Click the button in the upper right corner of the step you just created and select “Add child step”.
  • Select “Run a PowerShell script”.
    • Enter the following information.
      • A step name.
      • The PowerShell script.
      • Environments.
cd $OctopusParameters['Octopus.Action[Deploy UI Tests].Output.Package.InstallationDirectoryPath']
$testDLL = "OctopusDemoUITests.dll"
$fs = New-Object -ComObject Scripting.FileSystemObject
$f = $fs.GetFile("C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\MSTest.exe")
$mstestPath = $f.shortpath
$arguments = " /testcontainer:" + $testDLL
Invoke-Expression "$mstestPath $arguments"

Navigate to the Variables screen and create a variable named “URL”. Set it to “http://dustinsoctopusdemo.cloudapp.net/” and set its scope to Development. Now when we deploy our tests, the URL variable in the app config will be replaced with the Octopus variable.

Now when we build our application deployment and automated UI tests are ran. Next, we can utilize Lifecycles to automate performance testing.

Lifecycles
  • On the Octopus top bar, click Library.
  • On the left hand side, select Lifecycles.
  • Select the default lifecycle to edit it.
  • Set it up so that phase one contains the development environment and phase 2 contains the performance environment and phase 2 is deployed to automatically.

Now when a deployment to the development environment is successful, a deployment to the performance environment will automatically be triggered.

To run performance tests automatically, we’ll create a new step in the deployment process called Deploy Performance Tests.

  • Create a step that deploys the performance tests the same way we did with the UI tests except set the environment to Performance.
  • Use the following PowerShell script instead.
cd $OctopusParameters['Octopus.Action[Deploy Performance Tests].Output.Package.InstallationDirectoryPath']
$testDLL = "ProjectionPerf.DataMangement.dll"
$fs = New-Object -ComObject Scripting.FileSystemObject
$f = $fs.GetFile("C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\MSTest.exe")
$mstestPath = $f.shortpath
$loadTests = Get-ChildItem . *.loadtest
foreach ($loadTest in $loadTests) {
    $arguments = " /testcontainer:" + $loadTest
    Invoke-Expression "$mstestPath $arguments"
}
Conclusion

Octopus deploy provides a fun, easy way of automating your continuous deployment process. Providing automatic feedback regarding the quality of your code is a great way of streamlining your development process. Combining this with the use of lifecycles gives you the ability of gating your deployments in such a way that once a build is determined to be of high quality it can be deployed to the next phase, possibly all the way to production.

Just Behave! Part 2 of 2

In the previous post, we built the steps necessary to convert our coded HelloWorld test into a NBehave test.

In this post, we will build the NBehave test runner and use the steps we built in the previous post to run our HelloWorld test. To accomplish this we need 2 things.

  • Extensions class that contains methods for kicking off and validating results of the tests.
  • Feature files.

Extensions Class

Create a new class in the HelloWorld project named Extensions.cs.

The first method we will add is the ExecuteTest method.

public static FeatureResults ExecuteTest(this string path)
{
    return NBehaveConfiguration
        .New
        .DontIsolateInAppDomain()
        .SetEventListener(new OutputEventListener(new ConsoleWriter()))
        .SetAssemblies(new[] { typeof(HelloWorld.Google).Assembly.Location })
        .SetScenarioFiles(new[] { path })
        .Build()
        .Run();
}

This code sets up and kicks off a NBehave test and returns the results.

Next we need a method to validate the results.

public static void AssertTestResults(this FeatureResults results)
{
    if (results.NumberOfScenariosFound == 0)
        Assert.Inconslusive("No scenarios have been run");
    else
        Assert.AreEqual(results.NumberOfScenariosFound, results.NumberOfPassingScenarios);
}

This code returns an inconclusive result if no scenarios have been run. If scenarios have been run, we compare the number of scenarios ran to the number of scenarios that passed to determine if our tests passed or failed.

Next we need to build our .feature files. Feature files are the files that contain the NBehave tests. They are just regular old text files with a .feature extension.

Using Windows explorer, create a file named Google.feature in the same folder as the HelloWorld project. Then in the VS solution explorer right click on the HelloWorld project and select Add -> Existing Item… and add the new file to your project.

Add the following text to Google.feature:

Feature: Google
Validate Google functionality
	Scenario: Search for "Hello World!"
		Given a Firefox browser
		When I visit http://www.google.com/
		And I search for "Hello World!"
		Then the help button exists

This text defines a feature, “Google”, whose purpose is to “Validate Google functionality”. The scenario were testing is “Search for ‘Hello World!'” and the rest of the file contains references to the steps we built in part 1 of this tutorial.

The last thing to do is to change the SearchForHelloWorld method in the Google class from a coded test to an NBehave test runner. Replace the SearchForHelloWorld method with this method.

[TestMethod]
public void SearchForHelloWorld()
{
    string test = Environment.CurrentDirectory + @"\Google.feature";
    test.ExecuteTest().AssertTestResults();
}

When you kick off the SearchForHelloWorld test, its now driven by the feature file and executed with the NBehave steps!

Just Behave! Part 1 of 2

The advantages of using Gherkin with Cucumber are well documented. This is the first of two posts that describe how to integrate NBehave (the .NET port of Cucumber) into your test project, specifically our Hello World! test project.

There are frameworks that support Gherkin for many languages, such as JBehave for java, Behave.js for javascript, or Behave for python. This post will describe how to implement NBehave, which is a C# solution.

Get the NuGet Package

The first step in implementing NBehave is to get its NuGet package. The name of the package is NBehave and it can be added to your project using the process described in the Hello World! post.

Create the NBehave steps

The biggest advantage NBehave offers is the ability to write your tests in a language that non-technical users may be able to understand.

The individual units that make up NBehave tests are called steps. To create these steps, we will add a class named Steps to our Hello World! project. Steps allow sections of code, written in C#, to be executed via strings, thus allowing tests to be written by non-technical stakeholders, such as product owners or QA engineers w/o programming experience.

We will need the NBehave library as well as the Selenium libraries to build the steps.

using NBehave.Narrator.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Support.UI;

Directly before the Steps class declaration we need to add the ActionSteps tag. This tells the architecture that this class contains steps.

[ActionSteps]
public class Steps
{
}

The first step we’ll add to the Steps class is a browser step.

[Given("a Firefox browser")]
public void LaunchFirefox()
{
    FeatureContext context = FeatureContext.Current;
    IWebDriver webDriver = new OpenQA.Selenium.Firefox.FirefoxDriver();
    context.Add("browser", webDriver);
}

This method introduces a new concept, the context. NBehave consists of three contexts, a Feature context, a Scenario context, and a Step context. A context is essentially a dictionary whose scope is limited to the element it describes (a step context’s scope is a single step).

This provides two advantages.

  1. A place to store your variables.
  2. If something within your context implements the IDisposable interface, it’s automatically disposed of once the end of the context’s scope is reached.

Next, lets add a step for navigation.

[When("I visit $url")]
public void NavigateTo(string url)
{
    FeatureContext context = FeatureContext.Current;
    IWebDriver webDriver = context.Get<IWebDriver>("browser");
    webDriver.Navigate().GoToUrl(url);
}

The first thing we do in this step, is get the driver from the context. Then we can use the .Navigate().GoToUrl() syntax to navigate the browser to the URL described in the url variable. In NBehave ‘$” annotates that the text following the ‘$’ is a variable name.

Next, the SearchFor step.

[When("I search for \"$text\"")]
public void SearchFor(string text)
{
    FeatureContext context = FeatureContext.Current;
    IWebDriver webDriver = context.Get<IWebDriver>("browser");
    IWebElement textBox = webDriver.FindElement(By.Id("gbqfq"));
    textBox.SendKeys("Hello World!" + Keys.Enter);
}

Finally, the validation step.

[Then("the help button exists")]
public void HelpExists()
{
    FeatureContext context = FeatureContext.Current;
    IWebDriver webDriver = context.Get<IWebDriver>("browser");
    webDriver.WhenThisFails(dr =>
    {
        new WebDriverWait(dr, TimeSpan.FromSeconds(5)).Until<bool>(d =>
        {
            try
            {
                IWebElement helpButton = d.FindElement(By.XPath("//span[@id='fsl']/a"));
                Assert.AreEqual("Help", helpButton.Text);
                return true;
            }
            catch
            {
                return false;
            }
        });
    }).TakeScreenShot();
}

Ninja Level

  • Create a method that gets the WebDriver from the context to avoid redundant code.

Up Next

  • We will implement the NBehave test runner and replace the code in the Google class with a .feature file.