Quickduck

The following article tries to outline an approach for unit/integration testing using the Visual Studio Team System testing framework.

The article is long, but hopefully worth the read.

Defining terminology

Unit Testing vs. Integration Testing vs. System Testing

The following is taken from our good friends at Wikipedia.

Unit testing is a procedure used to validate that individual units of source code are working properly. A unit is the smallest testable part of an application, in OO the smallest unit is a method; which may belong to a base/super class, abstract class or derived/child class.

Ideally, each test case is independent from the others; mock objects and test harnesses can be used to assist testing a module in isolation. Unit testing is typically done by developers and not by end-users.

Integration testing is the phase of software testing in which individual software modules are combined and tested as a group. It follows unit testing and precedes system testing.

Integration testing takes as its input modules that have been unit tested, groups them in larger aggregates, applies tests defined in an integration test plan to those aggregates, and delivers as its output the integrated system ready for system testing.

System testing of software is testing conducted on a complete, integrated system to evaluate the system’s compliance with its specified requirements. System testing falls within the scope of black box testing, and as such, should require no knowledge of the inner design of the code or logic. [1]

As a rule, system testing takes, as its input, all of the “integrated” software components that have successfully passed integration testing and also the software system itself integrated with any applicable hardware system(s). The purpose of integration testing is to detect any inconsistencies between the software units that are integrated together (called assemblages) or between any of the assemblages and the hardware. System testing is a more limiting type of testing; it seeks to detect defects both within the “inter-assemblages” and also within the system as a whole.

Whitebox Testing (goes hand in hand with Unit Testing)

Unit Tests typically are usually written with intimate knowledge of the unit being tested – which allows all paths within the unit to be covered by the tests ensuring complete code coverage.

Mocking frameworks such as Rhino provided utilities to help assert code paths are tested.

For more information, see http://en.wikipedia.org/wiki/Whitebox_testing

Dependency Injection

Dependency injection aims to solve a particular problem in designing and constructing data structures dependent on other pieces of code, in a way that minimizes the coupling between them.

Dependency injection aids in helping to test particular units of code, substituting in your own versions of the dependency, removing the need to test the dependent code.

For more information, see
Dependency Injection & Testable Objects
Inversion of Control Containers and the Dependency Injection pattern

Mocking

Using the Dependency Injection pattern doesn’t require the use of a mocking framework to by-pass dependencies. It’s entirely plausible to create your own implementations and pass through. This however, will require a lot of time writing the code and cause your projects to bloat. A mocking framework should provide the facility to create dependency implementations (mocks) dynamically, taking the hard work away from the unit tester. The framework also, should provide functionality to help you assert that the mocked objects are used the way you expect them to behave.

There are various mocking frameworks out in the wild, I personally have used NMock and Rhino Mock. My experience with both favours Rhino Mock for flexibility, ease of use and performance. Type.Mock seems to get a lot of high praise too.

For more information, see:
Rhino Mocks – Introduction
Rhino Mocks – Documentation

Time for a Code Walkthrough

We have a CustomerServices class that provides business functions for a Customer entity. The CustomerServices class uses a CustomerDataAccess object to persist/read information from a database. Therefore, the CustomerServices class has a dependency on the CustomerDataAcess class.

A first cut of the two classes might look like this:

public class CustomerDataAccess : ICustomerDataAccess {

        public void InsertCustomer(string name)
        {
            // long running dependency
            Thread.Sleep(10000); // 10 seconds.
        }
}

public class CustomerServices {

  public void AddCustomer(Customer customer){
            new CustomerDataAccess().InsertCustomer(customer.Name);
  }

}

As you can see when we write a unit test for the CustomerServices.AddCustomer method it will make a call to the CustomerDataAccess class and take 100 seconds to complete.

[TestMethod]
public void TestAddCustomer () {

  Customer customer = new Customer();
  customer.Name = 'Ben';

  new CustomerServices().AddCustomer(customer);

 // To verify the test worked, a database lookup would be required.

}

Now we should already have appropriate unit tests for the CustomerDataAccess.InsertCustomer() method, so in our CustomerServices tests we can remove this dependency as it has already been tested.

Let’s re-write the CustomerServices class to utilize the dependency injection pattern:

public class CustomerServices {
        private readonly ICustomerDataAccess _customerDataAccess;
        public CustomerServices() : this (new CustomerDataAccess()) {}

        // using internal prevents the constructor from being publicly
        // available to other assemblies.
        internal CustomerServices(ICustomerDataAccess customerDataAccess)
        {
            _customerDataAccess = customerDataAccess;
        }

        public void AddCustomer(Customer customer) {
             _customerDataAccess.InsertCustomer(customer.Name);
        }
}

//NOTE:  To allow our unit testing project to access the internal members of the project we’re testing, an attribute is added to the AssemblyInfo.cs file.

#if DEBUG
  [assembly: InternalsVisibleTo("CustomerServiceTests ")]
#endif

Now we have the ability to insert our own CustomerDataAccess object, we’ll create our own mock object and use it to remove the dependency – this is for illustration purposes only.

public class MockCustomerDataAccess : ICustomerDataAccess {

      private int _insertCustomerCount = 0;

      public void InsertCustomer(string name) {
            _insertCustomerCount++;
      }

      public int InsertCustomerCount {
            get { return _insertCustomerCount; }
      }
}

[TestMethod]
public void TestAddCustomerWithStaticMock() {
      Customer customer = new Customer();
      customer.Name = 'Ben';

      ICustomerDataAccess mock = new MockCustomerDataAccess();

      CustomerServices _customerServices = new CustomerServices(mock);
      _customerServices.AddCustomer(customer);

      Assert.AreEqual(1, mock.InsertCustomerCount);
}

As you can see here, creating our own mocks and keeping metrics will become tiresome and mean code bloat – especially as you might need many instances of the same interface for differing code paths etc.

Don’t worry though there is a way forward – using a mocking framework like Rhino Mocks.

[TestMethod]
public void TestAddCustomerWithDynamicMock() {
// create the Rhino mock repository
_mockRepository = new MockRepository();

// get a dynamic instance of the ICustomerDataAccess interface
_dynamicMock = (ICustomerDataAccess) _mockRepository.DynamicMock(typeof(ICustomerDataAccess));
_customerServices = new CustomerServices(_dynamicMock);

// Expect the CustomerDataAccess.InsertCustomer method to be called exactly once.
_dynamicMock.InsertCustomer(null);

LastCall.IgnoreArguments().Repeat.Once();
_mockRepository.ReplayAll();

Customer customer = new Customer();
customer.Name = ‘Ben’;
_customerServices.AddCustomer(customer);

// ensure the asserts are correct
_mockRepository.VerifyAll();
}

Using Rhino Mocks we don’t need to create hard-coded mock objects for our tests and we get built in metric checking and a whole lot more – all for free.

What to Test?

1. Any non-trivial piece of code.
2. All code is non-trivial.
3. Then again, something’s just aren’t worth your time.
4. Confused?

For every method that has some meat on it, there should be [TestMethod]s that test all execution paths.
Public properties that are just get/set would be a situation of overkill.

The VSTS code coverage tool is excellent for seeing what you’ve tested and what’s been missed.

To use the VSTS code coverage tool, open up VS 2005 and edit the selected test configuration run:

Click for Full Size Image

Then select the assemblies you wish to monitor:

Then all you have to do is run your tests (but not in debug mode) for it to calculate the metrics. Once finished open up the Code Coverage Window and inspect the results.

Click for Full Size Image

What about Integration Testing?

We can still use the VSTS unit testing framework to write integration tests. The difference though, comes from the point of view that the test methods will not substitute in mocks for the dependencies, allowing the tests to run end-to-end testing a module/function of code right through.