Saturday, May 24, 2008

ASP.Net Unit Testing

Some tell there are many solid solutions by HP (Mercury) or Microsoft's TeamSystem, some claim that all those are not Unit, but Integrative tests. And there are many that mostly perform manual testing... Who is right and what is the best approach?

As usual, let's start from the requirements from the testing framework.

  1. Granularity. We are talking about unit testing, the name suggests that we want to test application units, separately.
  2. Ability to automate. Definitely we want an ability to run the tests automatically, for example after a build. This requirement also yields a requirement for results stability through the application life cycle.
  3. Coverage. Every part that is not covered by a unit test joins the risk group of deferred bugs. Even if there is integrative testing, it probably does not run frequently and therefore our control over correct system status reduces when the un-unit-tested group grows.

Let's partition our application and see how we can test every part.

  1. Libraries, that do not require any "web context". Here we can simply write plain unit tests. No problems.
  2. Custom and User controls. In fact, those are also libraries, providing a well defined API, that can be systematically tested... But how to do it if they require "web context"?
  3. Pages, pages, pages, which actually produce our application on the server side and...
  4. Client side logic - let's not forget all the Java Script we run on the client browsers.

Unfortunately most of the "well known" solutions, including those mentioned at the beginning address mainly point 3 above. The common pattern is to record the requests/responses sent/returned by the server; later sent those requests back to the servers, receive the response and validate it again previously recorded one.

This introduces several problems:

  1. Granularity (since 'Custom and User controls' part is handled on the 'Page' level). When the test fails, we know that the Page failed, not some specific control and therefore had to invest more time in regression diagnostics.
  2. Ability to automate. Since the page evolves over time, our old recording may become invalid, even if everything is correct. This produces many false alarms, preventing automation.
  3. Coverage. Client side logic is left untested.

In other words we have a problem in every aspect of our requirements, what can we do?

  1. Re-introducing test-driven development. Since not every code can be easily tested, we must include its testability into the very beginning of its design phase. The rule of dumb says that testability reduces when we move forward over the following path: 'Simple Library' -> 'Custom/User control' -> 'Page' -> 'Client Side logic'. This means that we must strive to move our application code as 'left' as possible.
  2. Create unit tests for our Custom/User controls. We use to create a separate test for every class in 'Simple Library', the same we can do with controls. Once we have a dedicated test page per control, it does not change when the application changes, improving our 'Ability to Automate' over time.
  3. Client Side logic testing. Since client logic runs in the browser, in order to test it, it must run in the browser, period. Any other option simply does not do the job. One of the best tools I know to run tests on the browser is Selenium - it will be covered in the next article.

Summary

  1. Partition your application as described above.
  2. Try to move as much code as possible 'up' within the partition.
  3. For each User/Custom controls write a dedicated test page to test its functionality.
  4. Use some tool for testing client side logic, my recommendation is Selenium.
  5. If you go with Selenium for client side logic, for consistency use it also for Pages testing.

No comments: