Sunday, September 2, 2012

Using the MVP Pattern

I've mentioned in previous posts that I prefer the MVP pattern to MVC or MVVM. In truth, there are advantages and disadvantages in all the MV patterns (here's a quick explanation of the differences.) I prefer the MVP pattern mainly because it is simple to inplement and involves no binding magic. In this article, I'll describe the pattern and provide some guidence for how to use it that I have found helpful.

Description of the MVP Pattern

The Model View Presenter (MVP) pattern is a simple pattern. As you can see from the diagram below, the View and the Presenter reference each other, but only the Presenter references the Model:

The Model is your entity or domain object model. It contains the data and, in the case of a rich domain model, the domain behavior.

The View consists of Properties, mainly. Each control on the view that manipulates or displays data from the model will have one or more properties. These properties are exposed to the Presenter to allow it to interact with the View without knowing how the view is implemented. The View's properties may be base types, like string or decimal, or they may be more complex types, such as an IList. Although the View does not know the model, it may use the model types. It does, however, know the Presenter. This is because the View must call the methods of the Presenter whenever something occurs in the View. For example, the view might call a Load() method to get the Presenter to load data into the View, or Save() when a user clicks a "Save" button (or when some other circumstance dictates saving.)

The Pesenter does all the work. It has a reference to the View so that it can set or get the View's properties and it provides the methods that the View needs as I mentioned above. The Presenter is also responsible for maintaining the model, whether this is maintaining the model's state, interacting with a service or manipulating the model is some way, such as adding or removing objects from collections. The Presenter is similar to a code-behind except that it is abstracted behind an interface, thus allowing it to be easily unit tested.

View and Presenter Interaction

The View and Presenter are interdependant. Since the View depends on the Presenter to perform work on its behalf, it must have a reference to its Presenter. Likewise, the Presenter must have a reference to the View in order to get or set data. Because of this, there is most often a one-to-one relationship between Views and Presenters in an application.

The Presenter should be abstracted from the implementation of the View and visa-versa through interfaces. This is necessary in order to do good unit testing and to allow the Presenter to be used with Views that target different implementations. You might, for example, have the need to implement the same View as both an ASP.NET page and as a Windows Form. Because of the abstracted nature of the Presenter, the same Presenter is used for both, thus minimizing the coding effort.

Because of the possibility that the View may have different implementations, it is important that there be as little logic in the View as possible. Consider the following View definition:
 
public interface UserEditorViewInterface
{
   
UserEditorPresenterInterface Presenter { get; set; }

    string UserId { get; set; }
    string UserName { get; set; }

}
   

And the corresponding Presenter definition:

public interface UserEditorPresenterInterface
{
    UserEditorViewInterface View { get; set; }

    void Load(UserEditorViewInterface view);
    void Save();     
}
When implementing this view in a Windows Form, it might look like this:

public class UserEditor : Form, UserEditorViewInterface
{
    public UserEditorPresenterInterface Presenter { get; set; }

    public UserEditor()
    {
        this.Presenter.Load(this);
        this.saveButton.Click += (a,b) => {this.Presenter.Save();};
    }
    public string UserId
    {
        get { return this.userIdTextBox.Text; }
        set { this.userIdTextBox.Text = value; } 
    }
    public string UserName { get; set; }
    {
        get { return this.userNameTextBox.Text; }
        set { this.userNameTextBox.Text = value; }
    }
}


Notice how I implemented the two properties of the View in the form by simply reading and writing to the controls. I called the Load() method of the Presenter in the constructor (it could just as well be in the OnLoad() method of the form) and I also setup an anonymous delegate to handle clicking the "Save" button. In this way, I have condensed the code quite a bit.

Note: I like binding control events with anonymous delegates like this, but once you get more than a couple statements, it would be more readable to create a concrete delegate and reference a method.

The Presenter is just as straightfoward:

public class UserEditorPresenter : UserEditorPresenterInterface
{
    public UserEditorViewInterface View { get; set; }

   
    public void Load(UserEditorViewInterface view)
    {
       this.View = view;
       // do some load logic
    }
    public void Save()
    {
        UserInfo user = new UserInfo()
        {
            UserId = this.View.UserId,
            UserName = this.View.UserName
        };
        // save the object
    }
}

I left out the details about interacting with a service to retrieve and persist the UserInfo object, but there are a couple noteworth points in this implementation: First, I passed the View through the Load() method. I prefer to do this to be sure that the View is provided, but you could also set it using the property. Second, the domain object, UserInfo in this case, was recreated in the Save() method.

I often reconstruct the domain object in the Presenter simply because I cannot depend on the Presenter being stateful. Of course, if you were limiting it to an implementation where its reference would be maintained you could build it that way; but if it might be used in another implmenation, one that is stateless, then you cannot depend on it maintaining its own state. The simple solution, then, is to reconstruct the object.

An alternative to reconstructing the domain object is to use the View for state. For all practical purposes you are already doing this, but there may be more properties in the object than are exposed in the view. So, the View could hold a reference to the object. You would, however, depend on the View to handle state properly. That's not unreasonable, but it is possibly a point of failure.

Use of Interfaces and Factories in MVP

You may have noticed that both the View and the Presenter have interfaces. In fact, they reference each other through these interfaces. This is important for testing the presenter because it allows a unit test to create a mock-up of the view instead of referencing the actual view, which might not be possible. So, if the View does not reference the Presenter directly, how does it get the Presenter? An interface, after all, cannot be instantiated.

This is where factories come into play. Factories build objects. When implementing MVP, you should create a presenter factory. Such a factory will likely be very simple. Here is an example of a factory and its interface:

public interface PresenterFactoryInterface
{
    
ServiceFactoryInterface ServiceFactory { get; set; }

   UserEditorPresenterInterface CreateUserEditorPresenter(); 
}
public class PresenterFactory : PresenterFactoryInterface
{
   public ServiceFactoryInterface ServiceFactory { get; set; }
  public UserEditorPresenterInterface CreateUserEditorPresenter()
  {
       return new UserEditorPresenter()
      {
         Service = this.ServiceFactory.CreateService()
      };
  }

}
Note that I included the ServiceFactory. The factory for each layer would natually reference the factories for the next layer so that it may set dependency properties in the objects it creates. See my article on Factories for more details on how this works.

So where does the presenter factory get created? That will depend largely on the type of application you have. Here are some possibilities:
  • For Windows applications, the factory could be created when the application starts and stored in a static variable that is accessible throughout the application.
  • If the Application Controller pattern is used, you could create this factory in the controller, or created it on an as needed basis.
  • In Web applications, you might create the factory during the Application.OnStart event and store it in the Application state.
  • Also in web applications the factory might be created in a base class when a form in created (that could be costly, though, as it would occur on each post-back, but it might not be avoidable if the site is truly stateless.)
I find that creating the factory is an implementation detail best determined when designing the application. In all cases, there will be some code in a root level class that insantiates all concrete implementations of each factory. In the following example, each layer of factory is instantiated in the constructor of the consuming layer.

this.PresenterFactory = new PresenterFactory(
    new ServiceFactory(
       new DataAccessFactory()
    )
);

Instantiating the Presenter

Part of determining where to create the presenter factory depends on how you mean to create the presenter itself. That may very well depend on the platform you are using and the mechanism you have in place to instantiate views. For example, a Windows application may have the luxury of an Application Controller that can create a view and inject a presenter (I like that.) On the other hand, in an ASP.NET application, the web host controls creating the view; in that case, while injecting a dependency is possible, it is rather difficult.

Let's consider the first scenario. In the Application Controller pattern, a controller object determines which view to show the user. It is often responsible for creating that view, although that could be delegated. When it is responsbile for creating the view, it will have a reference to the presenter factory, thus allowing it to inject the presenter into the view, like so:

public void ShowUserManager()
{
    UserManagerViewInterface view = new UserManagerForm()
    {
        Presenter = this.PresenterFactory.CreateUserManagerPresenter()
    };
    // other implementation details
    ...
    view.Show();
    return view;
}

Other implementation details might include setting the View property of the presenter or calling a load method on the presenter or even setting MDI properties if that 's how the application works. So, in this case, all the factories would be created either in the Application Controller itself or in a startup method (like, static void Main()) when the Application Controller is created, which would mean the PresetnerFactory is injected into the controller.

This is fine for a Windows or WPF application, but an ASP.NET application cannot call a "Show" method on the view, so the controller cannot be so direct. The simplest approach is to put the presenter factory in an Application state variable and create the presenter in the Page_Load() method:
       
protected void Page_Load(object sender, EventArgs e)

{
     this.Presenter = ((ApplicationController)Application["Controller"])
             .PresenterFactory.CreateUserManagerPresenter()
}


Sunday, July 22, 2012

Practical Thoughts on Using ORMs

Object Relational Mapping (ORM) appears to be a polarizing topic. I recently read a very interesting article (ORM is an Antipattern) that made some very good points against ORMs. Actually, it was interesting for the comments on the article as much as for the content. A more realistic assessment (in my opinion) by Martin Fowler (ORM Hate) points out that ORMs are difficult to use because the problems that they attempt to solve are hard to solve, no matter what the strategy. I take a more practical approach to ORMs. I neither hate ORMs nor think they are the answer to mapping relational data to objects. In this article, I'd like to explore my thoughts and experiences on the subject.

The Benefits or ORMs

Most business data is stored in relational databases. These databases are ubiquitous and well understood in the software development industry. They are great at slicing and dicing data in all the ways we need to analyze the business, which is the rationale for our programs. Yet the two concepts, objects and relational data, don't always match up very well. Our objects don't always map directly to our tables. When we program, we must make the transitions back and forth between our object models and our data models as best we can. ORMs exist to make that easier.

The truth of the matter is that I've always been hesitant about ORMs because it seemed to me that they did the easiest work in my applications. Writing INSERT, UPDATE, DELETE and SELECT statements is easy. But ORMs do attempt to do more than just CRUD work; they manage concurrency and provide lazy loading for example. Yet, they do none of that better than I can do it. The value, then, in an ORM is found in the development time savings they provide, which can be substantial. At least, this is one of their more appealling values on the surface.

Another even more appealing value, in my opinion, is the ability to use LINQ (see Language Integrated Query.) In fact, I would not want to use an ORM if it did not have this magnificent feature. Understandably, some might not like the thought of learning another syntax that is similar, but nearly enough, to T-SQL. I wrestled with the language when I first started and still do. It's worth it, though, for the simple fact that it is refactorable. If I need to change something about a class, my LINQ queries will get flagged by the compiler - or perhaps updated by the refactoring capabilities of my development tool. By contrast, a SQL statement inside a string is only evaluated at runtime.

Another valuable benefit, if the ORM supports it (Entity Framework does), is the capability to build your database during development. This is very valuable when using doing TDD and Outside-In development because you do not have to focus on building the database before you work through your feature set. As you build your object model and run tests, the ORM builds the database. However, you do have to think about your data model to make sure that the ORM builds it correctly and you may have to adjust the data model when it is done, but much of the work will be done as you go through the process, especially if you use a database that allows you to regenerate each time you run your tests, like SQL Compact Edition.

ORM Drawbacks

ORMs are not with out drawbacks - significant drawbacks. Every ORM is different, so I will limit my points to the one with which I am most familiar: Entity Framework.

The most significant issue that I've had is how Entity Framework tracks objects and maintains connections. Because objects in .NET are passed by reference, when you query a database, the ORM holds onto references to the objects it gives you. This is critical for how it functions. You update the objects and then tell the ORM to update the database. That's great. However, this can lead to a number of confusing situations.

The first situation occurs when you treat your objects as if they are not connected. In truth, they are not connected to the database, but they are connected to the ORM. It is quite easy to query objects in such a way that they interfere with the persistence of other objects. For example, if you get your hands on an object and do something to cause it to be invalid, you may get a validation error when you try to save some other object. But the error will appear to be unrelated. To be fair, from the ORM's perspective, you made a mistake; you probably have a design flaw somewhere, but it is sure difficult to troubleshoot. What all this comes down to is that you are not free to manipulate your objects however you want. There can be unintended consequences, thus raising the level of difficulty in working with ORMs.

Another drawback of ORMs, and probably more often cited, is query performances. ORMs don't claim to improve query performance, but they can sometimes make it considerably worse without your realizing it. The particular situation that comes to mind is when the ORM makes multiple queries when you might have only made one. This happens when it must get child objects or collections. Fortunately, some ORMs provide methods to alleviate this problem (the Include() method in Entity Framework, for example.) Yet even these helper methods are limited and you might find an ORM making more queries than you intended, or building a query that is not the way you would have liked it. In the end, you are more suited to optimize reads on writes to the database than any ORM - and that may be critical to your application.


A Practical Approach to Using ORMs

I think in many circumstances the benefits of an ORM out weighs the drawbacks. With the exception of query performance, good design practices can alleviate most drawbacks. But even query performance can be mitigated to a large degree. I have a few recommendations for using an ORM to get the most out of it.

Use Database Views

Displaying data is a fundamental function in any business application. The data you need to display is often a summary or aggregate of the very data that your objects represent. You cannot reasonably display this data with outview objects. View objects, like database views, are not updatable (necessarily) and generally denormalized. If you are going to use such objects, and all you are really doing is pulling the data from the database to show in a report of some sort, you might as well use database views.

With a database view you can optimize the query as much as you like. Mapping qn object to such a view is trivial. The queries are easliy made and you get all the benefits of LINQ - plus for most of us writing the query to create the view is easier than writing the equivalent LINQ query.
The one drawback that I have encountered in using database views occurs when I'm initially developing an application and I'm using the features of the ORM to create the database. The view objects from an ORM's perspective represent tables like any other object. That could be a problem during testing if you don't account for it. The resolution is simply enough: let the view be a table during development and add test data as you need it. Once you are ready to implement the real database, change the generated DDL.

Separate Database and Domain Classes

This is a difficult decision to make. It seems to easy to create classes that define the application domain and the database. That's Code First development and will generally work well enough, most of the time. But it does not take long to get into situations when dealing with the inherent problems of ORMs holding onto references will kill your productivity. So, use Code First techniques to create a data model and build a data access layer, but create a separate set of entities for your domain model.

One of the main reasons I have begun to adopt the approach is because it forces me to organize my project from the beginning in the way that I will inevitably want it once I start having problems with the ORM. It may seem like a lot of extra work, but these classes are fairly simple and, quite frankly, look very similar. However, don't be tempted to use inheritance; your ORM may interpret that in ways you do not intend. Another reason is that I find it often more convenient to denormalize certain properties. Sometimes, it just works better to do that in the application, even though you do not want to do it in the database.

Forget Lazy Loading

Lazy Loading was one of the draws to ORMs. You get an object, but you don't have to pay for loading child collections unless you access them. That's a great concept for a very deep model. I implemented a Lazy Loading collection once for a project that had such a need and it worked great. But, I had a lot of control over how it worked. Not so with ORMs.

I have found that because I have this feature, I will try to use it. The problem is, while the object is connected to the database, accessing a property causes a database call. That's fine once, but it can get out of hand quickly, especially if you try to iterate through the object graph. Once the object is no longer connected to the database, the Lazy Loaded property, which was a proxy class, will throw an exception. I avoid using it for those reasons, so what's the point in even having the feature?

Fortunately, if you follow my advise about separating the domain and database models, you won't be tempted by the feature, since you won't have it. I find it a better design practice to be intentional when I query to get the objects and collections I need. Even if I have to make separate queries for child collections. Also, a little denormalization in the domain layer can go a long way to alleviate the need for additional queries.

Conclusion

Some developers will take a long time to accept ORMs. Perhaps a day will come when a better tool is created. For now, I treat ORMs like any other tool: if it's right for the job, use it. I've only scratched the surface on determining whether an ORM is right for the job, but I hope I've provided some practical ideas that will help make using ORMs less painful. Right SQL in strings is still viable and good unit testing will still catch changes to the data model. So, in the end, even though I'll typically choose Entity Framework in a business application, it's not going to hurt the application if I don't.

Wednesday, July 4, 2012

Factories

The design pattern for a factory includes an interface for the factory with methods that return interfaces. Here is a diagram that includes the implementation of the factory:



FactoryInterface contains methods that return interfaces for, in this case, ServiceAInterface and ServiceBInterface. The implementation, Factory, is responsible for determining which objects that implement those services should be created.

A factory could be created without the use of interfaces, of course, but you would be limited in how you could use that factory in testing. See Developing Software with BDD  for more on how factories can be used in testing.


Factory Trees

In most business application where factories would be used, you are likely to need factories in each layer (see Layered Design Pattern for Enterprise Applications .) For example, you will need a factory for the Presentation, Service and Data Access Layer. In this scenario, the creation of concrete implementations at each layer will depend on creating the dependant layers, visa-vi Dependency Injection. To accomplish this, each factory must have a dependent property for the factory (or factories) beneath it. The following diagram illustrates this:


Here you see that the Factory class has a property for FactoryB.

You will notice that this property is not a part of the interface. It is not necessary because the fact that the factory is even needed may be considered an implementation detail. For example, suppose you have an interface that defines a service. Naturally, that service needs some data access objects, so the service factory will have a data access factory. However, you might also use the same service in a thick click scenario where the service is really a proxy for a web service. In that case, the data access factory is not required. Since it is the factory itself that will know whether another factory is needed, the interface need not define that dependency. Of course, it won't really hurt if you included it anyway; you'd just not set it for the service proxy factory.

Here is what the example factory tree would look like:



Instantiating Factories


Using a Factory in code is quite simple:

ServiceAInterface servicea = factory.CreateServiceA();

The challenge is when and where to instantiate the factory. If the factory is by itself, you might just create it when you need it, like so:

FactoryInterface factory = new Factory();
ServiceAInterface servicea = factory.CreateServiceA();

Of course, the problem with this approach is that you hardly get any value out of abstracting the factory itself. A better approach is to create the factory early, such as when the application starts, and use that object throughout the application. You might do this in the Main() method of a Windows Forms application or in the Application.Start of an ASP.NET application.

Wherever you instantiate the factory, you will specify the concrete factories that are used in the tree. By doing this in one place in your application, you are able to switch out factories at different layers when you need to, as would be the case for unit tests. Here is an example of how you would instantiate the above factory tree:

PresenterFactory presenterfactory = new PresenterFactory(
     new ServiceFactory(
           new DataAccessFactory()
     )
);

This would be typical for an ASP.NET application. The variable presenterfactory would end up in Application state or you might even invoke this at each post back, assuming these constructors did nothing.

If you were instantiating the same PresenterFactory for a thick client that used a web service proxy, it would look a bit different:

PresenterFactory presenterfactory = new PresenterFactory(
      new ServiceProxyFactory()
           );

No need for the DataAccessFactory here. Note two that both the PresenterFactory and any presenters it creates will have no knowledge of the fact that you used a different service factory. This allows the presentation layer to be entirely independent of the service layer.

Using Factories in Unit Tests

Finally, a word about factories in unit tests. Since unit tests are granular, i.e. you test a single method in isolation, the dependencies that factories create would normally already be created by your test. For example, if I'm writing a test for some service method, and using Moq, I might setup my service like so:

Mock<DataAccessInterface> dataAccessMock = new Mock<DataAccessInterface>();
ServiceInterface service = new Service(dataAccessMock.Object);

I did not use a factory like I would normally in the application. That's perfectly fine as long as I test my factory methods somewhere.

On the other hand, I might gain a benefit by letting the factory do its job even in unit tests. In that case, I would use the factory implementation along with the service implementation, but I would provide mocks for other layers. So, my test setup would then look like this:

Mock<DataAccessInterface> dataAccessMock = new Mock<DataAccessInterface>();
Mock<DataAccessFactoryInterface> dataAccessFactoryMock
     = new Mock<DataAccessFactoryInterface>();
dataAccessFactoryMock.Setup(f=>f.CreateDataAccess())
     .Returns(dataAccessMock.Object);

ServiceFactoryInterface serviceFactory
     = new ServiceFactory(dataAccessFactoryMock.Object);

Notice how I setup a mock for the data access factory that will provide the data access mock for the service. This is useful in a base class for service tests (or whatever layer) where you can setup all the data access factory methods you need to create all the service objects. This allows your setup to consist of a single line, assuming that the base test fixture exposes the service factory.

Note: When doing this, you will probably NOT validate the data access factory mock because you will have setup all the methods when only a few are needed in any circumstance. That will not pose a problem for testing since the absence of the data access mocks will cause the test to fail IF you did not properly implement the factory. To do this with Moq, use MockBehavior.Loose in the constructor and don't call VerifyAll().


Monday, July 2, 2012

Layered Design Pattern for Enterprise Applications

In this post, I'll briefly describe how I've been layering applications lately. I wanted to capture the concepts and provide some explanations. I do not pretend to think this is the only way to design applications, but I do think it is a good way. It is also fairly typical.


I'll start with the most basic of designs:


Data Access - this layer in my mind if very granular. The classes involved have the single mission to read and write data to the database. There may be a one to one relationship between data access classes and database tables, but not always. There are some cases when it is prudent to denormalize data.

Service - the service layer maps to application functionality from a logical perspective. Take the functionality of your application and partition it into logical modules and you have a basis for services. Thus, service classes are more coarse than data access classes, but you may have multiple services depending on the complexity of the application.

Presentation - the presentation layer provides the business functionality behind the user interfaces. Here is where the interactions between the user and the services are orchestrated. I typcially will have a one to one relationship between Presenters and UIs (views), although sometimes a UI may have a composition of complex user controls that each have their own presenter.

User Interface - this layer depends on the platform (web, windows, modile, etc.) I want this layer to be as simple as possible and contain no business logic. It can get complicated, though, as more and more advanced controls are introduced.


Depending on how you are implementing an application, you might have additional layers. For instance, in a client server environment, there is the need for a web service and proxy layer:


You also might need a layer to sit on top of the UI and Presenter layers. This layer (e.g. Application Controller) would orchestrate the interaction between the User Interfaces and it would interact with the Presenters (assuming an MVP pattern):



Organizing a Solution

Thus far what I've laid out is straightforward and fairly ubiquitous. Now I want to layout some details about how all this fits together in a .NET Solution.

It is reasonable to think that each layer would have its own assembly, or set of assemblies. While that is perfectly fine, There is a little more complexity than that. For example, each class in each layer will likely have an interface. Although those interfaces can coexist with the classes that implement them, I find it makes more sense to separate them. I do not, however, see the need to have a separate assembly for interfaces at each layer. Instead, I prefer a single assembly to contain interfaces. This assembly then becomes something of a definition for the application.

Some may prefer to break this definition assembly into two separate assemblies, one for data access interfaces and another for the other interfaces. This would be meaningful in a client server application where the client hardly needs any knowledge of the data access layer (in truth, it would not need the service layer, either, but the service interfaces are used to define the web service proxies, so they are needed.)

There are other classes that cannot be relagated to a specific layer that will also need assemblies. Domain objects or data access objects for one, but also any framework classes you may require (your framework, not .NET's.) With these considerations, there are at least 6 assemblies needed:

1. Framework and Domain
2. Definitions
3. Data Access
4. Service
5. Presentation
6. Client

I actually like separating Framework classes and Domain classes. When using Entity Framework, it may also be useful to separate the database context and entity classes (assuming they are different from the domain classes) into their own assembly - but that's a topic for another post.

To put a little more meat on these thoughts, here's how I would organize projects in a solution for application X:

  • X.Framework
  • X.Domain
  • X.Definition
  • X.DataAccess
  • X.Service
  • X.Presentation
  • X.Client (a Web App, perhaps?)
If I were using Entity Framework, it would be:

  • X.Framework
  • X.Domain
  • X.Definition
  • X.EntityFramework
  • X.DataAccess
  • X.Service
  • X.Presentation
  • X.Client 
If I needed a client server application, it would be:

  • X.Framework
  • X.Domain
  • X.Definition
  • X.EntityFramework
  • X.DataAccess
  • X.Service
  • X.WebService
  • X.WebServiceProxy
  • X.Presentation
  • X.Client

Tuesday, June 26, 2012

Developing Software with BDD

In a previous post, Developing Software Outside-In, I described how to use TDD practices to design and develop somewhat on the fly. The same concepts can be extended to Behavior Driven Development (BDD, see Dan North's article Introducing BDD for an introduction.)  BDD is an evolution of TDD in that it brings the same concepts to a higher level in the process. BDD focuses on defining the features of an application in a vocabulary understood by business users and developers. This vocabulary is used to define features and scenarios that are testable. Using these principles, and some nice tooling like SpecFlow, the advantages of TDD can be seen at an even higher level, earlier in the process.

In this article, I'll discuss how principles of BDD can be used in Outside-In development. I'll begin with how features are defined in a Functional Specification and then brought into a tool to use in the development process. I'll discuss how tests can be created based on those same feature definitions and then used as a starting point for building the application.


Functional Specifications


These days, I like to express specifications using the Gherkin syntax, somewhat. This syntax expresses the features of an application with feature and scenario statements. When I put specs together, I'll often include a UI mock up and possibly some definitions along with these scenario statements. Thus, my specs will look something like this:



The Gherkin portion is the "Feature:" and "Scenario:" statements.

Features represent specific functionality in the application. They should generally represent a single unit of work relevant to the user. Features should not be as granualar as "select something from a dropdown list", nor should they be as coarse as "Order Entry Module." They should generally be a distinct task, such as "Create a new Sales Order" or "Add an Item to a Sales Order."

Scenarios represent permutations of a feature based on the state of the system. You would express normal, adnormal and exception circumstances as in any test. When defining scenarios, you should be comprehensive. Features and scenarios are not only the specifications for the system, but also the tests that you will create for the system and even the its documentation. In addition, because features and scenarios are expressed in a way that the user will understand, they can survive the development cycle to be used in the next iteration of the application.

Let's look at a sample feature, then:

In order to ensure that deleting an Account Type does not impact any Customers
As an Account Administrator
I cannot Delete an Account Type if there is it assigned to a Customer
Scenario: Should warn user when there are Customers of the Account
          Type when Deleting the Account Type
Scenario: Should warn user when there is an error when Deleting the Account
          Type
Scenario: Should Delete an Account Type when there are no Customers of
          the Account Type when Deleting the Account Type

This feature simply states a business rule that Account Types cannot be deleted if the account type is associated with any customers. It's simple enough to understand that there are account types for customers, so you can't very well go deleting account types without dealing with those customers. The data integrity rules must not allow a null account type for a customer and we would rather avoid a nasty referential integrity error. So, we have a few basic scenarios that might occur and that we need to test for. Any business user would understand this, although we are not explicit about what "warn user" means. That's an implementation details that does not necessary involve the feature (although, it could if we wanted to specify it.)

Of course, this feature will not exist all by itself. There's not enough here to define an application. We can say, however, that there must be "Account Types" and there must be "Customers" and a "Customer" has a property that references an "Account Type" and that property cannot be null. In the context of the entire application, there will be other features that define forms and other domain objects and how they interact. The feature above merely serves as an example. You would have other features, such as:

Feature: Customers can be Searched by Name
Feature: Customers can be Searched by Number
Feature: Customers can be Created
Feature: Customers can be Deleted
Feature: Customers can be Changed
Feature: Account Types can be Created
Feature: Account Types can be Deleted

These features may elicit the response "duh." Yet, as every developer knows, they still must be built and they must be tested. So, we layout every little feature along with the various scenarios needed to validate that it will work property, according to the needs of the business. Such is a functional specification.


Mock Applications

The normal method of going from functional specs to development is to start laying out the designs needed to implement the specs. I used to use a lot of UML to do this, but I think there is now a better way. If I am building an application with a graphical user interface, my functional specs will have mockups of all the UIs that I am going to build. It does not matter how these mocked UIs are created, all that matters is that I've clearly defined what the user is going to get. So, thinking along the lines of building Outside-In, I can start with the UI, right? Well, I could, but I'd run into a significant testing hurdle.

Automated tests are not easily created for UIs. That's one of the reasons that we have all those "MV" patterns: MVP, MVC, and MVVM. They all attempt to remove the business logic from the UI so that we can have more comprehensive testing (among other things.) I suggest, then, that you start not quite with the UI, but with the View (they all have a view definition, which is an interface.) I like the MVP pattern and will focus on that one in particular, but the principles apply to the others as well.

So, just like I demonstrated in Building Software Outside-In, we can create the interfaces for the view and the presenter just based on the specs. From there we can build the rest of the application using TDD methods. However, BDD features often describe interactions between views, and I do want to test not only the granular interaction (one form should call a method that's supposed to open another form) but also a somewhat more robust interaction (when one form opens another form the application state should change in a certain way.) I want more than what mocking tools provide. I want a mock application.

A "Mock Application" is only a "Mock" in that a person cannot really do anything with it because the UI is not really a UI. It exists for testing purposes only. Consider the intent of the "View" in the MVP (or any other) pattern. By its very nature it is supposed to have no functionality. True, it will have functionality relevant to the platform (managing state in ASP.NET, for example,) but that functionality has nothing to do with our application's true features. If you were to implement the view using a plain old class with nothing but properties, it could still function as the view layer of the application. You would need another application to actually perform actions in the application, and that is precisely what tests are for.

To build the view layer of an application, all you really need to do is implement the view interfaces and any infrastructure required by the platform to allow your views to interact. I like the Application Controller pattern for this, which is described fairly well in The Presenter in MVP Implementations. The Application Controller handles the navigation between forms, which can itself be abstracted behind transition objects so that the mechanics of opening a view (Response.Redirect, for example, or Form.Show) does not have to interfere with the application logic. Using an Application Controller can minimze the platform specific implementation to just the views and a few transition objects. The only other consideration is maintaining which view is active, although in many circumstances that is not even relevant.


The Process of BDD Tests

The Gerkin language specifies a syntax for Scenarios. This syntax is a set of statements that begin with "Given", "When" and "Then". They are directly related to the typical "Setup", "Execute" and "Validate" steps of a test. The logic goes as follows:

Given that the application is in a certain state
When I perform some operation
Then certain things happen

These statements should be expressed in a way that is easily understood by the user base. They may be tedious at times, but they still make sense to everyone. So, from the functional specs, each scenario is further refined to express this level of detail.  Consider one of our sample scenarios:

Scenario: Should warn user when there are Customers of the Account
Type when Deleting the Account Type
I might express the steps in the scenario as follows:

Given that I have opened the Account Type List
And that I have selected an Account Type that is associated with a Customer
When I Delete that Account Type
Then I am warned that the Account Type cannot be Deleted

These are, in fact, the steps I need to take in code in order to test the scenario. If I use a tool like SpecFlow, then I can have individual, re-usable methods generated for each step that I can then implement.


Using BDD Tests and Mock Applications to Design and Build

How BDD works with SpecFlow is beyong the scope of this article. What I really want to discuss is the process of defining these tests, which came from a functional spec, so that you can build an application from the Outside-In. To be sure, in order for this to work, you must have done the following:

1. Decided on the architecture and platform and build a foundation. This is typically the same old stuff you build for every application. It's a starting point and does not express any business functionality.
2. Built a testing infrastructure. This is important only in some circumstances. For example, if I want to use Entity Framework Code First development, it would really be helpful if I put together the the infrastructure to generate my database on the fly to SQL CE or something.
3. Created the functional specs - obstensibly, these would have been reviewed by and approved by the business unit the application is for.

You might flesh out all the scenarios before coding. That can be helpful to clarify how the application works. On the other hand, you might just as likely start with a few scenarios and work from there. That would fit nicely into an Agile process. Either way, once you have the groundwork done, you begin in the same way: Designing the View and Domain Objects.

One of BDD's intentions is to establish a vocabulary that is understood by both developers and the business unit. That's also integral to Domain Driven Development. You might have noticed my odd use of capitalization in the scenario definitions. I did this to highlight potential candidates for domain objects, properties and methods. I do not believe these will always be everything you need, but it will be the most important parts. Once you work through a few scenarios, you'll see what I mean. So, it should be fairly straightforward to define the domain objects, views and presenters you need to implement the scenarios you have defined. To be sure, at this point we are not saying you have completely defined any of this; you've just defined enough to complete a particular scenario of a particular feature - very Agile, I think.

Defining these three parts - the view, present and domain - is all you need to implement the step definitions to complete the BDD feature tests - but, you must implement them. Hmm...in order to implement the presenter, you need to define the service and in order to implement the service you need to defined the data access layer. Well, not exactly. We can simply use a mocking framework for the presenter and be done with it.

Well, I don't actually mean to be done with it because I want these tests to run the entire stack. Yet, I am building from Outside-In and I'd like to validate each layer as I go. I can use unit tests to do that, and I should write unit tests per typical TDD methodology, but those tests are granular. My BDD tests are defined as application level features. I want to include the Application Controller in these tests and let the presenters be properly instantiated. For me, that means created through a factory. It is, in fact, the power of factories that will allow me to run these BDD tests before I even implement my service (or design my data access layer, for that matter.)

Using Factories to Assist in Outside-In Development

Factories are important in a well designed application. Factories allow us to leverage Dependency Injection to accomplish Outside-In development. When we define each layer of our application (presentation, service and data access) we defined factories for those layers. To be sure, these are abstract factories as they are implemented behind an interface. In our mock application, we would implement factories that produce mock-ups of the layers we have not yet created. In this way, the tests can run and validate the work that we have done.

Consider the following model for an abstract factory graph:


The Presenter Factory will require a Service Factory which will require a Data Access Factory. All this is wired together when the application starts. In our mock application, we simply create a mock up of the factory for the layers that we have not yet coded. So, if we are working on the Presenter, which is requried for the BDD tests, we can create mocks for the service layer factory and the service layer. In this way, our mocks will provide the expected results and allow our tests to run.

Once the presentation layer is complete and all tests pass, we move on to the service layer. Again, we would create a mock for the data access layer objects and factory which will allow our service layer to work. Our BDD tests should again pass, once the service layer is complete.

Conclusion

I hope this article will provide sufficient insight to get you started with BDD and Outside-In development. I have really only laid out the pricinciples for doing this. There are a lot of details that need to be fleshed out to get an application to work no matter how you build it. The goal here, however, is to build an application in a manner that most closely reflects the specifications. I believe the BDD plays an important role in that, but not without a reliable way to build applications using it. Hopefuly the approach outlined here will provide such a way.

Wednesday, June 13, 2012

Building Software Outside-In

I've been working with TDD (Test Driven Development) concepts heavily for the last year or so. I have to say, it has radically changed the way I do software development. The basic concept of writing tests before implementing has proven to be a very good way to ensure the development of solid, maintainable code. This is true not only because you end up with a set of comprehensive tests, but because the process of creating those tests forces you to make better design decisions.

Recently, I've begun to think about the overall process of designing and building software using TDD. As I've focused on abstracting each layer for better testing, the benefits of Outside-In development has become more and more apparent. So here I'd like to layout how I've begun to go about the work of designing and building using this approach. I do not pretend that this approach is by any means novel, but I do hope it will be explained well enough here to be of benefit to others.

Starting from the UI (the Outside)

Suppose you have a typical business application that has a user interface (UI) and some backend database. Assuming you have done the work to define what the application is supposed to do, you may very well have a number, if not all, the UIs defined. I typically do this in a functional spec with collaboration with (or at least sign off from) the business unit. Chances are, you will use a fairly familiar architecture and technology stack. I currently use .NET, Entity Framework and the MVP pattern. That coupled with best of practices for software architecture gives us the layers we need. Starting from the UI, they are:
  • UI (or View)
  • Presentation
  • Service
  • Data Access
There is, of course, a domain layer as well, which may traverse some of these layers depending on how it is implemented. For our purposes here, we'll keep it simple and assume all we need are basic CRUD functions, so the domain objects are simply passed between layers.

Most UIs can be broken down into data elements and actions. The MVP pattern separates these nicely between a View and a Presenter. When I look at a mock-up of a UI, I can start to see what those data elements and actions are. Thus, my design work starts by defining the View and Presenter interfaces.

Suppose we have a UI that looks like this:

It's pretty simple. From this you can see the properties that a view would require quite easily. So, I'd define the view like so:

public interface GroceryListViewInterface
{
      List<Grocery> GroceryList {get;set;}
      Grocery SelectedGrocery {get;}
}

I'll also need to start defining what a Grocery is:

public class Grocery
{
     public int? Id {get;set;}
     public string Name {get;set;}
}

The actions that a presenter must perform are fairly evident as well:

public interface GroceryListPresenterInterface
{
     Load();           // I've got to start somewhere
     New();            // that's a toolbar option
     Close();          // another toolbar option
     Edit();           // this is a link in the grid
     Delete();         // again, a link in the grid
          ConfirmDelete();  // see below
}

Designing through Testing

Were I creating UML sequence diagrams in a tool like Visio, I would next consider what I must do in order to perform the functions in the presenter. It's no different here. But in this case, so that I have something useful that validates my code, I will design the service layer by building the unit tests for the presenter that I have just defined. This will require that I create the presenter class and provide default implementations for the methods (they all will throw the NotImplementedException) and I will start an interface for the service.

In the unit test for the presenter class, I will use a mocking framework to define the service. I use Moq, but there are others out there that would work as well. The advantages of a mocking framework is that you do not need to implement the class you are mocking; you merely need the interface. I will create a single test fixture for a class and then stub out the tests that I need. For brevity, I've only included a few in the following example:

[TestClass]
public class GroceryList_TestFixture
{
    private Mock<GroceryListServiceInterface> serviceMock;
    private Mock<GroceryListViewInterface> viewMock;

    #region Load Tests
    [TestMethod]
    public void Load_Should_Load_Existing_Groceries()
    {
          throw new NotImplementedException();
    }
    #endregion
   
    #region New Tests
    [TestMethod]
    public void New_Should_Add_A_Blank_Grocery()
    {
         throw new NotImplementedException();
    }
    #endregion
}
Notice the long names for the tests. That makes them easy to read, especially with an underscore. I also like to express them using an subjunctive voice, i.e. I say "should" instead of "will" or "can". Another thing to note is the use of regions. I only showed a single test for each method here, and that may be all that's required. However, when there are many scenarios, I like them well organized. Finally, since I haven't implemented any of the tests, they all throw an exception. This will ensure that the tests fail if I happen to run them. That frees me to stub out as many tests as I need without worrying that I might forget to implement them.

When defining these tests, I want to consider all the various scenarios that might occur in a method. Lets consider the ConfirmDelete() method. Whenever I delete something, I want to confirm that operation with the user. "Delete" to me should always ask the question of the user, "Do you really want to do this?" Since I cannot assume my presenter can open a modal dialog, I always have a follow up method with a name like "ConfirmDelete" to be used if the user confirms the action.

Thinking about what happens during the ConfirmDelete() method, where the real work to delete the Grocery is done, a couple scenarios occur to me:
  1. When I confirm the deletion of a Grocery, the Grocery should be deleted.
  2. If an error occurs while confirming the deletion of a Grocery, I should see an error message.
More complicated applications would have more scenarios, but these will suffice. I will then stub out tests for each of these scenarios, using a test name that best describes the scenario:

[TestMethod]
public void ConfirmDelete_Should_Delete_A_Grocery_When_No_Errors_Occur()
{
     throw new NotImplementedException();
}

[TestMethod]
public void ConfirmDelete_Should_Show_A_Warning_When_An_Exception_Occurs()
{
     throw new NotImplementedException();
}

The next part of this step is where the real design work is done. Let's take the first test and work through it.

I'll first divide the test method into the three typical parts of a test. Different people have different names for these parts, but they are basically the same and fairly self explanatory:

[TestMethod]
public void ConfirmDelete_Should_Delete_A_Grocery_When_No_Errors_Occur()
{
    // setup
    // execute
    // validate
}

I setup the test by initializing the object I am going to test and by setting up the mocks. I know I'll need to instantiate the mocks and the presenter object for each test, and that effort is the same, so I'll go ahead and create a private method to do that. I like to return the interface instead of the actual object.

private GroceryListPresenterInterface SetupPresenter()
{
     serviceMock = new Mock<GroceryListServiceInterface>(MockBehavior.Strict)
            { CallBase = true };
     viewMock = new Mock<GroceryListViewInterface>(MockBehavior.Struct)
            { CallBase = true };
     return new GroceryListPresenter(
            serviceMock.Object, viewMock.Object);
}

You can see that the implementation of the presenter takes the service and the view as parameters on the constructor. I could have done this differently, but this is a simple approach that guarantees that the presenter will get its dependencies.

Now that the presenter object is ready and the mocks have been instantiated, I can proceed with the setup portion of the test, which is ultimately the design of the method.

This method does not require much. All I need to do is get the selected Grocery and then pass it to the service through a delete method. So, I'll setup the mock objects to express this:

...
// setup
IGroceryListPresenterInterface presenter = SetupPresenter();
viewMock.SetupGet( v => v.SelectedGrocery).Returns(new Grocery());
serviceMock.Setup( s => s.Delete(It.IsAny<Grocery>());
...

See the Moq documentation for details on how the mock objects work.

That's about it. All I need to do now is execute the presenter method and validate the mocks:

// execute
presenter.ConfirmDelete();

// validate
viewMock.VerifyAll();
serviceMock.VerifyAll();

The next step will be to implement the test for the exception scenario. In that case, I would have the serviceMock throw an exception and express how the it would be shown in the view, through some other method or property on the view. That means, of course, I will amend my view interface. That's the very kind of detail I want to discuss in writing these tests!

Once the tests for all the scenarios have been implemented - and, of course, they all fail - I will have accomplished an Outside-In approach to designing and developing using TDD principals. The service layer is now succinctly defined for this particular feature. I have the methods I need and no more. Even the domain objects have been designed without anything extra.


Building from the Outside-In

The next step is to implement the presenter itself. I can do this before I even consider the design of the service methods because of the tests I have written. I expect them all to eventually pass before I proceed with the service design. I might even move on to other UIs before designing the service. This is a significant part of the Outside-In approach. By doing this, I can discover flaws in my designs before I've gone very far at all. Since the service layer is just an interface, it is really quite easy to adjust. That holds true for the domain objects, as well.

After the presenter has been implemented and all tests have passed, I can proceed in the same manner with designing and building the service layer. After that, the data access layer is next. At that point I'll need to deal with how the domain objects will be persisted and queried. It is noteworthy that until this point, I will not have built a database at all. In fact, using the Code First features of Entity Framework (see Code First Development with Entity Frameworkf 4) I am able to complete the data access layer without really building the database (you can have it automatically generated into SQLCE or MSDE, but that's a topic for another post.)

There are many advantages to this approach for designing and building software:
  1. By starting with the requirements (the UI in this case) I start with the customer's perspective and build only what is necessary to fulfill those requirements.
  2. Missing functionality becomes more evident early in the process. In fact, functionality in the back-end is far less likely to be missed because I will not have defined what the backend looks like until I've completed the tests for the front end.
  3. Using unit tests as a low-level design tool provides me with a complete set of unit tests throughout the development process. Continuous refactoring is part of the TDD process; the unit tests make sure my code continues to work as I refactor.