Expression Builder

A layer that provides a fluent interface over a regular API

APIs are usually designed to provide a set of self standing methods on objects, ideally these methods can be understood individually. This is in conflict to a fluent interface which is designed around the readibility of a whole expression, which leads to methods that make little sense individually.

An Expression Builder provides a fluent interface as a separate layer on top of the regular API. This way you have both styles of interface and the fluent interface is clearly isolated, making it easier to follow.

How it Works

A Expression Builder is an object, or group of objects that provides a fluent interface which it then translates into calls on an underlying API. You can think of it as a translation layer that translates the fluent interface into the underlying API.

There are two styles of organizing Expression Builder: as a single builder object or as a builder model structure.

The single builder object approach uses one object to implement all of the fluent calls. This object also keeps track of the state of the fluent expression, so it knows the context for each call.

A builder model typically creates an new object for each term in the expression. Each of these term objects has methods the remaining terms in the expression. The overall builder model is rather like a parse tree.

Builder objects are often easier to put together, their main disadvantage is that they can get quite complex as the fluent interface gets more complex. A builder model allows you to have a cluster of smaller objects, but it can be harder to follow how these objects interact.

A hybrid between these is to use a single builder object but to have it implement multiple interfaces where the interfaces map to term objects in a builder model. This way you get the advantages of the term interfaces showing which expressions come next, particularly useful with modern IDEs, while still keeping an easy-to-follow builder object to implement the expression.

When building an expression, there is a fair amount of error handling involved. One issue is where to place this error handling: in the Expression Builder or in the abstract representation. Placing it in the abstract representation often makes the checks easier to write, it also ensures validation can be done if the Expression Builder isn't used. The difficulty with this, however, is that the resulting error messages can be quite cryptic, since the user is working with the syntax of the builder but error messages are couched in terms of the abstract representation. Placing the validation checks inside the Expression Builder makes it easier to provide more understandable diagnostics but can lead to repetition in the error checks.

As useful halfway house is to call validation checks in the expression buidler but implement the checks in the abstract representation. The builder is then responsible for receiving the any errors from the abstract representation and using the builder's knowledge of the context to provide better error diagnostics.

When to use it

Expression Builder is a good choice whenever there is a mismatch between a sensible interface for the objects in the abstract representation and the interface you need for your DSL. If you find yourself putting an awkward interface on objects to provide the fluency needed for a DSL you should consider using an Expression Builder.

Some DSL contstruction techniques particularly need DSLs and others need it less. You don't need an Expression Builder for static factories, but they are usually very useful for method chaining.

Example: Lair Builder Object (C#)

To illustrate how Expression Builder works we'll explore a couple of ways of handling a small DSL for LairCo. In this example we'll use a builder object and the following one will use a builder model. In both cases the DSL expression we'll use for our example looks the same.

class RulesBuilder : Builder...
    public void build()
    {

      Item("secure air vent");
      
      Item("acid bath")
        .Uses.Acid
          .Grade(5)
          .Type.HCL
        .Uses.Electricity(12);
      
      Item("small power plant")
       .Provides.Electricity(11);
    }

In both cases I'm embedding the dsl code inside a method defined on a subclass of the appropriate builder. This allows use to use the Item method on the builder.

For this part of the example I'm using a single builder object to do the translation. The builder holds onto the configuration in the abstract representation.

class Builder...
    private Configuration subject = new Configuration();
    public Configuration Subject {
      get { return subject; }
    }

    internal Builder Item(String id) {
      currentItem = new Item(id);
      subject.addItem(currentItem);
      return this;
    }
    private Item currentItem;

The Item method creates a new item in the abstract representation and adds it to the configuration. It also keeps hold of the item in a variable for later reference. You'll see this pattern recur constantly. At each call the builder is responsible for updating the abstract representation and updating its own data that captures the state of the parsing.

In this DSL, resources can play two roles: either to be provided or to be used. When we use method chaining for the resource, we need to know which context the resource being used so we can do the right thing during the resource method call. The main task of the Uses method is to capture that context. The context comes in two parts. Firstly we have an enum to indicate whether we are using or providing the upcoming resource. The second bit of state is the resource itself, which we set to null here as it's an error to use it until it's set by a later call.

class Builder...
    internal Builder Uses{ get {
      ResourceResponse = ResourceResponseEnum.USING;
      return this;
    }}
    
    private enum ResourceResponseEnum {NONE, USING, PROVIDING}
    private ResourceResponseEnum resourceResponse = ResourceResponseEnum.NONE;
    private ResourceResponseEnum ResourceResponse {
      get { return resourceResponse; } 
      set {
        currentResource = null;
        resourceResponse = value;
      }
    }

By taking advantage of C#'s properties, I can avoid the empty parentheses that plague these kinds of constructs in many languages. The notion of a getting property that changes state is usually something that would trigger pointed questions in any code review I'd be involved in. This is another case where our desire for a convenient DSL trumps our usual coding rules.

The simplest resource call is that for electricity. I respond by creating the resource and using the state captured in the uses call to decide where to place it.

class Builder...
    internal Builder Electricity(int power) {
      currentResource = new Electricity(power);
      captureResource();
      return this;
    }
    private void captureResource() {
      switch (ResourceResponse) {
        case ResourceResponseEnum.USING: 
          currentItem.addUsage(currentResource);
          break;
        case ResourceResponseEnum.PROVIDING:
          currentItem.addProvision(currentResource);
          break;
        default:
          throw new Exception("No resource to process");
      }
    }

When handling acid, thing are more complex because I need to handle arbitrary further calls to set up the attributes of the acid. This is where the field that hold the current resource becomes useful.

class Builder...
    internal Builder Acid {get {
      currentResource = new Acid();
      captureResource();
      return this;
    }}

Furher calls based on the acid set these attributes on the acid.

class Builder...
    internal Builder Type{ get {
      return this;
    }}
    internal Builder Grade(int arg) {
      ((Acid) currentResource).Grade = arg;
      return this;
    }
    internal  Builder HCL {get {
      ((Acid) currentResource).Type = AcidType.HCL;
      return this;
    }}

The Type method is actually fairly useless here and I'd be inclined to remove it from the DSL. I've left it in only because I want to use the same DSL for both builder implementations, and as we'll see it can be handy when using a builder model.

Although this is a small example it does give a good example of the complexity of state handling that a single builder object can lead to. Growing complexity also leads to a larger object than we would be comfortable with. Despite these issues this approach can work nicely with simple DSLs, and of course the best DSLs are simple.

Example: Lair Builder Model (C#)

I'll now take the identical DSL and implement an Expression Builder using by building up a parse tree of simple builder objects. I am using the same DSL code, so here it is again.

class RulesBuilder : ConfigurationBuilder...
    public void build()
    {

      Item("secure air vent");
      
      Item("acid bath")
        .Uses.Acid
          .Grade(5)
          .Type.HCL
        .Uses.Electricity(12);
      
      Item("small power plant")
       .Provides.Electricity(11);
    }

Again this code is in a method of a subclass of our builder, in this case the root of our builder tree - a configuration builder.

  class ConfigurationBuilder {
    private Configuration subject = new Configuration();
    public Configuration Subject { get { return subject; } }

    public ItemBuilder Item(String id) {
      ItemBuilder item = new ItemBuilder(this, id);
      subject.addItem(item.Subject);
      return item;
    }
  }

class ItemBuilder...
    private Item subject;
    public Item Subject { get { return subject; } }
    private ConfigurationBuilder parent;
    public ItemBuilder(ConfigurationBuilder parent, String id) {
      subject = new Item(id);
      this.parent = parent;
    }

One immediate difference between a builder object and a builder model is that at each point where we introduce a new element we create a new builder object and return it instead of returning this. It's useful for child builders to know their parent in order to be forward calls, as we shall see.

Even though we are using a separate builder for the items, we still have the problem of capturing context so that when the resource calls come we know whether they are usage or provision. I could use the same switch statement that I did in the previous example, but frankly switch statements always give me a queasy feeling in the stomach - so here is another way.

class ItemBuilder...
    public ItemBuilder Uses {get {
      resourceResponder = delegate(Resource r) {
                            subject.addUsage(r);
                          };
      return this;
    }}
    delegate void ResourceAction(Resource r);
    ResourceAction resourceResponder;

In this case the uses method is storing some code that I will execute during the appropriate resource call. Storing a closure in a field isn't possible with all languages but you can often fake it with a little strategy object. In C#, however, the delegate is much more convenient.

When we get the resource call, here the simple one for electricity, we invoke the code we stored earlier.

class ItemBuilder...
    public ItemBuilder Electricity(int power) {
      resourceResponder.Invoke(new Electricity(power));
      return this;
    }

As usual, acid is the more complicated case, and for this I need another builder.

class ItemBuilder...
    public AcidBuilder Acid { get {
      AcidBuilder acid = new AcidBuilder(this);
      resourceResponder.Invoke(acid.Subject);
      return acid;      
    }}

class AcidBuilder...
    private Acid subject = new Acid();
    private ItemBuilder parent;
    public Acid Subject { get { return subject; } }
    public AcidBuilder(ItemBuilder parent) {
      this.parent = parent;
    }

The acid builder can handle the call to set the grade.

class AcidBuilder...
    public AcidBuilder Grade (int arg) {
      subject.Grade = arg;
      return this;
    }

Here we begin to see the advantage of using a builder model, each builder has less methods on it keeping each one simpler. I can take this even further by having a builder for the acid type.

class AcidBuilder...
    public AcidTypeBuilder Type {get {
      AcidTypeBuilder result = new AcidTypeBuilder(this);
      return result;
    }}

class AcidTypeBuilder...
    private AcidBuilder parent;
    public AcidTypeBuilder(AcidBuilder parent) {
      this.parent = parent;
    }
    public AcidBuilder HCL {
      get {
        parent.Subject.Type = AcidType.HCL;
        return parent;
      }
    }

Since I know the acid type builder will only be relevent for one call, I can safely return its parent.

At this point I'll uncover the messy secret of using the builder model - children need to handle ancestor calls. In this case the acid builder needs to handle any call that might appear on an item, such as Uses.

class AcidBuilder...
    public ItemBuilder Uses { get {
      return parent.Uses;
    }}

It's a simple delegation, but it still has to be done. With many levels in the tree it can get messy as you have to ensure all descendents can handle the ancestor's call.

If you have Dynamic Reception, you can override the unknown method handler to forward to the parent, which can greatly simplify this code.

Example: JMock - Builder Object with Multiple Types (Java)

One the most interesting evolutions in the Java world has been that of Mock Object libraries. The original ideas behind mock object testing developed in the fading years of the last century and have been steadily developing in since with new versions of libraries to support this style of testing.

An essential part of approach to testing is to define Mock Objects which can look like real objects to collaborators but in fact are Test Doubles. When you create a mock, you prepare it with expectations - indications of what messages the mock should receive in a particular test scenario. You can then run the test and check that the mock received calls that matched its expectations.

This isn't the place to describe the mock object style of testing in full, or its implmentations. However defining expectations for mock objects is something well suited to an internal DSL. The early mock object libraries used APIs, but later libraries have layered an Expression Builder on top of these APIs. This Expression Builder is a good example of this approach in the context of a curly-brace language.

(I shameliess stole this example from Freeman and Pryce. I used the code from the JMock CVS repository tagged NV1_1_0.)

We'll walk through some of JMock's Expression Builder by looking at how it handles a simple expectation. In this case imagine a system that automates trading. This agent needs to buy some stock when a price reaches a certain threshold. As part of this trade the agent needs to call a mainframe interface's buy method with a set quantity and call a audting system giving it a ticket which it got from the return value of the mainframe interface. The expectation can be encoded like this:

Mock mainframe = mock(Mainframe.class);
Mock auditing = mock(Auditing.class);
Agent agent =
  new Agent( QUANTITY,
             (Mainframe)mainframe.proxy(),
             (Auditing)auditing.proxy() );


public void testBuysWhenPriceEqualsThreshold() {
  mainframe.expects(once())
    .method("buy").with(eq(QUANTITY))
    .will(returnValue(TICKET));
  auditing.expects(once())
    .method("bought").with(same(TICKET));
  agent.onPriceChange(THRESHOLD);
}

The first three statements here show some variables that need to set up: mock objects for the mainframe and auditing objects and an instance of the agent which refers to these mocks.

The method testBuysWhenPriceEqualsThreshold is the test case method. Like most mock-style testcases it comes in two parts. The first two statements set expectations on the mainframe and auditing mocks. The final statement tells the agent to act - machinery within the testing framework will verify that the mocks received the right messages. We'll ignore that machinery here and just concentrate on those two expectations, which are defined using the JMock expectation DSL.

In this discussion I'll only talk about how the Expression Builder works to build up the abstract representation for JMock's expectations. I won't talk about how JMock uses that abstract representation to do its mock object behavior as that topic, although interesting, is really outside the scope of what this book is about. I'll only look at how the Expression Builder takes the DSL expressions and uses them to populate an abstract representation - once the abstract representation is there our explanation will stop and I'll refer you writings on JMock if you want to go further.

The expectation is triggerred by the call to expects.

  
  mainframe.expects(once())
    .method("buy").with(eq(QUANTITY))
    .will(returnValue(TICKET));

The expects method is defined on the mock object

class Mock...
    public NameMatchBuilder expects( InvocationMatcher expectation ) {
        NameMatchBuilder builder = addNewInvocationMocker();
        builder.match(expectation);
        return builder;
    }
    private NameMatchBuilder addNewInvocationMocker() {
        InvocationMocker mocker = new InvocationMocker(new InvocationMockerDescriber());
        addInvokable(mocker);
        return new InvocationMockerBuilder( mocker, this, getMockedType() );
    }

One of the dangers of a real-world example like this is that we don't want to wander off into how the JMock library works, we're only interested in it's use of Expression Builder. In this context the crucial information from this code fragment is that the expects method returns an instance of InvocationMockerBuilder, which is the builder object for the expectation DSL. As is usual for build objects, they contain a reference to the abstract representation - in this case core mock object framework objects that are passed into the constructor.

One of the interesting features of JMock's exptectation DSL is that although it uses a single builder object to handle building its expressions, it uses multiple interfaces on that builder to help guide programmers in building expressions. So although expects returns an instance of InvocationMockerBuilder the return type is the NameMatchBuilder interface. This will limit what methods can be called next on the builder.

But before we move onto these further chained methods, we need to look at the next term in the DSL, which is the top level method call once.

  
  mainframe.expects(once())
    .method("buy").with(eq(QUANTITY))
    .will(returnValue(TICKET));

Since the expectation DSL is always used in the context of defining tests, JMock's designers are able to use a limited evaluation context, in this case set by MockObjectTestCase which is a superclass of all the specific test case classes where you can use the DSL. As with most xunit test case superclasses, MockObjectTestCase provides all the various assertion methods that you might need, together with machinery for running the tests. For our exploration, however, we are only interested in the evaluation context it provides for the DSL.

The once method is one of these evaluation context methods. There are a variety of these methods, all of which set up constraints about how often, or when, the expected method should be called. Their names make it pretty clear what they mean - names such as atLeastOnce, exactly(int expectedCount), and never. Each of these calls returns a different class that satisfies a common interface of InvocationMatcher.

The top level method just returns the particular invocation matcher, the expects method (above) passes it on to the builder via the match method:

class InvocationMockerBuilder...
    public MatchBuilder match( InvocationMatcher customMatcher ) {
        return addMatcher(customMatcher);
    }
    private InvocationMockerBuilder addMatcher( InvocationMatcher matcher ) {
        mocker.addMatcher(matcher);
        return this;
    }

So we see that the builder just adds the returned invocation matcher to the abstract representation - so we don't need to bother with it any further in this discussion.

We see here a common situation where the DSL uses a combination of method chaining on the builder with top-level procedure calls.

Now we move onto the next term 'method'. This term is handled using chaining with the InvocationMockerBuilder.

  
  mainframe.expects(once())
    .method("buy").with(eq(QUANTITY))
    .will(returnValue(TICKET));

Here we see the use of interfaces. The earlier term expects was a method that declared a return type of NameMatchBuilder.

public interface NameMatchBuilder extends ArgumentsMatchBuilder {
    ArgumentsMatchBuilder method( String name );
    ArgumentsMatchBuilder method( Constraint nameConstraint );
}

As we have this interface only method is allowed as the next term. The expects still returned the builder object to hanlde this but the interface limits the set of legal methods in the chain, so although only one builder object does the work, the correct sequence can be enforced by the interfaces.

I said 'enforced' above, but it's not just about enforcment (indeed knowing the JMock authors as I do, it isn't even primarily about enforcement). Most Java developers work with modern IDEs that use code completion to narrow your list of methods in a particular context. So the IDE can use the NameMatchBuilder interface to tell you what legal methods are next. This makes it easier to write expressions in the DSL as the code completion can guide you to use only legal expressions in the right places.

Using interfaces to constrain and suggest is also something that works with top-level methods too. The argument to expects is of type InvocationMatcher. Modern IDEs can limit their code completion suggestions to only those that are type-safe for that position, so in this case the IDE will only show methods that return an InvocationMatcher.

The method method is implemented in InvocationMockerBuilder.

class InvocationMockerBuilder...
    public ArgumentsMatchBuilder method( String name ) {
        checkLegalMethodName(name);
        checkExistingMethodName(name);
        addMatcher(new MethodNameMatcher(name));
        builderNamespace.registerMethodName(name, this);
        return this;
    }

Notice the validity checking being done in the Expression Builder; done here to support better diagnostics, even though it does complicate the builder.

As is typical with builder object implementations it finishes by returning itself, and as is common with JMock it declares an interface return type to guide the next term.

  
  mainframe.expects(once())
    .method("buy").with(eq(QUANTITY))
    .will(returnValue(TICKET));

In this case the next interface is ArgumentsMatchBuilder

public interface ArgumentsMatchBuilder extends MatchBuilder {
    MatchBuilder with( Constraint[] argumentConstraints );
    MatchBuilder with( Constraint arg1 );
    MatchBuilder with( Constraint arg1, Constraint arg2 );
    MatchBuilder with( Constraint arg1, Constraint arg2, Constraint arg3 );
    MatchBuilder with( Constraint arg1, Constraint arg2, Constraint arg3, Constraint arg4 );
    MatchBuilder withNoArguments();
    MatchBuilder withAnyArguments();
}

class InvocationMockerBuilder...
    public MatchBuilder with( Constraint arg1 ) {
        return with(new Constraint[]{arg1});
    }
    public MatchBuilder with( Constraint[] constraints ) {
        return addMatcher(new ArgumentsMatcher(constraints));
    }

This with term allows you to indicate what arguments should be passed to the expected method. One of JMock's features is that you have a great deal of flexibility in describing constraints on these arguments. As a result there are quite a lot of implementations of the Constraint interface that's declared in with's parameter list (25 files in the relevant package). Looking at the name of the classes gives you a good sense of what's there: IsCollectionContaining, IsNothing, IsGreaterThan; even Or and And. People who have run into it will recognise the specification pattern in play, with its characteristic use of composite logic terms and leaf predicates. Using specification here allows you to build a very wide range of possible conditions on method parameters.

Building up a specification through constructors is a classic example of where a fluent interface can make things much more readable, so there's no suprise that pretty much all of the constraint classes can be created through top level methods defined on MockObjectTestCase (well actually a superclass: MockObjectSupportTestCase), which JMock uses for Object Scoping. In the example we are tracing, it's a simple equality case.

class MockObjectSupportTestCase...
    public IsEqual eq( Object operand ) {
        return new IsEqual(operand);
    }

The final term in our example expression is will.

  
  mainframe.expects(once())
    .method("buy").with(eq(QUANTITY))
    .will(returnValue(TICKET));

By now the implementation scheme should be quite familiar to you. The choice of methods for this clause is driven by the interface returned from the previous clause in the chain - in this case a MatchBuilder.

public interface MatchBuilder extends StubBuilder {
    MatchBuilder match( InvocationMatcher customMatcher );
    MatchBuilder after( String previousCallID );
    MatchBuilder after( BuilderNamespace otherMock, String previousCallID );
}

public interface StubBuilder extends IdentityBuilder {
    IdentityBuilder will( Stub stubAction );
    IdentityBuilder isVoid();
}

The interface is implemented in the builder.

class InvocationMockerBuilder...
    public IdentityBuilder will( Stub stubAction ) {
        setStub(stubAction);
        return this;
    }
    private void setStub( Stub stubAction ) {
        mocker.setStub(stubAction);
    }

The implementation takes an argument which it adds to the abstract representation. I create this argument object using a top level method.

class MockObjectTestCase...
    public Stub returnValue( Object o ) {
        return new ReturnStub(o);
    }

JMock provides an interesting example of an internal DSL built in a language environment that isn't considered particulrly friendly to building them. It's further interesting in how it deals with design choices, in particular

• the choices between Method Chaining and Object Scoping
• the use of interfaces to guide usage

Some terms in JMock are implemented through methods on InvocationMockerBuilder, which are chained by having each method return this. Examples of Method Chaining include method, with and will. Other terms, such as once, eq and returnValueare defined through Object Scoping. The rationale for this division was ease of extension; Method Chaining was used to fix the expected order of the core evaluation, while Object Scoping was used for ease of extension.

As an example consider the argument matching system, represented in my example by eq. As I said earlier, this introduces a specification for the method arguments. If I want to extend what I have in this specification I can create the necessary objects for the abstract representation and then introduce new top level methods just by defining them into my specific test class. In this case top level methods can easily be added.

The basic chaining structure of method-with-will is fixed, however. If I want to extend that I'd need to slide in my own subclass of InvocationMockerBuilder which requires much more intricate extending of the framework. But by fixing this sequence JMock can provide better diagnostics.

There's no inherent reason why a builder can't be more easily extended. If you want to make it easier to extend a builder, there are ways to do it. As usual with any system the designers have to choose what is is easy to extend and what is hard. Trying to make everything easy to extend results in too little structure which only makes everything much harder to use.

Using interfaces to guide the Method Chaining is one of the most interesting ideas in JMock. The core grammer of JMock is method-with-will. This sequence is enforced by having each method in the chain returning an interface that restricts the next method in the chain.

One thing that isn't obvious from what I've shown so far is that these clauses are optional. You can leave out any of method, with and will. An expression such as

.method("buy").will(returnValue(TICKET))

is legal, as is

  
  mainframe.expects(once())
    .with(eq(QUANTITY))
    .will(returnValue(TICKET));

or even

  
  mainframe.expects(once())
    .will(returnValue(TICKET));

Optional clauses like this are provided by using inheritance between the interfaces. The with clause is introduced with the ArgumentsMatchBuilder, the will clause comes from the MatchBuilder interface. The with clause is made optional by having ArgumentsMatchBuilder be a subtype of MatchBuilder, which effectively means you can use will whenever you want to introduce a with. Similarly the method clause is made optional by having its interface (NameMatchBuilder) be a subtype of ArgumentsMatchBuilder.