Templated Generation

Generate output by hand-writing an output file and placing template call-outs to generate variable portions.

How it Works

The basic idea behind Templated Generation is to write the output file you desire inserting call-outs for all the bits that vary. You then use a template processor with the template file and a context that can fill the call-outs to populate the real output file.

Templated Generation is a very old and familiar technique. Anyone who has used mail-merge facilities in a word processor uses Templated Generation. Templated Generation is very common in web development as many web sites with dynamic content use Templated Generation. In those forms the entire document is a template, but Templated Generation also works in smaller contexts too. The old faithful printf function in C is an example of using Templated Generation to print out a single string at a time. In the context of code generation, I usually use Templated Generation for the cases where the whole output document is a template, but printf reminds us that Templated Generation and Transformer Generation can be very inter-mixed. Textual macro processors, another old standby in software development, are also another form of Templated Generation.

With Templated Generation there are three main components: a templating engine, template, and context. The template is is the source text of the output file, with the dynamic parts represented by call-outs. The call-outs involve references to the context which will populate the dynamic elements when the generation takes place. The context therefore acts as a source for dynamic data, essentially the data model for the template generation. The context may be a simple data structure, but it also may be a more complex programmatic context, different templating tools use different forms of context. The templating engine is the tool that brings template and context together to produce the output. A controlling program will execute the templating program with a particular context and template to produce an output file, and may run the same template with multiple contexts to produce multiple outputs.

The most general form of template processors allow arbitrary host code expressions to be placed in the call-outs. This is a common mechanism used by tools like JSPs and ASPs. Like any form of Host Code Embedment it needs to be used with care, otherwise the structure of the host code can overwhelm the template. Thus I strongly recommend that if you have a template processor that embeds arbitrary host code, you confine yourself to simple function calls only within the call-outs, preferably using an Embedment Helper.

Because it's common for template files to get thoroughly messed up due to putting in too much host code, many template processors don't allow arbitrary host code in the call-outs. Such tools provide a specific templating language to use in the call-outs instead of host code. This templating language is usually quite restricted to encourage simpler call-outs and preserve the clarity of the template structure. The simplest kind of templating language treats the context as a map and provides expressions to look up values in that map and insert them into the output. While this mechanism is sufficient for simple templates, there are common cases where you need more.

A common driver for more complex templating is where you need to generate output for items in a collection. This usually requires some kind of iterative construct, such as a loop. Conditional generation is another common need, where different template output is needed depending upon a value in the context. Often you find duplication of chunks of template source, which suggests the need for some kind of subroutine mechanism inside the template language itself.

I'm not going to delve into the different ways that various templating systems handle these various cases, although that is an interesting diversion. My general advice here is to be as minimalist as possible since the strength of Templated Generation is directly proportional to how easy it is to visualize the output file by looking at the template.

When to use it

The great strength of Templated Generation is when you can look at the template file and easily understand what the generated output will look like. This is most useful when there is quite a lot of static content in the output and the dynamic content is occasional and simple.

So the first indicator to use Templated Generation is where there is a lot of static content in the generated file. The greater the proportion of static content, the more likely that it will be easier to use Templated Generation. The second driver is the complexity of what dynamic content you need to generate. The more you use iterations, conditionals, and advanced templating language features, the less likely that you can easily comprehend what the output will look like from the template file. When this happens you should consider Transformer Generation instead.

Example: Generating the Secret Panel State Machine with Nested Conditionals (Velocity and Java generating C)

Generating code for nested conditionals in a state machine is a good case where the static output is relatively large and the dynamic part fairly simple - good cases for Templated Generation. For this example I'll generated the code that I discussed in the example for Model Ignorant Generation. So you can get a sense of what we want, here is the entire output file.

#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
#include <string.h>
#include "sm.h"
#include "commandProcessor.h"

#define EVENT_doorClosed "D1CL"
#define EVENT_drawOpened "D2OP"
#define EVENT_lightOn "L1ON"
#define EVENT_doorOpened "D1OP"
#define EVENT_panelClosed "PNCL"

#define STATE_idle 1
#define STATE_active 0
#define STATE_waitingForDraw 3
#define STATE_unlockedPanel 2
#define STATE_waitingForLight 4

#define COMMAND_lockDoor "D1LK"
#define COMMAND_lockPanel "PNLK"
#define COMMAND_unlockPanel "PNUL"
#define COMMAND_unlockDoor "D1UL"

static int current_state_id = -99;


void init_controller() {
  current_state_id = STATE_idle;
}



void hard_reset() {
  init_controller();
}

void handle_event_while_idle (char *code) {
  if (0 == strcmp(code, EVENT_doorClosed)) {
    current_state_id = STATE_active;
  }
  if (0 == strcmp(code, EVENT_doorOpened)) {
    current_state_id = STATE_idle;
    send_command(COMMAND_unlockDoor);
    send_command(COMMAND_lockPanel);
  }
}
void handle_event_while_active (char *code) {
  if (0 == strcmp(code, EVENT_lightOn)) {
    current_state_id = STATE_waitingForDraw;
  }
  if (0 == strcmp(code, EVENT_drawOpened)) {
    current_state_id = STATE_waitingForLight;
  }
  if (0 == strcmp(code, EVENT_doorOpened)) {
    current_state_id = STATE_idle;
    send_command(COMMAND_unlockDoor);
    send_command(COMMAND_lockPanel);
  }
}
void handle_event_while_waitingForDraw (char *code) {
  if (0 == strcmp(code, EVENT_drawOpened)) {
    current_state_id = STATE_unlockedPanel;
    send_command(COMMAND_unlockPanel);
    send_command(COMMAND_lockDoor);
  }
  if (0 == strcmp(code, EVENT_doorOpened)) {
    current_state_id = STATE_idle;
    send_command(COMMAND_unlockDoor);
    send_command(COMMAND_lockPanel);
  }
}
void handle_event_while_unlockedPanel (char *code) {
  if (0 == strcmp(code, EVENT_panelClosed)) {
    current_state_id = STATE_idle;
    send_command(COMMAND_unlockDoor);
    send_command(COMMAND_lockPanel);
  }
  if (0 == strcmp(code, EVENT_doorOpened)) {
    current_state_id = STATE_idle;
    send_command(COMMAND_unlockDoor);
    send_command(COMMAND_lockPanel);
  }
}
void handle_event_while_waitingForLight (char *code) {
  if (0 == strcmp(code, EVENT_lightOn)) {
    current_state_id = STATE_unlockedPanel;
    send_command(COMMAND_unlockPanel);
    send_command(COMMAND_lockDoor);
  }
  if (0 == strcmp(code, EVENT_doorOpened)) {
    current_state_id = STATE_idle;
    send_command(COMMAND_unlockDoor);
    send_command(COMMAND_lockPanel);
  }
}

void handle_event(char *code) {
  switch(current_state_id) {
  case STATE_idle: {
    handle_event_while_idle (code);
    return;
  }
  case STATE_active: {
    handle_event_while_active (code);
    return;
  }
  case STATE_waitingForDraw: {
    handle_event_while_waitingForDraw (code);
    return;
  }
  case STATE_unlockedPanel: {
    handle_event_while_unlockedPanel (code);
    return;
  }
  case STATE_waitingForLight: {
    handle_event_while_waitingForLight (code);
    return;
  }
  default: {
    printf("reached a bad spot");
    exit(2);
  }
  }
}

The templating engine I'm using here is Apache Velocity, which is a common and easy to understand templating engine available for Java and C#.

I can look at this overall file as various segments of dynamic content that needs to be generated. Each segment is driven by a collection of elements that I can iterate through to generate the code for that segment.

I'll begin by looking at how I generate the event definitions (#define EVENT_doorClosed "D1CL" etc.) If you follow how this works, the rest pretty much falls into place.

I'll start with the code in the template

template file...
#foreach ($e in $helper.events)
#define $helper.eventEnum($e) "$e.code"
#end

Unfortunately one of the confusions here is that both the C pre-processor (itself a form of Templated Generation) and velocity both use '#' to indicate template commands. #foreach is a command for velocity while #define is a command for the C pre-processor. Velocity will ignore any commands it doesn't recognize so it will just treat #define as text.

#foreach is a velocity directive to iterate over a collection. It takes each element from from $helper.events in turn and runs its body with $e set to that element. In other words a typical foreach style construct.

$helper.events is a reference to the context of the template. I'm using an Embedment Helper and have thus placed just the helper, in this case an instance of SwitchHelper, into the velocity context. The helper is initialized with a state machine and the events property provides access to it.

class SwitchHelper...
  private StateMachine machine;

  public SwitchHelper(StateMachine machine) {
    this.machine = machine;
  }
  public Collection<Event> getEvents() {
    return machine.getEvents();
  }

Each event is an object from the Semantic Model. As a result I can use the code property directly. However creating a constant to reference in the code is a little more work, for this I put some code in the helper.

class SwitchHelper...
  public String eventEnum(Event e) {
    return String.format("EVENT_%s", e.getName());
  }

Of course there's no absolute need to use a constant here, I could just use the event code itself. I generate the constant because I prefer even my generated code to be readable.

As is usually the case, the commands use exactly the same mechanism as the events, so I'll leave that code to your imagination.

To generate the states, I need to sort out an integer constant.

template file...
#foreach ($s in $helper.states)
#define $helper.stateEnum($s) $helper.stateId($s)
#end

class SwitchHelper...
  public Collection<State> getStates() {
    return machine.getStates();
  }
  public String stateEnum(State s) {
    return String.format("STATE_%s", s.getName());
  }
  public int stateId(State s) {
    List<State> orderedStates = new ArrayList<State>(getStates());
    Collections.sort(orderedStates);
    return orderedStates.indexOf(s);
  }

Some readers may be uncomfortable with me generating and sorting a list of states every time I need an id. Rest assured that if this was a performance issue I'd cache the sorted list, but since it isn't I don't.

With all the declarations generated, I can now generate the conditionals. First the outer conditional switches on the current state.

template file...
void handle_event(char *code) {
  switch(current_state_id) {
#foreach ($s in $helper.states)
  case $helper.stateEnum($s): {
    handle_event_while_$s.name (code);
    return;
  }
#end
  default: {
    printf("reached a bad spot");
    exit(2);
  }
  }
}

The inner conditionals switch on the input event, I've broken these into separate functions.

template file...
#foreach ($s in $helper.states)
void handle_event_while_$s.name (char *code) {
#foreach ($t in $helper.getTransitions($s))
  if (0 == strcmp(code, $helper.eventEnum($t.trigger))) {
    current_state_id = $helper.stateEnum($t.target);
#foreach($c in $t.target.commands)
    send_command($helper.commandEnum($c));
#end
  }
#end
}
#end

To get the transitions for each state I need both the transitions defined in the Semantic Model and the reset event transitions.

class SwitchHelper...
  public Collection<Transition> getTransitions(State s) {
    Collection<Transition> result = new ArrayList<Transition>();
    result.addAll(s.getTransitions());
    result.addAll(getResetTransitions(s));
    return result;
  }

  private Collection<Transition> getResetTransitions(State s) {
    Collection<Transition> result = new ArrayList<Transition>();
    for (Event e : machine.getResetEvents()) {
      if (!s.hasTransition(e.getCode()))
        result.add(new Transition(s, e, machine.getStart()));
    }
    return result;
  }