StoryTest
24 April 2013
tags:
Story tests are BusinessFacingTests used to describe and verify the software delivered as part of a UserStory. When a story is elaborated the team creates several story tests that act as acceptance criteria for the story. The story tests can be combined into a regression suite for the software and provide traceability from the requirements (user stories) to tests and (through execution) to the behavior of the system. Story tests are usually BroadStackTests.
User stories are popular because they offer a simple workflow, each story adds new tests to the story test suite. However story tests often lead to problems. Regularly adding story tests leads to a large body of tests, often with significant duplication between them. When behavior needs to change in later iterations of the project, duplication in tests can take a painful amount of time to update. Furthermore broad-stack story tests take a long time to execute, which is why having a lot of them violates the TestPyramid. As a result many people recommend using just a few UserJourneyTests together with business-facing ComponentTests.
UserJourneyTest
24 April 2013
tags:
User-journey tests are a form of BusinessFacingTest, designed to simulate a typical user's "journey" through the system. Such a test will typically cover a user's entire interaction the system in order to achieve some goal. They act as one path in a use case.
They are usually BroadStackTests and as such, are usually slow to execute and prone to being brittle. Consequently suites of user journey tests usually aren't built to be comprehensive tests of a system's behavior. Usually you will have only a few user journey tests to exercise the integration of the system as a whole - probably only one path for each use case (usually the happy path). Verification of all the variations in behavior is left to tests done in different styles, usually with more focused coverage.
In contrast to StoryTests, user journey tests are not tied to user stories. When you play a story you look at the existing user journey tests and modify them to support any change in behavior implied by the user story, only rarely does a user story lead to an entirely new user journey test.
BusinessFacingTest
24 April 2013
tags:
A business-facing test is a test that's intended to be used as an aid to communicating with the non-programming members of a development team such as customers, users, business analysts and the like. When automated, they describe the system in domain-oriented terms, ignoring the component architecture of the system itself. Business-facing tests are often used as acceptance criteria, having such tests pass indicates the system provides the functionality that the customer expects.
Automated business-facing tests are often represented in some form of DomainSpecificLanguage, since this helps communication with non-programmers and also helps give programmers a mechanism that helps them step back from the details of the code. Tools like Cucumber and Twist help design such DSLs and provide mechanisms to bind them to the system under test.
Business-facing tests are commonly implemented as BroadStackTests since their user-oriented expression suggests treating the system under test as a black box. However there are significant advantages to implementing business-facing tests as ComponentTests since this often results in easier maintenance and faster execution.
I'm a big fan of automated testing, but it's important to recognize that manual tests play a significant role in business-facing testing. Techniques such as exploratory testing and usability testing are inherently manual activities and are essential parts of a well-balanced test portfolio.
StoryTests and UserJourneyTests are two common forms of business-facing test. The term business-facing test comes from Brian Marick's test quadrant.
UserStory
22 April 2013
tags:
User Stories are chunks of desired behavior of a software system. They are widely used in agile software approaches to divide up a large amount of functionality into smaller pieces for planning purposes. You also hear the same concept referred to as a feature, but the term "story" or "user story" has become prevalent in agile circles these days.
Kent Beck first introduced the term as part of Extreme Programming to encourage a more informal and conversational style of requirements elicitation than long written specifications. The essence of a story can be written on a single note card (Kent and I prefer 3" by 5"). Stories are deliberately not fleshed out in detail until they are ready to be developed, you only need enough understanding to allow prioritization with other stories.
Bill Wake came up with the INVEST mnemonic to describe the characteristics of good stories:
- Independent: the stories can be delivered in any order
- Negotiable: the details of what's in the story are co-created by the programmers and customer during development.
- Valuable: the functionality is seen as valuable by the customers or users of the software.
- Estimable: the programmers can come up with a reasonable estimate for building the story
- Small: stories should be built in a small amount of time, usually a matter of person-days. Certainly you should be able to build several stories within one iteration.
- Testable: you should be able to write tests to verify the software for this story works correctly.
A common way to formulate stories is the "As a … I want … So that …" form. The "As a" clause refers to who wants the story, "I want" describes what the functionality is, "so that" describes why they want this functionality. The "so that" part provides important context to understand to help get from what the customer think they want to providing what they actually need.
Mike Cohn wrote what is now the standard
book on writing user stories
ComponentTest
22 April 2013
tags:
A component test is a test that limits the scope of the exercised software to a portion of the system under test. It is in contrast to a BroadStackTest that's intended to exercise as much of the system as is reasonable.
The difference between broad-stack and component tests is one of degree rather than an absolute difference. Component tests can be as large or small as you define your components. The essence of the difference is that component tests deliberately neglect parts of the system outside the scope of the test. This is usually done by manipulating the system through internal code interfaces, using tools like xunit testing tools, and by using TestDoubles to isolate the code under test from other components.
Component tests are usually easier to write and maintain than broad-stack tests. They are also faster to run, since they only hit part of the code base. In theory a system with excellent component test coverage should be free of bugs, but in practice bugs like to lurk in the interactions between components. Therefore it's good to use the TestPyramid and combine a large quantity of component tests with a smaller amount of broad-stack tests.
BroadStackTest
22 April 2013
tags:
A broad-stack test is a test that exercises most of the parts of a large application. It's often referred to as an end-to-end test or full-stack test. It lies in contrast to a ComponentTest, which only exercises a well-defined part of a system.
The difference between a broad-stack test and a component test is a continuum rather than a clear line. One area where bits can be missing from the fullness of the stack is how the test manipulates the application. Broad-stack tests often manipulate the application through a UI, such as testing web applications with tools like Selenium and Sahi. However a SubcutaneousTest can also be a broad-stack test if it continues to exercise most of the rest of the software. To further limit the scope, a test that exercises an application through a service interface can also be considered to be broad-stack test of the the server.
The other area where these tests don't cover the full breadth of the stack lies in connection to remote systems. Many people, including myself, think that tests that call remote systems are unnecessarily slow and brittle. It is usually better to use TestDoubles for these remote systems and check the doubles with IntegrationContractTests
Broad-stack tests have the advantage of exercising the application with all its parts connected together and thus can find bugs in the interaction between components in the way that component tests cannot. However broad-stack tests also tend to harder to maintain and slower to run than component tests. As a result the TestPyramid suggests using fewer broad-stack tests.
JavascriptPromise
22 April 2013
tags:
In Javascript, promises are objects which represent the pending result of an asynchronous operation. You can use these to schedule further activity after the asynchronous operation has completed by supplying a callback.
aPromise = someAsyncOperation();
aPromise.done(function() {
// runs if all went well
});
aPromise.fail(function() {
// runs if something went wrong
});
aPromise.always(function() {
// runs either way
});
As well as providing a clear interface to schedule activity with asynchronous tasks, they also compose.
composedPromise = $.when(anAsyncFunction(), anotherAsyncFunction());
In this form (using jQuery promises) the composed promise will run its done handlers when all the passed promises succeed and its fail handlers if any of them fail.
There are various forms of promises in javascript, annoyingly they have subtly different APIs and vocabularies. Probably the most used is jQuery's Deferred Object.
You also hear these concepts described as futures and deferreds. These concepts appear in many languages, not just javascript, often with concurrency in mind as much as asynchrony.
For more information I suggest getting a copy of Trevor Burnham's Async JavaScript. If you want a web article, I found Burnham has a short but useful article summarizing them.
