Here is a list of recent updates the site. You can also get this information as an RSS feed and I announce new articles on twitter.
I use this page to list both new articles and additions to existing articles. Since I often publish articles in installments, many entries on this page will be new installments to recently published articles, such announcements are indented and don’t show up in the recent changes sections of my home page.
Thu 20 Jan 2022 10:50 EST
Yesterday Ian Cartwright, Rob Horn, and James Lewis
described the Critical Aggregator and how it can metastasize into an
invasive form. When a legacy system has such an Invasive Critical
Aggregator it's often best, if a little counter-intuitive, to Divert
the Flow of data by building a new critical aggregator first. Once
this is done, we have far more freedom to change or relocate the various
upstream data sources.
Wed 19 Jan 2022 10:53 EST
Business Leaders often need to make decisions that are influenced by a
wide range of activity throughout the whole enterprise. To support this
systems often have a what Ian Cartwright, Rob Horn, and James
Lewis call a Critical Aggregator: a component whose job is to
visit various other systems and pull this information together. A critical
aggregator is important, but often metastasizes into an Invasive Critical Aggregator
Tue 18 Jan 2022 12:42 EST
Continuing his exploration of important patterns to maintain consistency
across a cluster, Unmesh Joshi now looks at Two Phase
Commit. It's broadly the most familiar approach, but comes with lots of
complexities to make it work in practice over unreliable networks.
Wed 12 Jan 2022 10:11 EST
Ian Cartwright, Rob Horn, and James Lewis are also
back with the New Year with a couple more articles from Patterns of Legacy
Displacement in the funnel for the next couple of weeks. This one
describes a Legacy Mimic: a part of the new system designed
to make the old system think that nothing has changed.
Tue 11 Jan 2022 10:09 EST
One of the core challenges in a distributed system is keeping the state
synchronized across all the nodes, especially when neither the nodes, or
the connections between them, are reliable. The core approach to handle
with is a replicated log: using the write-ahead log pattern
over the cluster. Unmesh Joshi shows how this works using
its most common implementation: the Raft protocol.
Wed 05 Jan 2022 10:53 EST
Unmesh Joshi is ready to start the New Year with a few
more of his Patterns of Distributed Systems. With this one he attempts the
tricky task of explaining Paxos. This is a well-known
protocol developed by Leslie Lamport, for nodes to reach a reliable
consensus when both they, and the network, are prone to unexpected
failure. Although it's well-known, it's also notoriously difficult to
understand, indeed we had considerable difficulty getting our heads around
it. We hope this description makes it a bit easier for those who follow
Sun 26 Dec 2021 11:21 EST
I listen to a lot of interesting music, so picked out my six favorite
new-to-me albums this year. I hope you find something interesting to
groove to in there.
Wed 15 Dec 2021 10:10 EST
Andrew finishes his article on how to scale software architecture by
looking at how this technique works in practice, and also outlines how
things can go wrong.
Tue 14 Dec 2021 09:27 EST
Brandon finishes his article on how we should look at
integration by arguing that it is a strategic element of an enterprise's
infrastructure. You can buy the best products in the world but "none of it
will make you competitive in a digital world if you continue to treat
integration as a tactical nuisance to overcome so you take advantage of
those new systems."
Mon 13 Dec 2021 10:05 EST
Andrew's fourth supporting element for the Advice Process is using a
tech radar to capture and map out your local version of the technology
trends you see across your organization.
Thu 09 Dec 2021 11:44 EST
Having architectural principles is not new, but in a world of
highly-autonomous-teams they become essential because they are the means
by which an aligned delivery direction is achieved without the need for
control. In the latest installment, Andrew talks about what makes a good
principle, and how they work with the Advice Process.
Thu 09 Dec 2021 10:35 EST
Thus far, Brandon has has explained why general purpose languages are
better for integration. In this latest installment he
explains that there are cases when commercial integration tools make
Wed 08 Dec 2021 10:07 EST
Andrew uses the Architecture Advisory Forum as a regular and recurring place and
time for conversations. It's a weekly, hour-long, open invite meeting used
to gather advice and share information across a broad group.
Tue 07 Dec 2021 09:50 EST
Many commercial integration tools market their ability to own the
integration landscape and call out to general purpose languages as needed.
While Brandon can appreciate the marketing behind such messaging — it promotes
product penetration and lock-in — as architectural guidance, it is exactly
backwards. Instead, we should almost always manage the interface evolution
in a general purpose language for at least two reasons: so we can better
manage the complexity of maintaining a clean interface, and so that we
avoid the gravitational pull of our tool's mental model when making
strategic integration decisions.
Wed 01 Dec 2021 10:59 EST
The Advice Process works when supported by four elements. Andrew
describes the first of these, Decision Records, which act as a tool for
thinking about and recording the decision process.
Wed 01 Dec 2021 10:50 EST
Brandon points out that while we have historically drawn up our project plans and costs around
the boxes—the digital products we are introducing—the lines are the
hidden and often primary driver of organizational tech debt. They are the
reason that things just take longer now than they used to.
Tue 30 Nov 2021 09:45 EST
“Build versus buy” decisions are everywhere today, and rightly so.
Building software is risky and expensive, and software product companies
can spread that risk and expense across multiple customers. But my
colleague Brandon Byars argues that the kinds
of tools that are available to buy for systems integration are not
products that directly solve a business problem.
Tue 30 Nov 2021 09:43 EST
Like many modern software architects, Andrew Harmel-Law struggles with
the need to scale architectural thinking to larger organizations while
allowing teams to be as autonomous as possible. The approach he's
currently using is the "Advice Process", that encourages and supports
these teams to be engaged in broader architectural decision making. In
this first installment, Andrew describes this advice process, later
installments will dig into four supporting elements that help make it work.
Wed 10 Nov 2021 13:10 EST
Evan Bottcher understands that good technical design decisions
are very dependent on context. Teams that regularly work together on common
goals are able to communicate regularly and negotiate changes quickly. These
teams exhibit a strong force of alignment, and can make technology
and design decisions that harness that strong force. As we zoom out in a
larger organisation an increasingly weak force exists between teams
and divisions that work independently and have less frequent collaboration.
Recognising the differences in these strong and weak forces allows us to
make better decisions and give better guidance for each level, allowing for
more empowered teams that can move faster.
Wed 10 Nov 2021 13:04 EST
Within each normal-sized team, limit the choice of alternatives for any class
of technology to three. These are: the current sensible default, the one we're
experimenting with as a trial, and the one that we hate and want to retire.
The conversation goes like this: We want to introduce a new messaging
technology. How many do we have already in place? Oh we have three in active
use, including one that's considered legacy and we're partway through
migrating off and one that we experimented with previously but didn't gain
traction. Ok, so we're at our limit now. If we want to add another messaging
tech then we have two choices. Either migrate all of our apps off the legacy
tech, or properly rid ourselves of the failed experiment. This is quite
closely related to the idea of capping the number of Innovation Tokens in
use within your teams.
At a team level these kinds of limits are relatively easy
to maintain and discuss and act upon, because we have common priorities and
ways of working and high trust, high bandwidth communication. At the scope of
the whole organisation the challenge is similar, but getting alignment takes a
lot longer and doing actual migration and consolidation work can take a long
time - so we sometimes have to allow for more variation in technology. We also use
different techniques to discuss and communicate the status of our preferred technologies.
An approach we use at MYOB to engage our whole organisation in broader decisions about
technology is by publishing our own MYOB Technology Radar, following the
format of the Thoughtworks Technology Radar.
This approach of building our own radar
involves taking input from all of our verticals and teams, and making a clear
statement on what technologies we encourage teams to adopt, trial, or more
importantly which ones to keep clear of.
Tue 02 Nov 2021 09:39 EDT
Integrating the necessary security controls and audit capabilities to
satisfy compliance requirements within a DevOps culture can capitalize on
CI/CD pipeline automation, but presents unique challenges as an
organization scales. Understanding the second order implications and
unintended consequences caused by the chosen implementation is key to
building an effective, secure, and scalable solution. My colleague Carl
Nygard describes how to think of these choices through a
series of four patterns for handling compliance.
Tue 26 Oct 2021 10:24 EDT
James Shore has revised his book "The Art of Agile Development". I'm
pleased to write a foreword for this book as it is solid guide to learning
how to get past faux-agile and develop the skills you need to get the
benefits of the agile way of work.
Thu 14 Oct 2021 14:29 EDT
Bromley Mtn, VT (2021)
Wed 06 Oct 2021 10:30 EDT
Those of us developing software don’t need to be told what a big impact it’s had on humanity this century. I’ve long maintained that this places a serious responsibility on our profession. Whether asked to or not, we have a duty to ensure our systems don’t degrade our society. But in the tumult of so many software projects, it can be difficult to step back and understand the implications of our work. In the last few months my colleagues at Thoughtworks have been putting together a catalog of techniques that we can use to address this problem, which they published as the Responsible Tech Playbook.
The playbook is a free PDF download of about 50 slides, the bulk of which is a summary of a dozen tools and methods that teams can use to better understand their responsibilities. Each summary is a couple of slides outlining the basics of the technique: what is it, who created it, when we should use it, how it works, and our perspective on its place in our development efforts.
My colleagues highlighted three techniques that are particularly well-suited as first steps into this collection:
Ethical Explorer (Omidyar Network) is a set of cards that use a metaphor of an explorer and risk zones to help a team discuss and understand potential dangers in their product plans.
Consequence Scanning (Doteveryone, now hosted by the Open Data Institute) is some workshop activities and materials to uncover the unexpected consequences associated with a technology. It encourages teams to uncover these consequences and develop plans to mitigate any problems that they find.
Tarot Cards of Tech (Artefact) are cards describing “provocations” such as: how a product could be used in unexpected ways, what users may be excluded, and what might happen if the product is over-used.
As these example suggest, these techniques are mostly ways to conduct structured workshops that help people explore different perspectives on their product. They also encourage engaging a wider range of stakeholders into the discussion and coming up with ways to monitor for bad outcomes and deal with them should they show up.
As professionals, we must take responsibility for the outcomes of our work - whether or not it matches our intention. We can’t avoid that responsibility by saying we are following our manager’s instructions, nor can we avoid the necessity of considering how our algorithms may lead to malignancies. Techniques like this help us explore our responsibilities and take thoughtful action to live up to them.
You can download the Responsible Tech Playbook as a PDF from the Thoughtworks web site.
Last year my colleague JoJo Swords wrote an article that explains more about why it matters that we understand our responsibility for our technology. In conjunction with that, Rebecca Parsons and Chad Wathington addressed the topic in a webinar.
Wed 08 Sep 2021 09:43 EDT
I've written a fair bit about how using pull requests can encourage a
low integration frequency, increasing cycle time and discouraging
refactoring. Rouan Wilsenach has had success using an approach that categorizes
changes as Ship/Show/Ask - using this classification to decide whether and
how to use pull requests.
Thu 26 Aug 2021 12:15 EDT
A couple of months ago I announced that I was stepping back from
speaking. A few people wondered whether I would still be writing. I did
indicate in that article that I am, but I felt it may be worth saying a
bit more about what I’m concentrating on these days.
Tue 10 Aug 2021 10:51 EDT
We often need to access APIs from foreign codebases, and these foreign
codebases usually have different vocabularies to ours. I've found it
useful to encapsulate this interaction with a gateway that
translates between our code and foreign code.
Thu 29 Jul 2021 09:49 EDT
To illustrate how these patterns work in practice, Ian, Rob, and James
describe an example of how one of our teams used a number of Legacy
Modernization Patterns to successfully replace integration middleware
critical to the operation of their client's business as part of a larger
legacy modernization programme. They combined patterns and refactorings to
successfully manage risk to the business, and facilitate eating this
particularly gristly part of the elephant.
Tue 27 Jul 2021 09:08 EDT
On many occasions when my colleagues find themselves talking to IT
executives they hear how the executives have a suite of aging applications
built using soon to be, if not already end of life technologies. More
often that not these systems are hosted in costly data centers managed by
3rd parties and with inflexible contracts. These applications are critical
to the successful operation of the business, while at the same time being
one of the largest sources of business and operational risk.
One approach in this situation is to try to minimize the impact of
replacement on the broader organization by 'simply' replacing the
technology while leaving everything else 'as is'. Whilst Feature Parity
often sounds like a reasonable proposition, we have learnt the hard way
that people greatly underestimate the effort required, and thus misjudge
the choice between this and the other alternatives.
Wed 21 Jul 2021 10:00 EDT
To do effective legacy displacement, we need to figure out how to break
down the problem into manageable pieces. Extract Product Lines does this
by identifying product lines and using them as the basis for migration.