Writing bliki

Neal Ford and Rebecca Parsons will be working with me again on a DSL tutorial at JAOO and QCon San Francisco.

The two biggest event I have coming up are two more conferences run by JAOO. The first is the mother-ship conference in Århus, which has become my favorite conference of the last few years. Later I'll be at QCon San Francisco. At both conferences I'll be giving a tutorial on Domain Specific Languages with Neal Ford and Rebecca. In addition I'm giving a shorter talk in the regular JAOO session on the patterns for internal DSLs, working directly on the topics that I've recently got into a coherent state for the book.

On a different note, I'll be talking at Steve McConnell's Construx Executive Summit. I find it rather odd to be invited to this as it's a conference aimed at executives and that's not my usual audience. I'll be talking about why software design is important and to get good design - which boils down to getting good designers and how to get and keep those.

I don't not write much production code these days, but I still spend quite a few hours writing code. This code is a particular form of code, meant for explaining ideas in books. Book code isn't quite like real code, there are some different forces to consider when writing it.

One question is the choice of language. I need to write code in a language that as many of my readers can read and follow. I try to write about ideas that are platform independent, but I need code to be precise. So I need to pick some widely readable language that people can follow.

In my early days the two largest OO languages were Smalltalk and C++. Both had faults, Smalltalk was too weird for non-smalltalkers and C++ was too full of sharp edges to get right. Java was a godsend for me. Anyone with a C/C++ background could read it. Even most smalltalkers could hold their noses and understand what I was coding. That's why the refactoring book was in Java.

Later on .NET appeared. The nice thing here was the C# was mostly the same as Java, so I could use the two pretty interchangeably. I liked to use both to reinforce that the ideas I was writing about were useful in either case.

That situation is getting more difficult these days, particularly with writing about DomainSpecificLanguages. Java and C# are diverging, and some things I want to illustrate require features that neither of them have. I do much of my personal programming in Ruby, which is very well suited to DSLs, so I will use Ruby as my first choice for this situation. But other languages have contributions too. I need to balance illustrating various language capabilities for DSLs against letting the book become a hodgepodge of language tidbits.

Another issue with book code is to beware of using obscure features of the language, where obscure means for my general reader rather than even someone fluent in the language I'm using. A good example of this is that when I write examples using Ruby, I've often shied away from using CollectionClosureMethods, even though I use them heavily in my own Ruby code. This is because I consider that programmers from a curly-brace background will find it harder to understand those kinds of expressions. So I sacrifice fluent Ruby in order to reach those readers.

Again this is much harder for a DSL book. Internal DSLs tend to rely on abusing the native syntax in order to get readability. Much of this abuse involves quirky corners of the language. Again I have to balance showing readable DSL code against wallowing in quirk.

Last week I got the newest book in my signature series: xUnit Test Patterns by Gerard Meszaros. I've been working with Gerard with it on and off for a couple of years, so I'm fairly familiar with its contents, but somehow seeing the physical copy gave me quite a shock. Somehow it hadn't dawned on me how big the book was - 883 pages, easily the biggest book in my series.

On the whole I'm not keen on big books, I was very proud of keeping UML Distilled so small. A book this big scares me, how will I ever get time to read it?

But xUnit Test Patters isn't as scary as it looks, because it's actually two books in one set of covers. In this it follows a style I also used in P of EAA. The first book is a narrative book, designed to be read "cover to cover". The narrative book is something small enough to be digestible, in xUnit Test Patterns it's 181 pages, 106 in P of EAA. The second book is reference material, which is designed not be be read cover to cover (although some people do) but instead to be dipped into when needed. The idea is that you read the narrative book to get a good understanding of the field, then put it on your shelf so it's handy to grab when you need to delve into the reference section.

I read a lot of history books and I've often wished that the author had written a duplex book. History books often need detail to expand upon a particular topic or to describe the evidence for a point of view. The result, however can be a long book. One example is a favorite book of mine: Guns, Germs and Steel. I'm really glad I read it, and I do recommend it. But it did feel long, and I wonder how it would have worked in duplex.

How to Read a Book suggests that when you read a book for the first time you should deliberately skip-read it, unhesitatingly skipping over detail sections. I read very fast, which helps me, but although fast or skip reading is a plus, I'd rather the book was designed to help. My personal rule of thumb is that 200 pages is the limit for a narrative cover-to-cover technical book. If it goes over that you need some way to allow people to get the core information with less bulk. A duplex book isn't the only way to it, such as selected bolded paragraphs in the Timeless Way of Building, which worked pretty well for me. I'm sure there are other techniques that I haven't run into yet, but the duplex is currently top of my list.

A duplex book is really a specific case of a more general principle to organize a book in gradually increasing sections. The Pick Axe is a good example of this. The first two chapters are a quick overview of ruby in 24 pages. Then you have 280 pages of tutorial, followed by 500 odd pages of reference material. The first few chapters of the Enterprise Integration Patterns book is an overview (95 pages) with rest (~550 pages) reference. Books like this often don't make these successive layers of revelation as clear as they might, however.

An interesting question is whether we can do more with the duplex book format. A lot people won't realize that xUnit Test Patterns is a duplex book because it looks like one thick book (the back cover actually claims it's three books by considering the reference section as two separate sections). Could we physically split the books, perhaps by packaging them as two volumes in a slip case? I don't think actually selling the books separately would make sense as they are too closely coupled together.

Naturally we think about online access. Reference material often works better on the web, so perhaps we could make the reference book web only (maybe with a CD or something in the physical book) and sell only the narrative book. What would that structure do to sales?

There's interesting questions here, but the final message is that I'd urge authors of big books to think more about how the book is structured in a way that helps someone who wants something within that 200 page limit.

It's one thing to idle away your productive hours reading this blog, but some people like to translate it too. So I'd like to welcome a Thai translation which is being done by a team of people led by Wee Witthawaskul. About fifteen years ago I visited Thailand as a typical backpacker western tourist. I have fond memories of the river buses in Bangkok, hiking near Pai, snorkelling at Ko Pi Pi, and some great food. I first met Wee while he was working with Ralph Johnson, he's now joined me at ThoughtWorks.

You can find permanent references to various translations on the right of the page. I'd like to thank the translators: Ma Haoming for Chinese, Jesús Pérez Sánchez for Spanish, younghoe for Korean, and Kado Masanori for Japanese

The recent fracas over death threats to Kathy Sierra has been bouncing around the blogs I read. The fact that I'm writing this shows it's triggered some thoughts of my own.

My first thought is sympathy for Kathy (and also Bert). I can't say I know her terribly well, but I've met her a couple of times and really enjoyed her ideas and company. Her blog is one of my favorites. I really hope she rides through this and is able to continue her work. Her point of view is too important to lose.

The valuable thing that's come out of this fracas is that it's triggered a serious discussion of nastiness on the net. Many people have run into the kind of stuff that Kathy's faced, but by speaking out she's managed to gain some attention to the issue.

As usual we've seen varied reactions to this. Many people have offered support to Kathy, considering the words and images that frightened her to be unacceptable. Tim O'Reilly has put together some valuable thoughts on what should be in a blogger's code of conduct. Other people have criticized Kathy for speaking out, saying that she's making a fuss out of something that happens all of the time on the net. It's a point of view perhaps best summed up by a commenter on Tim's piece: "not everyone has such a thin skin. Some do revel in the blood, gore, cut and thrust of no-holds-barred verbal combat. Those who can't take the heat should get out of the blogoshphere [sic]."

The question we need to ask is what kind of net do we want to see? Do we want an internet where all sorts of nastiness is accepted and encouraged, where people with a thin skin don't contribute to the conversation because they don't want to be targeted?

There's choice to be made here and I admit to being one of these thin-skinned people who thinks there's too much nastiness on the net. (I talking here about discussions on software development, I don't read political blogs and am told they are much worse.) I worry that people who have interesting things to say and questions to ask are put off by the cut and thrust. They don't feel free to speak. The freedom enjoyed by people who are nasty does deny freedom to others - and the nasty people belittle the fears of those they have silenced.

Net nastiness may also be harming our Diversity. A FooCamp session a couple of years ago pondered why there are so few women active in open source. (We were told that roughly 12% of the software workforce is female, but only 2% of those in open source.) One reason was the nastiness of the discussions, which also was seen as a reason for putting off people from developing countries. Open source may be the canary for the software development community as a whole.

It's one thing to feel the net is too nasty, it's another to question what can be done about it. And just in case the problem isn't hard enough to consider I should point out that this isn't just about blogs. It's also about mailing lists, newsgroups, forums, IRC channels, anywhere where people hang out and exchange their views. (And yes I'm implying parallels in the "real world" too. This issue is hardly new. Just look at some pamphlets from the 17th and 18th centuries.)

It seems when many people hear things like "the net is too nasty" they immediately go the attack saying that we must have freedom of speech; they warn of the leather boots of government. So let me be clear: I'm not in favor of any government regulations here (and indeed I don't see such a call in Tim's piece). I'm talking about individual responsibility for what we write and the communities we participate in. A code of conduct can be useful as a statement of how a community wishes to behave. I'm very wary of imposing such codes, but we should remember that there are legal consequences to things we say: death threats are criminal in many places, libel leads to lawsuits.

First off we are responsible for what we say and it thus follows that we should think about the consequences of what we say. By being nasty to people we hurt them. Are we happy about that? By our nastiness we discourage people from taking part in the communities we frequent. Do we want to silence thin-skinned people?

A hard part of this is that it's often hard to understand why something you say should be upsetting to others. This is a trap I've fallen in plenty of times. I say something that I think is harmless, and someone ends up really upset. I'm sure it's often happened without me even knowing about it. I have a strong personality and can easily be overbearing without realizing it.

You may be thinking that this is taking it too far, some people will take offense at anything; following this logic leads to people who either say nothing, or speak in the bland platitudes favored by PR companies. It's true that that is the result if we take it too far, but I'm not saying that we should do that. I'm saying that we should try to be aware of who is listening and how they are likely to react. There are plenty of times when I'm happy to for people to be offended. The point is that I try to make it a conscious and thoughtful decision. One of the reasons why the law has to tread lightly here is because people have very different views of what is nasty. Nastiness is a subjective judgment and one of our responsibilities is to decide where we, as individuals and as communities, draw the line.

This responsibility in what we say gets amplified as we take a prominent role in a community. As we gain in prominence we become an example that others will follow. We play a role in setting a tone for those communities.

I said above that we have responsibility for the communities we take part in. Am I seriously implying that we are responsible for comments other people make on a mailing list we frequent? Actually I am, at least a little.

One of the key points, for me, in Kathy's original post was "I do not want to be part of a culture--the Blogosphere--where this is considered acceptable." Communities make choices about what is acceptable within them. It's usually not a very formal process, but it happens all the same. If a mailing list routinely shrugs its shoulders at nasty posts, that means the people on that list are accepting the nastiness.

Some people believe that even if you think a community is overly nasty, there's nothing you can do about it. I don't think this is true. There are things to be done, often they aren't easy, but again we have a choice.

If we see a nasty post, we can reply to the author and say we didn't like it. Explain why it upset us and why it upsets others. We can use or post a reply on the group itself. If we make our voice heard then maybe others will agree.

Again this is doubly important as our prominence rises on the group. If people look up to us in our community we can have a bigger effect by making our views clear on what we think is acceptable within that group.

As many public newsgroups have found out, some people can be too persistent in their nastiness. Certainly if people don't heed a call to be reasonable then we have to turn to things like moderation. There are pros and cons to this kind of mechanism, but the point is that we don't have to let nastiness be accepted.

And if the tone of the group remains nasty what then? Leave. The net isn't short of places to hang out. If we want to reduce nastiness we shouldn't post on groups where nastiness is accepted, we shouldn't read and link to nasty blogs. By participating in a group we are supporting it and the tone it has. Again we have a choice.

This kind of advice applies to blog owners too. If bloggers allow comments (I don't out of laziness) then they have to decide when nastiness breaks out. I agree with Tim that a web site owner is a publisher who is responsible for the tone of their site. If we invite others in we have a choice as to what is acceptable. Whether we allow anonymity, or personal attacks is our choice - and we will be judged by others on that choice.

Interestingly Kathy is someone who has done something about this. She didn't like the way that forums such as comp.lang.java treated beginners asking questions. So she set up Javaranch, an online forum that's noted for its lack of tolerance of nastiness.

None of this is really that new. Society has wrestled with the limits and consequences of free speech for several hundred years. The web has changed this dynamic far less than the printing press did. Many of those standards still apply. Laws still apply. I don't know the law, but I'd be surprised if web site owners who passively accept death threats or libelous statements don't face legal consequences. I don't think that crying "the net is free" will impress a judge.

Lots of things will be said about code of conducts, laws, and the delights of verbal combat. But what really matters is what we do. The power is in our hands to make our communities the kinds of places we want to be in.

(Updated as Sam Ruby persuaded me to use second-precision on my dates.)

When I started this bliki, on that flight to Bangalore, I decided to use RSS 0.91 for my feed on the grounds that it was simple. All I had to do was look at an example (PragDave's as it happened) and I could easily create the XML provide that capability.) It's served me well, but I do get occasional complaints that the posts aren't dated.

I asked Ade Oshineye, the man who keeps ThoughtBlogs running for his advice. He gave me a page of carefully considered reasons for either staying with RSS 0.91 or choosing a new format (and which one). In the end I followed his rather more passionate conclusion: "For the love of god please use Atom."

Cutting to the chase, I now have atom feeds. I still have the RSS 0.91 feeds, but if I ever have to do any work to maintain them I will summarily drop them. So I suggest cutting over to the atom feeds when you can. I've updated the references on my web pages for them or you can find them in the now badly named RssFeeds page.

What follows are a few experiences and thoughts on the conversion.

Over Christmas I dug out what information I could find on atom. My first thought was to find and use Ruby libraries. Ruby has a pretty sophisticated feed processing library called FeedTools. It claimed it could produce a feed, and I believe it. However all the documentation was about consuming and converting feeds, caching them in a database, and the like. It introduced a bunch of dependencies and it wasn't glaringly obvious how to use it just to create a feed.

So I decided to create the XML file myself. After all this is very easy in Ruby, especially now we have the awesome builder library.

So the next trick was to figure out what an atom file looked like and what the bits meant. I found three things very helpful to me.

• Me being me, I always want a real life example. I reckoned Sam Ruby's feed should be a good exemplar.
• One big reason Ade gave for favoring atom was a solid specification. Like most specifications I skimmed it to answer the questions I needed. In general I prefer to start with an example and gradually tweak it until it works, going to the specification when I have a problem. This is the typical behavior of a moron.
• Possibly the best reason to use Atom is the excellent test framework: feedvalidator. I found this to be extremely helpful.

I have three feeds to work on: my updates feed, my bliki feed, and the feed for refactoring.com. The data for the feeds came from different formats, so this was the common but tedious task of data conversion from one arbitrary format to another. Much of enterprise software is like this, and it's not the fun part.

I started out by creating my own feed and entry objects to act as gateways. This way I could program to objects that made sense to me for the three conversions and keep the XML conversion and any atom weirdness that might appear in one place. Initially I wondered if this would be worthwhile, after all builder is so simple to use. I quickly found it to be worthwhile.

Most of the process was very straightforward. I just looked at how I created the RSS feed and did the same with the atom feed. (Yes I know I should have used gateways for the old RSS feeds. I get to be foolish too.) The tricky bits were really about things that were new to the atom feed.

The first of these were ids. Atom insists that you give each entry an id. This makes it easier for aggregators to spot multiple copies of the same entry that might come from different sources, or just to tell if a new entry is a truly new entry or an updated old entry. For my bliki it was easy to choose an id - the entries correspond exactly to the web bliki entries so I just used the URL of the bliki entry.

For news updates there isn't a particular page. Looking at Sam Ruby's page I saw he used tags. These were new to me, but googling found an explanation. I generated tags with my domain name, a date, and cleaned up text from the title - copying from Sam Ruby again.

def calculate_atom_id
  specific = title.gsub(/\W/,'-')		
  return "tag:#{domain_name},#{date.strftime("%Y-%m-%d")}:#{specific}"
end

The real driving purpose of this was to add dates and this introduced a couple of oddities. First was the RFC 3339 dates, which I had to look up to see how they worked. It didn't look like the Ruby date classes had a method to return an RFC 3339 date, but after some poking around I realized that the Time class has exactly what I need as the method Time.xmlschema.

One thing that wasn't clear in the spec was what the updated date really meant. The spec merely said it was "the most recent instant in time when an entry or feed was modified in a way the publisher considers significant." When I change a bliki entry it's either to fix a typo, or to revise the entry in some way. Typos I don't consider significant. I do expect aggregators to update their copy of the entry, but I don't expect them to highlight it as new or changed. The latter changes I do expect to be highlighted. It would have been helpful had the spec given some suggestions as to how aggregators and readers might interpret the date - after all it's that interpretation that conveys the true meaning of the field. I often find this problem, writers of specifications are reluctant to put in a standard what clients should do, because they don't want to constrain clients. I understand that concern, but I do think it's really helpful to say how they imagine it might be used with some scenarios.

The most awkward aspect of the updated date for me is the precision of the updated date. The atom spec says that "Date values should be as accurate as possible. For example, it would be generally inappropriate for a publishing system to apply the same time-stamp to several entries that were published during the course of a single day." However I've always looked at my updates as something with date precision. The time I upload the entry to the server isn't relevant to me, just which date I did it. My timestamps thus reflect that - they only mention the day (and indeed use Ruby's Date class which is Date precision).

My initial thought was to leave them at Date precision, picking an arbitrary 00:00Z for the time part to satisfy RFC 3339. The atom spec said 'should' rather than 'shall', which is an important distinction in StandardsSpeak and feedvalidator marked two entries with the same time-stamp as a warning rather than an error. Unless I could understand a downstream problem, I didn't see why I should put in the work to handle second-precision updates rather than date-precision.

Sam Ruby provided a compelling scenario. Some people, including him, aggregate multiple feeds and read them by starting at the latest and reading back till they read something they read earlier. My entries would usually be inserted earlier in the time log than they were supposed to and wouldn't get read. (I could give them an arbitrary late time part which would keep them at the top of the list, but that would just irritate the reader.)

So I decided to use second precision. I needed to replace the Date objects I was using to handle timestamps with Ruby's Time objects. I also now need to start putting full times in posts, which from this entry onwards I will do.

I downloaded a copy of feedvalidator to test my feeds out as I gradually filled them out. It was easy although I grimaced at actually having to install raw as opposed to just using apt-get - I guess I'm getting soft.

As a final aside story. A year or two ago a Very Large Software Company (one that makes software I'm sure you're familiar with) asked me if I minded having my feed aggregated into an architecture feed they were producing. My response, as it usually is, was "fine, that's what feeds are for". A month or two later I got an email saying they couldn't use my feed and I needed to change it to RSS 2.0. It was more effort than I fancied, so I declined. But I couldn't help chuckle that this big organization, which had clearly set up a full blown project to do this work, couldn't do what Ade does for us in his spare time.