Monday, May 17, 2004

Effective Documentation

by Asim Jalis

THE NURNBERG FUNNEL Here is a book I read yesterday: The Nurnberg Funnel by John Carroll (on minimalism). For more details take a look at The Nurnberg Funnel is a means of directly pouring information into a students brain through a funnel stuck into his head. Minimalism is an XP-like movement in technical writing. The main insight is that people don't learn systematically (or in phases), but rather organically. They learn in layers. The tradition that Minimalism reacts to is called the Systems Approach. THE SYSTEMATIC APPROACH In the systems approach the way to write a technical document is to decompose the subject into a "logical" order. To master a subject the reader must plod through the text cover-to-cover. Learning is an all-or-nothing affair. The only way to know something is to know everything. It is much like a waterfall architecture. To get any functionality you must get all the functionality. This is of course a caricature designed to make a point. In practice systematic writing incorporates some minimalistic ideas out of pure pragmatism. But the underlying philosophy of systematic writing is to pour the information into the reader's head. The concepts generally are organized bottom-up. You start with the basic building blocks and build up to more advanced concepts. You cannot understand the advanced concepts until you understand the elementary ones. UNIVERSITY USE SYSTEMATIC APPROACH Incidentally some universities are also organized this way. Courses are organized into a sequence of prerequisites. You can't jump straight into a 400 level course. You must first take a 100 level course, then a 200, and so on. HOW PEOPLE LEARN The minimalist approach is inspired by how people actually learn (according to the minimalists). They don't learn linearly, building one concept on top of another. Rather they learn to solve immediate problems. They are goal driven. For example, given a thick manual on how to use a word processor most people are eager to get started quickly. They stumble through the program, making mistakes and learning. They dig out the manual when they get stuck. The focus of people is on getting started quickly, on creating value. They don't like to invest days and weeks accumulating concepts with no obvious applicability. Minimalist documents get out of the way of users. They help the user get started, and then they help him solve problems as they arise. They can be read in any order. There is no right sequence. While systematic documents are focused on technology, minimalist documents are focused on the user. They address the concerns the user has. For example, to write a letter to mom in Word does not require reading the whole book. A short paragraph is sufficient to get most people started. Then later as they want to do more advanced things they can poke around to figure out how to do them. Systematic documents attempt to do Big Learning Up Front, instead of doing the learning incrementally, and evolutionarily, driven by user needs. There is a definite parallel here between Minimalism and XP. This kind of duality arises in a lot of different fields (possibly in all fields). I have now seen in it business (traditional Planned Manufacturing versus Just-In-Time and Lean Manufacturing), and now also in technical writing. A common example of minimalism are FAQ documents. These are completely user and problem focused, and they don't need to be read in a particular order. They are valuable because they give people what they want. They get out of the way of learning. Here are some virtues of FAQ's: They don't try to feed people concepts or theories. They solve their immediate problems. Each item has an immediate and well-defined goal. It does not get distracted. A common example of systematic documents are programming language texts. For example most Java and C++ books have a systematic organization. First you learn about integers, then about control structures, then about data structures and so on. Compare the difference between the generic systematic language text and K&R. K&R starts out with an extremely minimalist flavor. Their use of the hello world example is the essense of minimalism. Get the user started. Writing hello world is immediately gratifying and hooks the reader. But then the rest of the book falls back into the systematic trap. Larry Wall's books are also engaging for precisely the same reason. He layers the knowledge. He even talks about this layering (so it is deliberate). First he teaches you baby talk Perl. So like a baby you can say most of the things you want to say. As you grow older you acquire more layers of complexity that build up on the baby talk foundation. Larry's extremely readable writing might be a big reason for Perl's success. Larry makes you laugh. It's fun to read his book. He also talks about knowledge as an onion, with layers upon layers. Each layer is coherent. At each level you are able to produce real programs and to create value. You break the journey to becoming a wizard into tiny iterations. Another characteristic of minimalist documents is that they give you immediate gratification. The long-term goal is that you will learn language X. Systematic documents assume this is sufficient to stay inspired through tomes of insipid writing. Minimalist texts by gratifying and rewarding the reader immediately (by showing how to do cool things immediately) are much more engaging. They split the big goal into a lot of tiny meaningful goals. Systematic texts split up the big goal into tiny meaningless goals. HOW DO YOU CREATE MINIMALIST DOCUMENTS The minimalist folks have hit real insight, but I think they still haven't quite explained how to do it. They themselves say that writing a minimalist book is much harder, and much less mechanical, and requires much more creativity, than writing a systematic book. For example, how could you teach someone Java while achieving real goals the whole time. In reality most books fall somewhere in between. They opportunistically use minimalism wherever they can and then fall back on systematic writing when they can't. MATH EDUCATION Incidentally, math education faces similar issues. Systematic texts take the reader through the landscape of math, force feeding him the concepts, without trying to engage him, or to address any of his real problems. They teach concepts for the sake of concepts. And they are written in the mind-numbing sequential order. To understand chapter 3 you have to read chapter 2 and 1. There is no short-term gratification. So here's an idea: Take over the math textbook market by writing a minimalist math textbook. Perhaps a math FAQ. Something that solves real problems people have, that can be read in any order. USE INTERESTING EXAMPLES One way to make programming books more engaging might be to use more interesting examples. I am currently reading up on .NET Remoting. This is Microsoft's library for distributed components. A client news up an object and then calls methods on it. Meanwhile the object really lives on the server. The Remoting glue creates the (leaky) illusion that everything is happening locally. Most texts on this are mind-numbingly boring. They use the example of getting the server's time, or getting the value of a counter on the server. Boring. It occurred to me that using remoting you can write a simple napster application in about a page of code. Whoa! Now that's interesting. I wouldn't mind reading a page of code that showed me how to make my own napster. Other cool things you can do with remoting: Write a text based chat client. Write a program that lets you play Chess, or Go, or Tic-Tac-Toe on the internet. Write a program that lets you create a denial of service attack. Find the value of Pi on a network of computers. Solve NP complete problems by throwing hardware at them. The weird thing is that these are not even that hard to do. It is possible to do a spike program for each one of these areas that is about a page to two pages long. Of course, some of these might be legally risky (like napster and the denial of service program -- also worms and viruses could be easily written using remoting, but that too might not go well with corporate legal departments). Nevertheless, there are many other harmless applications too that are still interesting, like the Chess program. Here's another one: a pair programming editor based on notepad. MINIMALISM IS HARDER TO ACHIEVE Getting back to the topic of minimalism, it is more than just interesting examples. Minimalism taxes the mind in the same way as XP does. It forces you to look for small steps. In a BDUF design you do not need to worry about keeping the system functioning throughout its evolution. In XP and also in minimalism you have to keep the system coherent as it evolves. It puts more constraints on the writer. I am interested in brainstorming and learning more about this. I think economically this could be extremely profitable. And also I suspect it is something we would be very good at. It changes the game and takes it out of the field of the heads-down nose-to-the-grindstone systematic writers. Suddenly being ADD, or easily distracted is no longer a liability. It magically turns into the most valuable asset. A book that can maintain an ADD's interest could blow the competition out of the water. Carroll reported results in which people were able to accomplish twice as many tasks in the same amount of time when they were trained using minimalist docs compared to when they used systematic docs. ECONOMIC IMPLICATIONS Incidentally, this is not academic hoo-ha. It has some powerful economic implications. A software program with good usable documentation generates fewer support costs. It becomes popular and can dominate its market. Docs are big business. Consider Tim O'Reilly, who is one of the open source millionaires and is funding Perl and many other open source technologies. He might be considered the Bill Gates of open source software. And he built his empire, not from creating software, but from writing docs for it. Here is what I am interested in exploring: What are some concrete techniques for creating minimalist documents. Carroll proposes task cards. The cards force the instructions to be brief. Each card explains how to accomplish a particular task. They are modelled after little cheat sheets people sometimes create for themselves. But there must be ways of creating more traditional books and learning tutorials using minimalist approaches. HOW TO READ A MATH BOOK This also connects with my experience in math, where I learned much more by solving problems than by reading texts. The problems engaged me, and made math come alive for me. I was able to get started immediately. In fact the best way to read a math book is to first do the problems, then read the text just-in-time as you get stuck on the problems. READ THE MANUAL FIRST Another interesting point Carroll makes is that most systematic docs always contain the warning: do not operate the program until you have read the manual. They place the warning in many places to ensure compliance. The warning is futile. Most people operate the system anyway. MINIMALIST WRITING Incidentally, I tried to make this note somewhat minimalistic by giving headings to all the topics, so you can skip the ones that don't interest you and zoom in on the ones that do. You can read the items in any order. It is ridiculous to assume that I can achieve a nurnberg funnel and do a binary copy from my brain to yours. I can't. Our conversations are much more enjoyable because they have this non-linear, goal-driven, un-systematic quality.