Writing helpful API documentation

Many APIs are surprisingly poorly documented considering they are supposed to be “documented programmatic interfaces”. Developers prefer writing code over documentation, rarely showing the same enthusiasm and thoughtfulness for the latter. Some developers claim they write self-documenting code. Others like to point out that “nobody reads the documentation”. Such excuses create a vicious circle: we dislike documenting because we are not skilled enough and our skills do not improve because we pass up on opportunities to practice them. We need to make a conscientious effort to break this trend.

How developers use documentation

We already proved that self-documenting APIs are an unreachable ideal. Let’s refute the “Nobody reads the documentation” claim as well. In Two studies of opportunistic programming, Joel Brandt and his colleagues report that, on average, participants in their studies spend 19% of their time foraging the Internet for information. Web access logs to Adobe Flex documentation show 24,293 programmers making 101,289 queries during the month of July 2008 alone. Are these numbers we expect from documentation nobody reads? Then why is the “Nobody reads the documentation” misconception so widespread? Figure 1 compares the percentage of developers skimming the documentation, focusing only on prominent text and headers (“Skim”) to those systematically reading the pages line-by-line (“Line-by-line”). Another axis compares the number of those starting with the provided PDF overview (“PDF overview”) to those preferring to go straight to the reference manual, expecting it to be self-explanatory (“Self-explanatory”).

Figure 1: How developers use documentation

The conclusion is clear: documentation is referenced, not read. If reading the documentation cover-to-cover line-by-line is the only way to find information, developers will not find it, creating the false impression that they don’t read the documentation. Let’s take a real-life example from stackoverflow.com as illustration:

Question

With java.sql.ResultSet is there a way to get a column’s name as a String by using the column’s index? I had a look through the API doc but I can’t find anything.

Answer

See ResultSetMetaData:

ResultSet rs = stmt.executeQuery(“SELECT a, b, c FROM TABLE2”);

ResultSetMetaData rsmd = rs.getMetaData();

String name = rsmd.getColumnName(1);

This developer skimmed the 139 methods of the java.sql.ResultSet class, looking for getColumnName() or something similar, but skipped getMetadata() because it looked irrelevant. He preferred asking for help on the Internet to closely inspecting all 139 methods, which illustrates our main point: developers don’t want documentation; they want assistance with the task at hand.

In Documentation Usability: A Few Things I’ve Learned from Watching Users, Tom Johnson writes:

Invariably when I ask people how they prefer to learn new software, they respond, ‘I like someone to show me,’ or ‘I like to play around in the system and then ask a colleague if I get stuck.’ I’ve yet to hear the response, ‘I like long software manuals with lots of text in small print.’ Usually people that prefer this also like to slam their fingers in car doors and chew on tin foil.

Answering questions like a friend

We should act like a friend assisting the programmer when writing API documentation. Someone working with the Java Messaging Service (JMS) asks this question on StackOverflow.com:“What is the purpose of a JMS session? Why isn’t a connection alone sufficient to exchange JMS messages between senders and receivers?” Sounds like a legitimate question, right? How many developers would ask “Please enumerate the methods of the class javax.jms.Session in alphabetical order” from a friend? None? But often this is the only question answered by the Javadoc after clicking a class name!

If you document like a friend, you provide package and class overviews, answering useful questions like: “What is the purpose of this package?” “What can I use this class for?” “Are there any limitations?” “This is not quite what I need, what are some related classes?” Would you make a friend read dozens of pages of API minutiae just to infer the answers to such simple questions?

How do we know what questions to answer? In “Specifying behavior”, we explained that three-quarters of API questions are about behavior. Consequently, documenting preconditions, postconditions and invariants alone completes three-quarters of API documentation! We can also invite people unfamiliar with the API to review it and record the questions and answers in a FAQ. FAQs are easy to create and extend. API documentation is never quite complete and FAQs capture missing facts, clarify ambiguities, or document known issues. Most FAQs are temporary, kept only until the other parts of the documentation are updated. Long collections of FAQs are less useful.

Although we might think that callers don’t care, concepts and abstractions used in the design of the API, as well as the intent behind choosing them, need to be explained. As one developer eloquently says:

When you’re building a framework, there’s an intent … if you can understand what the intent was, you can often code efficiently, without much friction. If you don’t know what the intent is, you fight the system.

Supporting just-in-time learning

Research shows that developers learn APIs incrementally, interleaving short periods of studying documentation with writing code. Helpful API documentation matches this just-in-time learning pattern, consisting of small, self-contained, heavily cross-referenced sections. Developers spend no more than ten minutes with documentation before returning to code, the studies show. This sets the maximum size of an undivided documentation section at about half a page.

Because developers skim the documentation, each section needs to focus on a single subject and highlight what the subject is. Imagine that you work in customer support. You want to link to a section of the documentation as the answer to a specific customer question. When a section contains primarily irrelevant information, you are wasting the customer’s time. If the first sentence does not read like you are answering the question, you’d be too embarrassed to link to it.

Navigation and search are essential for finding the correct documentation section. Navigation produced by tools like Javadoc has some limitations, evident from the generated alphabetical index. Packages, classes and methods are listed there, but adding non-structural entries like “Performance” or “Thread safety” to the index is not directly supported by the tools. Many developers simply type their questions into an Internet search engine and expect it to find the correct answer. API documentation not available on the Internet or not optimized for this type of searching is less useful.

Illustrating use cases

Answers to “How do I?” questions, the kind of use-case driven questions tens of thousands of developers ask daily on Internet forums tend to be more helpful than answers to “What is this?” questions:

The problem is always, when I feel I can’t make progress … when there’s multiple functions or methods or objects together that, individually they makes sense but sort of the whole picture isn’t always clear, from just the docs.

Code snippets are often the straightforward answer to “How do I?” questions. The “How do I get the JDBC column name?” question above was answered with just 3 lines of code. If we follow the “consider the perspective of the caller” guideline, we write use case driven code samples right at the beginning of the API design process. We can turn these code samples effortlessly into a cookbook, an increasingly popular form of developer documentation, by describing briefly the use case each code snippet illustrates.

The code examples must be exemplary. They should closely follow all relevant programming best practices. A sloppy example “can become more of a hindrance than a benefit when there’s a mismatch between the tacit purpose of the example and the goal of the example user” warns Martin P. Robinard.

Tutorials serve a similar purpose, but differ in important aspects. They break up the building of the complete example into smaller, more manageable steps. Tutorials are intended to be completed from start to finish. They intend to teach. They are not reference material. Many programmers started learning Windows programming from the famous Scribble tutorial. New tools for recording video on a computer (screencasting) have made video tutorials very popular. Nevertheless, tutorials can be time-consuming to produce and are rarely needed for simple APIs.

Putting it all together

Developers need API documentation for four main activities:

1. Remind themselves of details deemed not worth remembering
2. Clarify and extend their existing knowledge
3. Engage in just-in-time learning of new skills
4. Experiment with (sample) code

Reference documentation only supports the first two activities. A separate Programmer’s Guide is needed to support just-in-time learning because research shows that developers waste a great deal of time guessing, inspecting and backtracking when learning directly from reference documentation. Unfortunately, the name Programmer’s Guide still evokes a heavy book, which is not what we are advocating. In addition to a high-level overview, a modern Programmer’s Guide contains only topics that don’t fit nicely into either the Reference Manual or the Cookbook format, such as describing new concepts, conventions, design patterns, and so on. With their just-in-time learning style, programmers make frequent jumps between Programmer’s Guide, Reference Manual and Cookbook, provided these are properly cross-referenced. The table below summarizes the various parts of a complete API documentation.

Conclusion

It is hard to over-emphasize the importance of good API documentation. Even exceptionally well designed APIs can be frustrating to use if poorly documented. On the other hand, the only way to improve an existing API, without a complete and expensive redesign, is to improve the documentation. API documentation is developer documentation. Since nearly all developer documentation we write is for internal use, we often compromise on quality. It is crucial to understand that public APIs need to be treated differently. The target audience includes external developers or system integrators, who are also customers. The same quality requirements apply as to customer documentation like administration guides or end-user manuals.

This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.