How to use the Java API design checklist

This before-and-after example illustrates how to use the Java API design checklist. We borrowed the original API from the free online eBook “OSGi in Practice” by Neil Bartlett. The author uses the API through the book to illustrate various OSGi features and considers it a “reasonable attempt” at defining an API.

Here is the description of the API strait from the book:

“Thanks to the internet, we are today bombarded by messages of many kinds. Email is an obvious example, but also there are the blogs that we subscribe to through RSS or ATOM feeds; SMS text messages; “microblogging” sites such as Twitter[?] or Jaiku[?]; IM systems such as AOL, MSN or IRC; and perhaps for professionals in certain fields, Reuters or Bloomberg newswire feeds and market updates. Sadly we still need to flip between several different applications to view and respond to these messages. Also there is no coherent way to apply rules and automated processing to all of our inbound messages. For example, I would like a way to apply my spam filters, which do a reasonable job for my email, to my SMS text messages, as spam is increasingly a problem in that medium.”

“To support multiple kinds of message sources, we need to build an abstraction over messages and mailboxes, so a good place to start is to think about what those abstractions should look like. In Java we would represent them as interfaces. Listing 1 contains a reasonable attempt to define a message in the most general way.”

“Objects implementing this interface are really just message headers. The body of the message could be of any type: text, image, video, etc. We need the header object to tell us what type the body data is, and how to access it.”

“Next we need a way to retrieve messages. The interface for a mailbox could look like Listing 2. We need a unique identifier to refer to each message, so we assume that an ID of type long can be generated or assigned by the mailbox implementation. We also assume that the mailbox maintains some temporal ordering of messages, and is capable of telling us about all the new messages available given the ID of the most recent message known about by the reader tool. In this way the tool can notify us only of new messages, rather than ones that we have already read.”

“Many back-end message sources, such as the IMAP protocol for retrieving email, support storing the read/unread state of a message on the server, allowing that state to be synchronized across multiple clients. So, our reader tool needs to notify the mailbox when a message has been read. In other protocols where the back-end message source does not support the read/unread status, this notification can be simply ignored.”

“Finally we need the code for the exceptions that might be thrown. These are shown in Listing 3 and Listing 4.”

Design review

The API design shown below is fairly typical for a first draft. All major functional requirements are met, but there are some remaining design issues. We will do a design review using the Java API design checklist to remember overlooked design requirements, spot mistakes, identify less-than-optimal design choices and opportunities for improvements. We marked the identified issues with “//see …” comments in the listings below. The hyperlinks point to the relevant checklist items in the list.

Listing 1: The original Message (see after)

1   package org.osgi.book.reader.api;  //see 1.2.7, 1.3.1

3   import java.io.InputStream;

5   public interface Message{ //see 2.1.8, 2.2.18, 2.4.1, 2.7.1

7   /**
8   * @return The unique (within this message’s mailbox) message ID. //see 3.9.3
9   */
10  public long getId();

12  /**
13  * @return A human-readable text summary of the message. In some
14  * messaging systems this would map to the "subject" field. //see 3.9.3
15  */
16  public String getSummary();

18  /**
19  * @return The Internet MIME type of the message content. //see 3.9.3
20  */
21  public String getMIMEType(); //see 2.2.3

23  /**
24  * Access the content of the message.
25  *
26  * @throws MessageReaderException
27  */
28  public InputStream getContent() throws MessageReaderException; //see 2.1.3, 3.2.3, 3.4.2

30  }

Listing 2: The original Mailbox (see after)

1   package org.osgi.book.reader.api;

3   public interface Mailbox{ //see 2.1.8, 2.4.1, 2.7.1

5   public static final String NAME_PROPERTY = "mailboxName";

7   /**
8   * Retrieve all messages available in the mailbox.
9   *
10  * @return An array of message IDs.
11  * @throws MailboxException
12  */
13  public long[] getAllMessages() throws MailboxException; //see 3.1.3, 3.2.3, 3.3.8, 3.4.2

15  /**
16  * Retrieve all messages received after the specified message.
17  *
18  * @param id The message ID.
19  * @return An array of message IDs.
20  * @throws MailboxException
21  */
22  public long[] getMessagesSince(long id) throws MailboxException; //see 3.1.3, 3.2.3, 3.3.1, 3.3.8, 3.4.2

24  /**
25  * Mark the specified messages as read/unread on the back-end
26  * messagesource, where supported,e.g.IMAP supports this
27  * feature.
28  *
29  * @param read Whether the specified messages have been read.
30  * @param ids An array of messageIDs.
31  * @throwsMailboxException
32  */
33  public void markRead(boolean read, long[] ids) throws MailboxException; //see 3.1.14, 3.3.25

35  /**
36  * Retrieve the specified messages.
37  *
38  * @param ids The IDs of the messages to be retrieved.
39  * @return Anarray of Messages.
40  * @throws MailboxException
41  */
42  public Message[] getMessages(long[] ids) throws MailboxException; //see 3.1.3, 3.2.3, 3.3.1, 3.3.8, 3.4.2

44  }

Listing 3: The original MessageReaderException (see after)

1   package org.osgi.book.reader.api;

3   public class MessageReaderException extends Exception{ //see 2.1.3, 2.6.2

5   private static final long serialVersionUID = 1L; //see 2.3.2

7   public MessageReaderException(String message) {
8      super(message);
9   }

11  public MessageReaderException(Throwable cause){
12     super(cause);
13  }

15  public MessageReaderException(String message,Throwable cause){
16     super(message,cause);
17  }

19  }

Listing 4: The original MailboxException (see after)

1   package org.osgi.book.reader.api;

3   public class MailboxException extends Exception{ //see 3.4.2, 2.7.1

5   private static final long serialVersionUID = 1L; //see 2.3.2

7   public MailboxException(String message) {
8      super(message);
9   }

11  public MailboxException(Throwable cause){
12     super(cause);
13  }

15  public MailboxException(String message,Throwable cause){
16     super(message,cause);
17  }

19  }

Redesign

Our design review highlighted several omissions, issues, and inconsistencies. Listings 5 6, 7, 8 and 9 show the redesigned API. During the redesign we had many tradeoffs to consider. In addition to the checklist items identified during the design review we considered several others. We show these additional checklist items within square brackets inside comments and using the //also … comments in code. The hyperlinks point to the relevant checklist items in the list.

Listing 5: package overview (package-info.java) for the redesigned API

1   /**
2   * <p>
3   * Provides simple and generic read-only access to messages from a variety
4   * of different sources like email, RSS and Atompub feeds, instant messaging
5   * services, Facebook, SMS and Twitter.</p> [1.3.3]
6   * <p>
7   * All concrete classes implement either {@link org.osgi.book.reader.MessageHeader}
8   * or {@link org.osgi.book.reader.Mailbox}. Together, these two abstract classes define the
9   * generic interface used in the package. All concrete classes are protocol-specific
10  * implementations and extensions like {@link org.osgi.book.reader.EmailHeader} or
11  * {@link org.osgi.book.reader.ImapMailbox}</p> [1.3.5]
12  * <p>
13  * Classes from this package are not intended for direct instantiation.
14  * They have no public constructors. Instead, pre-configured instances of mailboxes
15  * must be retrieved by their name through a JNDI lookup from the naming context
16  * "com/env/mailboxes"</p>
17  * <p>
18  * Classes from this package are not intended for extension. All concrete
19  * classes are final and abstract classes have no public or protected constructors.</p>
20  * <p>
21  * The code sample bellow shows how to read and print out messages
22  * from all configured mailboxes: </p> [1.3.6]
23  *
24  * <pre>
25  *     import org.osgi.book.reader.*;
26  *     import javax.naming.*;
27  *     import java.rmi.RemoteException;
28  *
29  *     try {
30  *         Context initialContext = new InitialContext();
31  *         NamingEnumeration mailboxNames = initialContext.list("com/env/mailboxes");
32  *         while(mailboxNames.hasMore())
33  *         {
34  *             NameClassPair pair = (NameClassPair) mailboxNames.next();
35  *             Mailbox<?> mailbox = (Mailbox<?>) initialContext.lookup(pair.getName());
36  *             for(MessageHeader h : mailbox.readAllMessageHeaders()) {
37  *                  System.out.println(h);
38  *             }
39  *         }
40  *     } catch (NamingException e) {
41  *         e.printStackTrace();
42  *     } catch (RemoteException e) {
43  *         e.printStackTrace();
44  *     }
45  * </pre>
46  *
47  * @version 2.0 [1.3.10]
48  *
49  * <br/>
50  * <p>This sample API is an adaptation of the original published in
51  * <a href="http://njbartlett.name/osgibook.html">"OSGi in Practice"</a> by Neil Bartlett
52  *
53  * <br/>
54  * <a rel="license" href="http://creativecommons.org/licenses/by-sa/3.0/">
55  * <img alt="Creative Commons License" style="border-width:0"
56  * src="http://i.creativecommons.org/l/by-sa/3.0/88x31.png" /></a>
57  *
58  * <br/>
59  * Sample API by Neil Bartlett and Ferenc Mihaly is licensed under a
60  * <a rel="license" href="http://creativecommons.org/licenses/by-sa/3.0/">
61  * Creative Commons Attribution-ShareAlike 3.0 Unported License</a>. [1.3.12]
62  */
63
64  package org.osgi.book.reader;

Listing 6: Message(Header) after redesign (see before)

1   package org.osgi.book.reader;  

3   import java.io.InputStream;
3a  import java.io.Serializable;
3b  import java.rmi.RemoteException;

4a  /**
4b  * Contains descriptive (structured) information about a message and
4c  * defines methods for retrieving the (unstructured) message body. [2.7.3]
4d  * Read-only (immutable) instances of this abstract type are
4e  * read from a Mailbox implementation. [2.7.5]
4f  * Concrete derived types may offer additional information or functionality, like {@link EmailHeader}. [2.3.4]
4g  * The natural ordering is the order of reception.
4h  * For a code sample, see {@link Mailbox}.. [2.7.6, 2.7.9]
4i  */
5   public abstract class MessageHeader implements Comparable, Serializable { //also 2.2.4, 2.2.8, 2.2.10, 2.3.11, 2.3.12, 2.3.17

6a  MessageHeader() {} //also 2.3.9, 2.3.21

6b  @Override
6c  public boolean equals(Object obj) {...}
6d  @Override
6e  public int hashCode() {...}
6f  @Override
6g  public String toString() {...}
6h  @Override
6i  public int compareTo(Object o) {...} //also 2.3.10, 2.3.11

7   /**
7a  * Returns the unique (within this message’s mailbox) message ID.
8   * @return The message ID.
9   */
10  public long getId() {...}

12  /**
12a * Returns a human-readable text summary of the message.
13  * @return A human-readable text summary of the message. In some
14  * messaging systems this would map to the "subject" field. Not null.
15  */
16  public String getSummary() {...}

18  /**
18a * Returns the Internet MIME type of the message content.
19  * @return The Internet MIME type of the message content. Not null.
20  */
21  public String getMimeType() {...}

23  /**
24  * Access the content of the message from the remote mailbox. [item 3.9.14]
25  *
25a * @return An input stream for reading the message content. Not null.
26  * @throws RemoteException In case of a communication error with a remote mailbox
26  * @throws MailboxException Unrecoverable internal mailbox error
27  */
28  public abstract InputStream streamContent() throws RemoteException, MailboxException;

30  }

Listing 7: Mailbox after redesign (see before)

1   package org.osgi.book.reader;
1a
1b  import java.rmi.RemoteException;
1c  import java.util.Collection;
1c  import java.util.SortedSet;

2a  /**
2b  * Represents a generic and abstract mailbox interface for accessing incoming messages. [2.7.3]
2c  * Several implementations based on various messaging protocols (POP3, IMAP, RSS, etc.) are available.
2d  * Configured instances of mailboxes are retrieved by performing a JNDI lookup. [2.7.5]
2e  * The code sample below prints out the summary of all messages from the default mailbox. [2.7.6]
2f  * <pre>
2g  *     Context context = new InitialContext();
2h  *     Mailbox<?> mb = (Mailbox<?>) context.lookup("com/env/mailbox/default");
2i  *     for(MessageHeader h : mb.readAllMessageHeaders()) {
2j  *         System.out.println(h); //uses toString()
2k  *     }
2l  * </pre>
2m  */
3   public abstract class Mailbox<T extends MessageHeader> implements Comparable {//also 2.1.10, 2.3.11

5   public static final String NAME_PROPERTY = "mailboxName";

6a  Mailbox() {...} //also 2.3.9, 2.3.21

6b  @Override
6c  public boolean equals(Object obj) {...}
6d  @Override
6e  public int hashCode() {...}
6f  @Override
6g  public String toString() {...}
6h  @Override
6i  public int compareTo(Object o) {...} //also 2.3.10, 2.3.11

7   /**
8   * Retrieve all messages available in the mailbox.
9   *
10  * @return The ordered set of all available message headers. Not null.
10a * @throws RemoteException In case of a communication error with a remote mailbox
11  * @throws MailboxException Unrecoverable internal mailbox error
12  */
13  public abstract SortedSet<T> readAllMessageHeaders()
13a                              throws RemoteException, MailboxException; //also 3.1.3, 3.1.9, 3.3.9

15  /**
16  * Retrieve all messages received after the specified message.
17  *
18  * @param last The last read message header; not null.
19  * @return The ordered set of message headers since the lst read message header. Not null.
19a * @throws NullPointerException If parameter is null
19b * @throws RemoteException In case of a communication error with a remote mailbox
20  * @throws MailboxException Unrecoverable internal mailbox error
21  */
22  public abstract SortedSet<T> readMessageHeadersSince(T last)
22a     throws NullPointerException, RemoteException, MailboxException; //also 3.1.3, 3.1.9, 3.3.9, 3.4.6, 3.4.12

24  /**
25  * Mark the specified messages as read on the back-end
26  * messagesource, where supported,e.g.IMAP supports this
27  * feature.
28  *
29  * @param headers The list of message headers to be marked; not null.
29a * @throws NullPointerException If parameter is null
29b * @throws RemoteException In case of a communication error with a remote mailbox
31  * @throws MailboxException Unrecoverable internal mailbox error
32  */
33  public abstract void markRead(Collection<T> headers)
33a     throws NullPointerException, RemoteException, MailboxException; //also 3.1.3, 3.1.9, 3.1.10, 3.3.9, 3.4.6, 3.4.12
33b public abstract void markRead(T header)
33c     throws NullPointerException, RemoteException, MailboxException; //also 3.1.3

34a /**
34b * Mark the specified messages as unread on the back-end
34c * messagesource, where supported,e.g.IMAP supports this
34d * feature.
34e *
34f * @param headers The list of message headers to be marked
29g * @throws NullPointerException If parameter is null
34h * @throws RemoteException In case of a communication error with a remote mailbox
34i * @throws MailboxException Unrecoverable internal mailbox error
34j */
34k public abstract void markUnread(Collection<T> headers)
34l     throws NullPointerException, RemoteException, MailboxException; //also 3.1.3, 3.1.9, 3.3.9, 3.1.10, 3.4.6, 3.4.12
34m public abstract void markUnread(T header)
34n     throws NullPointerException, RemoteException, MailboxException; //also 3.1.3

Listing 8: MessageReaderException after redesign (see before)

(no longer needed)

1   package org.osgi.book.reader;

3   public class MessageReaderException extends Exception{

5   private static final long serialVersionUID = 1L;

7   public MessageReaderException(String message) {
8      super(message);
9   }

11  public MessageReaderException(Throwable cause){
12     super(cause);
13  }

15  public MessageReaderException(String message,Throwable cause){
16     super(message,cause);
17  }

19  }

Listing 9: MailboxException after redesign (see before)

1   package org.osgi.book.reader;

2a  /**
2b  * Signals an unrecoverable internal mailbox error.
2c  */
3   public class MailboxException extends RuntimeException{

5    

7   public MailboxException(String message) {
8      super(message);
9   }

11  public MailboxException(Throwable cause){
12    super(cause);
13  }

15  public MailboxException(String message,Throwable cause){
16     super(message,cause);
17  }

18a /* IMPLEMENTATION STUFF */
18b private static final long serialVersionUID = 1L;
18c
19  }

Discussion

Is the redesigned version better than the original? This is a tricky question, a bit like asking which car is better or which city is more pleasant to live in. Your answer will depend on what aspects you consider important. There is no doubt that we managed to improve many aspects of the API. The new version is noticeably more consistent, safer to use, and better documented. Listing 10 shows the code we wrote to aggregate massages from several mailboxes before and after the redesign. Regardless of your stance on the arrays versus collections debate (some developers dislike generic Java collections, finding them too verbose), you’ll probably agree that the second version is safer to use.

Listing 10: Code for aggregating messages before and after the redesign

1  /**
2  * Before
3  */
4  public static Message[]
5  getAllMessages(Mailbox[] mailboxes) throws MailboxException {
6  	Message[] result = new Message[0];
7  	for (Mailbox mailbox : mailboxes) {
8  		Message[] messages = mailbox.getMessages(mailbox.getAllMessages());
9  		result = Arrays.copyOf(result, result.length + messages.length);
10 		System.arraycopy(messages, 0, result, result.length, messages.length);
11 	}
12 	return result;
13 }

1  /**
2  * After
3  */
4  public static SortedSet<MessageHeader>
5  readAllMessageHeaders(Set<Mailbox<?>> mailboxes) throws RemoteException {
6  	SortedSet<MessageHeader> result = new TreeSet<MessageHeader>();
7  	for(Mailbox<?> mailbox : mailboxes) {
8  		result.addAll(mailbox.readAllMessageHeaders());
9  	}
10 	return result;
11 }

This being said, some of our design choices are not the best. For example, we choose JNDI lookup over public constructors to illustrate the importance of proper documentation, not because we don’t recommend constructors. We used generic abstract classes to illustrate how to improve API safety with strong compile time type checks, intentionally disregarding that some developers may be uncomfortable with Java generics.

This brings us to an important point: the API design checklist is just one of the tools we use for API design and checklist items should not be applied mechanically, without thinking. This is especially true for checklist items introduced with the words Favor, Consider, and Avoid. Considering the perspective of the caller and focusing on improving the main use cases should help decide what design trade-offs to make.

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.

Java API Design Checklist

There are many different rules and tradeoffs to consider during Java API design. Like any complex task, it tests the limits of our attention and memory. Similar to the pilots’ pre-flight checklist, this list helps software designers remember obvious and not so obvious rules while designing Java APIs. It is a complement to and intended to be used together with the API Design Guidelines.

We also have some before-and-after code examples to show how this list can help you remember overlooked design requirements, spot mistakes, identify less-than-optimal design choices and opportunities for improvements.

Click the [explain] link next to a checklist item (where available) for details about the rationale, examples, design tradeoffs or other limitations of applicability.

This list uses the following conventions:

   (Do) verb...  - Indicates the required design
    Favor...     - Indicates the best of several design alternatives
    Consider...  - Indicates a possible design improvement
    Avoid...     - Indicates a design weakness
    Do not...    - Indicates a design mistake

1. Package Design Checklist

1.1. General

  • 1.1.1. Favor placing API and implementation into separate packages [explain]
  • 1.1.2. Favor placing APIs into high-level packages and implementation into lower-level packages [explain]
  • 1.1.3. Consider breaking up large APIs into several packages [explain]
  • 1.1.4. Consider putting API and implementation packages into separate Java archives [explain]
  • 1.1.5. Avoid (minimize) internal dependencies on implementation classes in APIs [explain]
  • 1.1.6. Avoid unnecessary API fragmentation [explain]
  • 1.1.7. Do not place public implementation classes in the API package [explain]
  • 1.1.8. Do not create dependencies between callers and implementation classes [explain]
  • 1.1.9. Do not place unrelated APIs into the same package [explain]
  • 1.1.10. Do not place API and SPI into the same package [explain]
  • 1.1.11. Do not move or rename the package of an already released public API [explain]

1.2. Naming

  • 1.2.1. Start package names with the company’s official root namespace [explain]
  • 1.2.2. Use a stable product or product family name at the second level of the package name [explain]
  • 1.2.3. Use the name of the API as the final part of the package name [explain]
  • 1.2.4. Consider marking implementation-only packages by including “internal” in the package name [explain]
  • 1.2.5. Avoid composite names [explain]
  • 1.2.6. Avoid using the same name for both package and class inside the package [explain]
  • 1.2.7. Avoid using “api” in package names [explain]
  • 1.2.8. Do not use marketing, project, organizational unit or geographic location names [explain]
  • 1.2.9. Do not use uppercase characters in package names [explain]

1.3. Documentation

  • 1.3.1. Provide a package overview (package.html) for each package [explain]
  • 1.3.2. Follow standard Javadoc conventions [explain]
  • 1.3.3. Begin with a short, one sentence summary of the API [explain]
  • 1.3.4. Provide enough details to help deciding if and how to use the API [explain]
  • 1.3.5. Indicate the entry points (main classes or methods) of the API [explain]
  • 1.3.6. Include sample code for the main, most fundamental use case [explain]
  • 1.3.7. Include a link to the Developer Guide [explain]
  • 1.3.8. Include a link to the Cookbook [explain]
  • 1.3.9. Indicate related APIs
  • 1.3.10. Include the API version number [explain]
  • 1.3.11. Indicate deprecated API versions with the @deprecated tag
  • 1.3.12. Consider including a copyright notice [explain]
  • 1.3.13. Avoid lengthy package overviews
  • 1.3.14. Do not include implementation packages into published Javadoc

2. Type Design Checklist

2.1. General

  • 2.1.1. Ensure each type has a single, well-defined purpose
  • 2.1.2. Ensure types represent domain concepts, not design abstractions
  • 2.1.3. Limit the number of types [explain]
  • 2.1.4. Limit the size of types
  • 2.1.5. Follow consistent design patterns when designing related types
  • 2.1.6. Favor multiple (private) implementations over multiple public types
  • 2.1.7. Favor interfaces over class inheritance for expressing simple commonality in behavior [explain]
  • 2.1.8. Favor abstract classes over interfaces for decoupling API from implementation [explain]
  • 2.1.9. Favor enumeration types over constants
  • 2.1.10. Consider generic types [explain]
  • 2.1.11. Consider placing constraints on the generic type parameter
  • 2.1.12. Consider using interfaces to achieve similar effect to multiple inheritance
  • 2.1.13. Avoid designing for client extension
  • 2.1.14. Avoid deep inheritance hierarchies
  • 2.1.15. Do not use public nested types
  • 2.1.16. Do not declare public or protected fields
  • 2.1.17. Do not expose implementation inheritance to the client

2.2. Naming

  • 2.2.1. Use a noun or a noun phrase
  • 2.2.2. Use PascalCasing
  • 2.2.3. Capitalize only the first letter of acronyms [explain]
  • 2.2.4. Use accurate names for purpose of the type [explain]
  • 2.2.5. Reserve the shortest, most memorable name for the most frequently used type
  • 2.2.6. End the name of all exceptions with the word “Exception” [explain]
  • 2.2.7. Use singular nouns (Color, not Colors) for naming enumerated types [explain]
  • 2.2.8. Consider longer names [explain]
  • 2.2.9. Consider ending the name of derived class with the name of the base class
  • 2.2.10. Consider starting the name of an abstract class with the word “Abstract” [explain]
  • 2.2.11. Avoid abbreviations
  • 2.2.12. Avoid generic nouns
  • 2.2.13. Avoid synonyms
  • 2.2.14. Avoid type names used in related APIs
  • 2.2.15. Do not use names which differ in case alone
  • 2.2.16. Do not use prefixes
  • 2.2.17. Do not prefix interface names with “I”
  • 2.2.18. Do not use types names used in Java core packages [explain]

2.3. Classes

  • 2.3.1. Minimize implementation dependencies
  • 2.3.2. List public methods first [explain]
  • 2.3.3. Declare implementation methods private
  • 2.3.4. Define at least one public concrete class which extends a public abstract class [explain]
  • 2.3.5. Provide adequate defaults for the basic usage scenarios
  • 2.3.6. Design classes with strong invariants
  • 2.3.7. Group stateless, accessor and mutator methods together
  • 2.3.8. Keep the number of mutator methods at a minimum
  • 2.3.9. Consider providing a default no-parameter constructor [explain]
  • 2.3.10. Consider overriding equals(), hashCode() and toString() [explain]
  • 2.3.11. Consider implementing Comparable [explain]
  • 2.3.12. Consider implementing Serializable [explain]
  • 2.3.13. Consider making classes re-entrant
  • 2.3.14. Consider declaring the class as final [explain]
  • 2.3.15. Consider preventing class instantiation by not providing a public constructor [explain]
  • 2.3.16. Consider using custom types to enforce strong preconditions as class invariants
  • 2.3.17. Consider designing immutable classes [explain]
  • 2.3.18. Avoid static classes
  • 2.3.19. Avoid using Cloneable
  • 2.3.20. Do not add instance members to static classes
  • 2.3.21. Do not define public constructors for public abstract classes clients should not extend [explain]
  • 2.3.22. Do not require extensive initialization

2.4. Interfaces

  • 2.4.1. Provide at least one implementing class for every public interface
  • 2.4.2. Provide at least one consuming method for every public interface
  • 2.4.3. Do not add methods to a released public Java interface
  • 2.4.4. Do not use marker interfaces
  • 2.4.5. Do not use public interfaces as a container for constant fields

2.5. Enumerations

  • 2.5.1. Consider specifying a zero-value (“None” or “Unspecified”, etc) for enumeration types
  • 2.5.2. Avoid enumeration types with only one value
  • 2.5.3. Do not use enumeration types for open-ended sets of values
  • 2.5.4. Do not reserve enumeration values for future use
  • 2.5.5. Do not add new values to a released enumeration

2.6. Exceptions

  • 2.6.1. Ensure that custom exceptions are serialized correctly
  • 2.6.2. Consider defining a different exception class for each error type
  • 2.6.3. Consider providing extra information for programmatic access
  • 2.6.4. Avoid deep exception hierarchies
  • 2.6.5. Do not derive custom exceptions from other than Exception and RuntimeException
  • 2.6.6. Do not derive custom exceptions directly from Throwable
  • 2.6.7. Do not include sensitive information in error messages

2.7. Documentation

  • 2.7.1. Provide type overview for each type
  • 2.7.2. Follow standard Javadoc conventions
  • 2.7.3. Begin with a short, one sentence summary of the type
  • 2.7.4. Provide enough details to help deciding if and how to use the type
  • 2.7.5. Explain how to instantiate the type
  • 2.7.6. Provide code sample to illustrate the main use case for the type
  • 2.7.7. Include links to relevant sections in the Developer Guide
  • 2.7.8. Include links to relevant sections in the Cookbook
  • 2.7.9. Indicate related types
  • 2.7.10. Indicate deprecated types using the @deprecated tag
  • 2.7.11. Document class invariants
  • 2.7.12. Avoid lengthy type overviews
  • 2.7.13. Do not generate Javadoc for private fields and methods

3. Method Design Checklist

3.1. General

  • 3.1.1. Make sure each method does only one thing
  • 3.1.2. Ensure related methods are at the same level of granularity
  • 3.1.3. Ensure no boilerplate code is needed to combine method calls
  • 3.1.4. Make all method calls atomic
  • 3.1.5. Design protected methods with the same care as public methods
  • 3.1.6. Limit the number of mutator methods
  • 3.1.7. Design mutators with strong invariants
  • 3.1.8. Favor generic methods over a set of overloaded methods
  • 3.1.9. Consider generic methods
  • 3.1.10. Consider method pairs, where the effect of one is reversed by the other
  • 3.1.11. Avoid “helper” methods
  • 3.1.12. Avoid long-running methods
  • 3.1.13. Avoid forcing callers to write loops for basic scenarios
  • 3.1.14. Avoid “option” parameters to modify behavior
  • 3.1.15. Avoid non-reentrant methods
  • 3.1.16. Do not remove a released method
  • 3.1.17. Do not deprecate a released method without providing a replacement
  • 3.1.18. Do not change the signature of a released method
  • 3.1.19. Do not change the observable behavior of a released method
  • 3.1.20. Do not strengthen the precondition of an already released API method
  • 3.1.21. Do not weaken the postcondition of an already released API method
  • 3.1.22. Do not add new methods to released interfaces
  • 3.1.23. Do not add a new overload to a released API

3.2. Naming

  • 3.2.1. Begin names with powerful, expressive verbs
  • 3.2.2. Use camelCasing
  • 3.2.3. Reserve “get”, “set” and “is” for JavaBeans methods accessing local fields
  • 3.2.4. Use words familiar to callers
  • 3.2.5. Stay close to spoken English
  • 3.2.6. Avoid abbreviations
  • 3.2.7. Avoid generic verbs
  • 3.2.8. Avoid synonyms
  • 3.2.9. Do not use underscores
  • 3.2.10. Do not rely on parameter names or types to clarify the meaning of the method

3.3. Parameters

  • 3.3.1. Choose the most precise type for parameters
  • 3.3.2. Keep the meaning of the null parameter value consistent across related method calls
  • 3.3.3. Use consistent parameter names, types and ordering in related methods
  • 3.3.4. Place output parameters after the input parameters in the parameter list
  • 3.3.5. Provide overloaded methods with shorter parameter lists for frequently used default parameter values
  • 3.3.6. Use overloaded methods for operations with the same semantics on unrelated types
  • 3.3.7. Favor interfaces over concrete classes as parameters
  • 3.3.8. Favor collections over arrays as parameters and return values
  • 3.3.9. Favor generic collections over raw (untyped) collections
  • 3.3.10. Favor enumeration types over Boolean or integer types
  • 3.3.11. Favor putting single object parameters ahead of collection or array parameters
  • 3.3.12. Favor putting custom type parameters ahead of standard Java type parameters
  • 3.3.13. Favor putting object parameters ahead of value parameters
  • 3.3.14. Favor interfaces over concrete classes as return types
  • 3.3.15. Favor empty collections to null return values
  • 3.3.16. Favor returning values which are valid input for related methods
  • 3.3.17. Consider making defensive copies of mutable parameters
  • 3.3.18. Consider storing weak object references internally
  • 3.3.19. Avoid variable length parameter lists
  • 3.3.20. Avoid long parameter lists (more than 3)
  • 3.3.21. Avoid putting parameters of the same type next to each other
  • 3.3.22. Avoid out or in-out method parameters
  • 3.3.23. Avoid method overloading
  • 3.3.24. Avoid parameter types exposing implementation details
  • 3.3.25. Avoid Boolean parameters
  • 3.3.26. Avoid returning null
  • 3.3.27. Avoid return types defined in unrelated APIs, except core Java APIs
  • 3.3.28. Avoid returning references to mutable internal objects
  • 3.3.29. Do not use integer parameters for passing predefined constant values
  • 3.3.30. Do not reserve parameters for future use
  • 3.3.31. Do not change the parameter naming or ordering in overloaded methods

3.4. Error handling

  • 3.4.1. Throw exception only for exceptional circumstances
  • 3.4.2. Throw checked exceptions only for recoverable errors
  • 3.4.3. Throw runtime exceptions to signal API usage mistakes
  • 3.4.4. Throw exceptions at the appropriate level of abstraction
  • 3.4.5. Perform runtime precondition checks
  • 3.4.6. Throw NullPointerException to indicate a prohibited null parameter value
  • 3.4.7. Throw IllegalArgumentException to indicate an incorrect parameter value other than null
  • 3.4.8. Throw IllegalStateException to indicate a method call made in the wrong context
  • 3.4.9. Indicate in the error message which parameter violated which precondition
  • 3.4.10. Ensure failed method calls have no side effects
  • 3.4.11. Provide runtime checks for prohibited API calls made inside callback methods
  • 3.4.12. Favor standard Java exceptions over custom exceptions
  • 3.4.13. Favor query methods over exceptions for predictable error conditions

3.5. Overriding

  • 3.5.1. Use the @Override annotation
  • 3.5.2. Preserve or weaken preconditions
  • 3.5.3. Preserve or strengthen postconditions
  • 3.5.4. Preserve or strengthen the invariant
  • 3.5.5. Do not throw new types of runtime exceptions
  • 3.5.6. Do not change the type (stateless, accessor or mutator) of the method

3.6. Constructors

  • 3.6.1. Minimize the work done in constructors
  • 3.6.2. Set the value of all properties to reasonable defaults
  • 3.6.3. Use constructor parameters only as a shortcut for setting properties
  • 3.6.4. Validate constructor parameters
  • 3.6.5. Name constructor parameters the same as corresponding properties
  • 3.6.6. Follow the guidelines for method overloading when providing multiple constructors
  • 3.6.7. Favor constructors over static factory methods
  • 3.6.8. Consider a no parameter default constructor
  • 3.6.9. Consider a static factory method if you don’t always need a new instance
  • 3.6.10. Consider a static factory method if you need to decide the precise type of object at runtime
  • 3.6.11. Consider a static factory method if you need to access external resources
  • 3.6.12. Consider a builder when faced with many constructor parameters
  • 3.6.13. Consider private constructors to prevent direct class instantiation
  • 3.6.14. Avoid creating unnecessary objects
  • 3.6.15. Avoid finalizers
  • 3.6.16. Do not throw exceptions from default (no-parameter) constructors
  • 3.6.17. Do not add a constructor with parameters to a class released without explicit constructors

3.7. Setters and getters

  • 3.7.1. Start the name of methods returning non-Boolean properties with “get”
  • 3.7.2. Start the name of methods returning Boolean properties with “is”, “can” or similar
  • 3.7.3. Start the name of methods updating local properties with “set”
  • 3.7.4. Validate the parameter of setter methods
  • 3.7.5. Minimize work done in getters and setters
  • 3.7.6. Consider returning immutable collections from a getter
  • 3.7.7. Consider implementing a collection interface instead of a public propertie of a collection type
  • 3.7.8. Consider read-only properties
  • 3.7.9. Consider making a defensive copy when setting properties of mutable types
  • 3.7.10. Consider making a defensive copy when returning properties of mutable type
  • 3.7.11. Avoid returning arrays from getters
  • 3.7.12. Avoid validations which cannot be done with local knowledge
  • 3.7.13. Do not throw exceptions from a getter
  • 3.7.14. Do not design set-only properties (with public setter no public getter)
  • 3.7.15. Do not rely on the order properties are set

3.8. Callbacks

  • 3.8.1. Design with the strongest possible precondition
  • 3.8.2. Design with the weakest possible postcondition
  • 3.8.3. Consider passing a reference to the object initiating the callback as the first parameter of the callback method
  • 3.8.4. Avoid callbacks with return values

3.9. Documentation

  • 3.9.1. Provide Javadoc comments for each method
  • 3.9.2. Follow standard Javadoc conventions
  • 3.9.3. Begin with a short, one sentence summary of the method
  • 3.9.4. Indicate related methods
  • 3.9.5. Indicate deprecated methods using the @deprecated tag
  • 3.9.6. Indicate a replacement for any deprecated methods
  • 3.9.7. Avoid lengthy comments
  • 3.9.8. Document common behavioral patterns
  • 3.9.9. Document the precise meaning of a null parameter value (if permitted)
  • 3.9.10. Document the type of the method (stateless, accessor or mutator)
  • 3.9.11. Document method preconditions
  • 3.9.12. Document the performance characteristics of the algorithm implemented
  • 3.9.13. Document remote method calls
  • 3.9.14. Document methods accessing out-of-process resources
  • 3.9.15. Document which API calls are permitted inside callback methods
  • 3.9.16. Consider unit tests for illustrating the behavior of the method

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.

API Design Webinar – Video and Slides

This is the recording of a webinar I presented to the OpenText R&D staff. It summarizes the material of the nine articles (totaling approximately 35 printed pages) published so far:

If you had no time to read the articles, perhaps you can spare 60 minutes to watch the video. If not, you can browse the slides to get an overall idea of the topics covered so far.  I apologize for the quality of the recording made over a regular phone line by our teleconferencing provider.  I do not apologize for my Hungarian accent, which is, of course, entirely  intentional…

Video Recording

Slides

 

 

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.

Writing helpful API documentation

Many APIs are surprisingly poorly documented considering they are supposed to be “documented programmatic interfaces”. Developers prefer writing code over documentation, rarely showing the same enthusiasm and thoughtfulness for the latter. Some developers claim they write self-documenting code. Others like to point out that “nobody reads the documentation”. Such excuses create a vicious circle: we dislike documenting because we are not skilled enough and our skills do not improve because we pass up on opportunities to practice them. We need to make a conscientious effort to break this trend.

How developers use documentation

We already proved that self-documenting APIs are an unreachable ideal. Let’s refute the “Nobody reads the documentation” claim as well. In Two studies of opportunistic programming, Joel Brandt and his colleagues report that, on average, participants in their studies spend 19% of their time foraging the Internet for information. Web access logs to Adobe Flex documentation show 24,293 programmers making 101,289 queries during the month of July 2008 alone. Are these numbers we expect from documentation nobody reads? Then why is the “Nobody reads the documentation” misconception so widespread? Figure 1 compares the percentage of developers skimming the documentation, focusing only on prominent text and headers (“Skim”) to those systematically reading the pages line-by-line (“Line-by-line”). Another axis compares the number of those starting with the provided PDF overview (“PDF overview”) to those preferring to go straight to the reference manual, expecting it to be self-explanatory (“Self-explanatory”).

How developers use documentation

Figure 1: How developers use documentation

The conclusion is clear: documentation is referenced, not read. If reading the documentation cover-to-cover line-by-line is the only way to find information, developers will not find it, creating the false impression that they don’t read the documentation. Let’s take a real-life example from stackoverflow.com as illustration:

Question

With java.sql.ResultSet is there a way to get a column’s name as a String by using the column’s index? I had a look through the API doc but I can’t find anything.

Answer

See ResultSetMetaData:

ResultSet rs = stmt.executeQuery(“SELECT a, b, c FROM TABLE2″);

ResultSetMetaData rsmd = rs.getMetaData();

String name = rsmd.getColumnName(1);

This developer skimmed the 139 methods of the java.sql.ResultSet class, looking for getColumnName() or something similar, but skipped getMetadata() because it looked irrelevant. He preferred asking for help on the Internet to closely inspecting all 139 methods, which illustrates our main point: developers don’t want documentation; they want assistance with the task at hand.

In Documentation Usability: A Few Things I’ve Learned from Watching Users, Tom Johnson writes:

Invariably when I ask people how they prefer to learn new software, they respond, ‘I like someone to show me,’ or ‘I like to play around in the system and then ask a colleague if I get stuck.’ I’ve yet to hear the response, ‘I like long software manuals with lots of text in small print.’ Usually people that prefer this also like to slam their fingers in car doors and chew on tin foil.

Answering questions like a friend

We should act like a friend assisting the programmer when writing API documentation. Someone working with the Java Messaging Service (JMS) asks this question on StackOverflow.com:“What is the purpose of a JMS session? Why isn’t a connection alone sufficient to exchange JMS messages between senders and receivers?” Sounds like a legitimate question, right? How many developers would ask “Please enumerate the methods of the class javax.jms.Session in alphabetical order” from a friend? None? But often this is the only question answered by the Javadoc after clicking a class name!

If you document like a friend, you provide package and class overviews, answering useful questions like: “What is the purpose of this package?” “What can I use this class for?” “Are there any limitations?” “This is not quite what I need, what are some related classes?” Would you make a friend read dozens of pages of API minutiae just to infer the answers to such simple questions?

How do we know what questions to answer? In “Specifying behavior”, we explained that three-quarters of API questions are about behavior. Consequently, documenting preconditions, postconditions and invariants alone completes three-quarters of API documentation! We can also invite people unfamiliar with the API to review it and record the questions and answers in a FAQ. FAQs are easy to create and extend. API documentation is never quite complete and FAQs capture missing facts, clarify ambiguities, or document known issues. Most FAQs are temporary, kept only until the other parts of the documentation are updated. Long collections of FAQs are less useful.

Although we might think that callers don’t care, concepts and abstractions used in the design of the API, as well as the intent behind choosing them, need to be explained. As one developer eloquently says:

When you’re building a framework, there’s an intent … if you can understand what the intent was, you can often code efficiently, without much friction. If you don’t know what the intent is, you fight the system.

Supporting just-in-time learning

Research shows that developers learn APIs incrementally, interleaving short periods of studying documentation with writing code. Helpful API documentation matches this just-in-time learning pattern, consisting of small, self-contained, heavily cross-referenced sections. Developers spend no more than ten minutes with documentation before returning to code, the studies show. This sets the maximum size of an undivided documentation section at about half a page.

Because developers skim the documentation, each section needs to focus on a single subject and highlight what the subject is. Imagine that you work in customer support. You want to link to a section of the documentation as the answer to a specific customer question. When a section contains primarily irrelevant information, you are wasting the customer’s time. If the first sentence does not read like you are answering the question, you’d be too embarrassed to link to it.

Navigation and search are essential for finding the correct documentation section. Navigation produced by tools like Javadoc has some limitations, evident from the generated alphabetical index. Packages, classes and methods are listed there, but adding non-structural entries like “Performance” or “Thread safety” to the index is not directly supported by the tools. Many developers simply type their questions into an Internet search engine and expect it to find the correct answer. API documentation not available on the Internet or not optimized for this type of searching is less useful.

Illustrating use cases

Answers to “How do I?” questions, the kind of use-case driven questions tens of thousands of developers ask daily on Internet forums tend to be more helpful than answers to “What is this?” questions:

The problem is always, when I feel I can’t make progress … when there’s multiple functions or methods or objects together that, individually they makes sense but sort of the whole picture isn’t always clear, from just the docs.

Code snippets are often the straightforward answer to “How do I?” questions. The “How do I get the JDBC column name?” question above was answered with just 3 lines of code. If we follow the “consider the perspective of the caller” guideline, we write use case driven code samples right at the beginning of the API design process. We can turn these code samples effortlessly into a cookbook, an increasingly popular form of developer documentation, by describing briefly the use case each code snippet illustrates.

The code examples must be exemplary. They should closely follow all relevant programming best practices. A sloppy example “can become more of a hindrance than a benefit when there’s a mismatch between the tacit purpose of the example and the goal of the example user” warns Martin P. Robinard.

Tutorials serve a similar purpose, but differ in important aspects. They break up the building of the complete example into smaller, more manageable steps. Tutorials are intended to be completed from start to finish. They intend to teach. They are not reference material. Many programmers started learning Windows programming from the famous Scribble tutorial. New tools for recording video on a computer (screencasting) have made video tutorials very popular. Nevertheless, tutorials can be time-consuming to produce and are rarely needed for simple APIs.

Putting it all together

Developers need API documentation for four main activities:

  1. Remind themselves of details deemed not worth remembering
  2. Clarify and extend their existing knowledge
  3. Engage in just-in-time learning of new skills
  4. Experiment with (sample) code

Reference documentation only supports the first two activities. A separate Programmer’s Guide is needed to support just-in-time learning because research shows that developers waste a great deal of time guessing, inspecting and backtracking when learning directly from reference documentation. Unfortunately, the name Programmer’s Guide still evokes a heavy book, which is not what we are advocating. In addition to a high-level overview, a modern Programmer’s Guide contains only topics that don’t fit nicely into either the Reference Manual or the Cookbook format, such as describing new concepts, conventions, design patterns, and so on. With their just-in-time learning style, programmers make frequent jumps between Programmer’s Guide, Reference Manual and Cookbook, provided these are properly cross-referenced. The table below summarizes the various parts of a complete API documentation.

Documentation Part Programmer Activities Contents Organization
Reference Manual
  • Remembering details
  • Extension of knowledge
  • Package overviews
  • Class overviews
  • Method descriptions
  • Structural top-down
  • Alphabetical Index
Cookbook
  • Experimenting with code
  • Extension of knowledge
  • Just-in-time learning
  • List of use cases
  • Description of use cases
  • Code snippets
  • By use-case
Programmer’s Guide
  • Just-in-time learning
  • Extension of knowledge
  • Introductory overview
  • Glossary
  • Concepts
  • Conventions
  • Design patterns
  • other…
  • By subject
  • Alphabetical Index
Code Example or Tutorial
  • Experimenting with code
  • Source files
  • Build or project files
  • Other resources
  • Video (optional)
  • Development project
FAQ or Knowledge Base
  • Extension of knowledge
  • Clarifications
  • Tips
  • Traps
  • Known issues
  • Questions and answers

Conclusion

It is hard to over-emphasize the importance of good API documentation. Even exceptionally well designed APIs can be frustrating to use if poorly documented. On the other hand, the only way to improve an existing API, without a complete and expensive redesign, is to improve the documentation. API documentation is developer documentation. Since nearly all developer documentation we write is for internal use, we often compromise on quality. It is crucial to understand that public APIs need to be treated differently. The target audience includes external developers or system integrators, who are also customers. The same quality requirements apply as to customer documentation like administration guides or end-user manuals.

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.

Anticipating evolution

API designers need to resolve an apparent paradox: how to keep APIs virtually unchanged yet respond to ever-changing customer requirements. It is more intricate a skill than simply applying specific API evolution techniques. It can be compared to a chess master’s ability to anticipate several upcoming moves of a game. Just like beginner chess players, we start by learning the specific API evolution techniques, but we become true experts when we are able to plan ahead for at least a couple of API releases. We are more likely to design long-lasting, successful APIs if we master this skill.

Let’s start with the fundamental rule of API evolution: existing clients must work with a new product release without any changes, not even a recompilation. While breaking changes can be tolerated in internal code, they are prohibited in public APIs. We must either limit ourselves to binary compatible changes or keep the old API unchanged while introducing a new API in parallel, a method called API versioning.

Maintaining backwards compatibility

Backwards compatible changes are preferable because clients can upgrade smoothly and without any human intervention, taking advantage of new features at their convenience. Conversely, API versioning demands an explicit decision to upgrade because code changes are required. Clients frequently choose to defer upgrades, requiring a long period of support for multiple API versions. We should plan to evolve APIs primarily through backwards compatible changes. We should avoid API versioning if possible.

Anticipating evolution means choosing designs which allow the largest number of backwards compatible changes. For example, C++ developers know that adding a field to a C++ class changes its size and breaks binary compatibility with client code into which size was hard-coded by the compiler. Similarly, adding a virtual method modifies the virtual method table, causing clients to call wrong virtual functions (see Listing 1). Because the need for new fields and methods is likely to arise, smart designers move all fields and virtual methods into a hidden implementation class (see Listing 3), leaving only public methods and a single private pointer in the public class (see Listing 2):

Listing 1: Original API class design is hard to evolve

#include <vector>  //exposed direct dependency on STL
#include "Node.h"  //exposed implementation class Node
class OriginalClass {

public:
	int PublicMethod(...);

protected:
	std::vector<Node> children; 

	// Adding a field modifies the size, breaks compatibility
	int count; 

	// Adding a method modifies the vtable, breaks compatibility
	virtual void ProtectedMethod(...);
};

Listing 2: New API class design using the Façade pattern

class ImplementationClass; //declares unknown implementation class

class FacadeClass {

public:
	int PublicMethod(...); 

private:
	ImplementationClass *implementation; //size of a pointer
};

Listing 3: The implementation details are never exposed to the client

#include <vector>  //OK, client code never includes it
#include "Node.h"  //OK, client code never includes it

class ImplementationClass {

public:
	int PublicMethod(...);

protected:
	std::vector<Node> children; 

	//OK, client never instanciates direcly
	int count; 

	//OK, the client has no direct accesses to the vtable
	virtual void ProtectedMethod(...);
};

Binary compatible changes are different depending on platform. Adding a private field or a virtual method is a breaking change in C++, but a backwards compatible change in Java. As one of our teams recently discovered, extending SOAP Web Services by adding an optional field is a compatible change in JAX-WS (Java) but a breaking change in .Net. Providing lists of compatible changes for each platform is outside the scope of this document; this information can be found on the Internet. For example, the Java Language Specification states the binary compatibility requirements and Eclipse.org gives practical advice on maintaining binary compatibility in Java. The KDE TechBase is a good starting point for developers interested in C++ binary compatibility.

While we are comparing platforms, we should mention that standard C is preferable to C++ for API development. Unlike C, C++ does not have a standard Application Binary Interface (ABI). As a result, evolving multi-platform C++ APIs while maintaining binary compatibility can be particularly challenging.

Keeping APIs small and hiding implementation details help maintain backwards compatibility. The less we expose to the clients, the better. Unfortunately, compatibility requirements also extend to implementation details inadvertently leaked into the API. If this happens, we cannot modify the implementation without using API versioning. Carefully hiding implementation details prevents this problem.

We can break backwards compatibility (without modifying method signatures) by changing the behavior. For example, if a method always returned a valid object and it is modified so that it may also return null, we can reasonably expect that some clients will fail. Maintaining the functional compatibility of APIs is a crucial requirement, one that requires even more care and planning than maintaining binary compatibility.

The only backwards compatible behavior changes are weakened preconditions or strengthened postconditions. Think of it as a contractual agreement. Preconditions specify what we ask from the client. We may ask for less, but not more. Postconditions specify what we agreed to provide. We may provide more, but not less. For example, exceptions are preconditions (we expect clients to handle them). It is not allowed to throw new exceptions from existing methods. If a method is an accessor, a part of its postcondition is a guarantee that the method does not change internal state. We cannot convert accessors into mutators without breaking the clients. The invariant is part of the method’s postcondition and should only be strengthened.

API behavior changes are likely to go undetected since developers working with implementation code often do not realize the full impact of their modifications. When we talked about specifying behavior, we already noted the importance of explicitly stating the preconditions, postconditions and invariants, as well as providing automated tests for detecting inadvertent modifications. Now we see that those same practices also help maintain functional compatibility as the API evolves.

SPIs (Service Provider Interfaces) evolve quite differently from APIs because responsibilities of the client and the SPI implementation are often reversed. APIs provide functionality to clients, while SPIs define frameworks into which clients integrate. Clients usually call methods defined in APIs, but often implement methods defined in SPIs. We can add a new method to an interface without breaking APIs, but not without breaking SPIs. The way pre- and postconditions can evolve is often reversed in SPIs. We can strengthen preconditions (this is what the SPI guarantees) and weaken postconditions (this is what we ask from the client to provide) without breaking clients. The differences between APIs and SPIs are not always clear. Adding simple callback interfaces will not turn APIs into SPIs, but callbacks evolve like SPI interfaces.

Surprisingly, we need to worry less about source compatibility, which requires that clients compile without code changes. While binary and source compatibility do not fully overlap, all but a few binary compatible changes are also source compatible. Examples of exceptions are adding a class to a package or a method to a class in Java. These are binary compatible changes, but if the client imports the whole package and also references a class with the same name from another package, compilation fails due to name collision. If a derived class declares a method with the same name as a method added to the base class, we have a similar problem. Source incompatibility issues are rare with binary-compatible APIs and require few changes in client code.

If we focus too much on source compatibility, we increase the risk of breaking binary compatibility since not all source compatible changes are binary compatible. For example, if we change a parameter type from HashMap (derived type) to Map (base type), the client code still compiles. However, when attempting to run an old client, the Java runtime looks for the old method signature and it cannot find it. The risk of breaking binary compatibility is real because during their day-to-day work, developers are more concerned about breaking the build than about maintaining binary compatibility.

Versioning

API versioning cannot be completely avoided. Some unanticipated requirements are impossible to implement using backwards compatible changes. Software technologies we depend on do not always evolve in a backwards compatible fashion (just ask any Visual Basic developer). Also, API quality may also degrade over time if our design choices are restricted to backwards compatible changes. From time to time, we need to make major changes in order to upgrade, restructure, or improve APIs. Versioning is a legitimate method of evolving APIs, but it needs to be used sparingly since it demands more work from both clients and API developers.

Anticipating evolution in the case of explicit versioning means ensuring that an incompatible API version is also a major API version. We should deliberately plan for it to avoid being forced by unexpected compatibility issues. The upgrade effort must be made worthwhile for clients by including valuable new functionality. We should also use this opportunity to make all breaking changes needed to ensure smooth backwards compatible evolution over the several following releases.

API versions must coexist at runtime. How we accomplish this is platform-dependent. Where available, we should use the built-in versioning capabilities; .Net assemblies have them and so does OSGi in Java, although OSGi is not officially part of the Java platform. If there is no built-in versioning support, the two API versions should reside in different namespaces, to permit the same type and method names in both versions. The old version keeps the original namespace while the new version has a namespace with an added version identifier. The API versions should also be packaged into separate dynamic link libraries, assemblies, or archives. Since C does not support namespaces, separate DLLs are needed to keep the same method names. We should make sure we change the service end point (URL) when versioning Web Services APIs, since all traffic goes through the same HTTP port. We should also change the XML namespace used in the WSDL. This ensures that client stubs generated from different WSDL versions can coexist with each other, each in its namespace.

It is often advantageous to re-implement the old API version using the new one. Keeping two distinct implementations means code bloat and increased maintenance effort for years. If the new API version is functionally equivalent to the old one, implementing a thin adaptor layer should not require much coding and testing. As an added benefit, the old API can take advantage of some of the improvements in the new code, such as bug fixes and performance optimizations.

Conclusion

Designing for evolution can be challenging and time consuming. It adds additional constraints to API design which frequently conflict with other design requirements. It is essentially a “pay now versus pay later” alternative. We can spend some effort up front designing easy-to-evolve APIs or we can spend more effort later when we need to evolve the API. Nobody can reasonably predict how an API is likely to evolve; hence nobody can claim with authority that one approach is better than the other. It is thought provoking, however, that nobody has yet come forward saying they regretted making APIs easier to evolve.

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.

Making it safe

Being safe means avoiding the risk of pain, injury, or material loss. A safety feature is a design element added to prevent inadvertent misuse of dangerous equipment. For example, one pin of the North American electric plug is intentionally wider to prevent incorrect insertion into a socket. But it was Toyota who first generalized the principle of poka-yoke (“mistake avoidance”), making it an essential part of its world-renowned manufacturing process. When similar principles of preventing, avoiding, or correcting human errors are applied to API design, the number of software defects is reduced and programmer productivity improves. Rico Mariani calls this the “pit of success”:

The Pit of Success: in stark contrast to a summit, a peak, or a journey across a desert to find victory through many trials and surprises, we want our customers to simply fall into winning practices by using our platform and frameworks. To the extent that we make it easy to get into trouble we fail.

Preventing unsafe use

Engineers place all dangerous equipment and materials – high voltage, extreme temperature, or poisonous chemicals – safely behind locked doors or inside sealed casings. Programming languages offer controlled access to classes and methods, but time and again we forget to utilize it. We leave public implementation classes in the API package. We forget to declare methods users shouldn’t call as private. We rarely disallow class construction, and seldom declare classes we don’t want callers to extend as final. We declare public interfaces even when we cannot safely accept implementations other than our own. These oversights are the equivalent of leaving the boiler room unlocked. When inadvertent access to implementation details is possible, accidents are likely to happen.

Our next line of defense is type checking. In a nutshell, type checking attempts to catch programming mistakes at the language level, either at compile time in statically typed languages, or at run time in dynamically typed languages. If interested in the details of what type checking can or cannot do for you in various languages, you should read Chris Smith’s excellent “What to know before debating type systems”. For various theoretical and practical reasons, type checking cannot catch all usage errors. It would be ideal if every statically typed API call which compiles executed safely, but present-day compilers are just not sophisticated enough to make it a reality. However, this does not mean that we should not take advantage of type checks where we can. We may be stating the obvious, yet we often see APIs which are not as type safe as they could be. The ObjectOutputStream class from the Java I/O library declares the

final void writeObject(Object obj) throws IOException

method which throws an exception if the argument is not Serializable. The alternative method signature

public final void writeObject(Serializable obj) throws IOException

could turn this runtime verification into a compile time check.

Every time a method only works for a small subset of all possible parameter values we can make it safer by introducing a more restrictive (read: safer) parameter type. Especially string, integer, or map parameter types deserve close examination because we often use these versatile types unsafely in programming. We take advantage of the fact that practically every other type can be converted into a string or represented as a map, and integers can be many more things than just numbers. This may be reasonable or even necessary in implementation code where we often need to call low-level library functions and where we control both caller and callee. APIs are, yet again, special. API safety is very important and we need to consider design trade-offs accordingly.

When evaluating design trade-offs it helps to understand that we are advocating replacing method preconditions with type invariants. This moves all safety-related program logic into a single location, the new type implementation, and relies on automatic type checking to ensure API safety everywhere else. If it removes strong and complex preconditions from multiple methods it is more likely to be worth the effort and additional complexity. For example, we recommend passing URLs as URL objects instead of strings. Many programming languages offer a built-in URL type; precisely because the rules governing what strings are valid URLs are complicated. The obvious trade-off is that callers need to construct an URL object when the URL is available as a string.

Weighing type safety against complexity is a lot like comparing apples and oranges: we must rely on our intuition, use common sense, and get lots of user feedback.  It is worth remembering that API complexity is measured from the perspective of the caller. It is difficult to tell how much the introduction of a custom type increases complexity without writing code for the use cases. Some use cases may become more complex while others may stay the same or even become simpler. In the case of the URL object, handling string URLs is more complex, but returning to a previously visited URL is roughly the same if we keep URL objects in the history list. Using URL objects result in simpler use cases for clients that build URLs from fragments or validate URLs independently from accessing the resource they refer to.

As a third and final line of defense – since type checking alone cannot always guarantee safe execution – all remaining preconditions need to be verified at run time. Very, very rarely performance considerations may dictate that we forgo such runtime checks in low-level APIs, but such cases are the exceptions. In most cases, returning incorrect results, failing with obscure internal errors, or corrupting persisted data is unacceptable API behavior. Errors resulting from incorrect usage (violated preconditions) should be clearly differentiated from those caused by internal problems and should contain messages clearly describing the mistake made by the caller. That a call caused an internal SQL error is not considered a helpful error message.

We should be particularly careful when providing classes for extension because inheritance breaks encapsulation. What does this mean? Protected methods are not a problem. Their safety can be ensured the same way as for public methods. Much bigger issues arise when we allow derived classes to override methods. Overriding is risky because callers may observe inconsistent state from within the method they override (known as the “fragile base class problem”) or may make inconsistent updates (known as the “broken contract problem”). In other words, calling otherwise safe public or protected methods from within overridden methods may be unsafe. There is no language mechanism to prevent access to public and protected methods from within overridden methods, so we often need to add additional runtime checks as illustrated below:

public Job {

   private cancelling = false;

   public void cancel() {
      ...
      cancelling =  true;
      onCancel();
      cancelling = false;
      ...
    }

    //Override this to provide custom cleanup when cancelling
    protected void onCancel() {
    }

    public void execute() {
      if(cancelling) throw IllegalStateException(“Forbidden call to
         execute() from onCancel()”);
      ...
    }
}

It is generally safer to avoid designing for class extension if it is possible. Unfortunately, simple callbacks may also expose similar safety issues, though only public methods are accessible from callbacks. In the example above, the runtime check is still needed after we make onCancel() a callback, since execute() is a public method.

Preventing data corruption

A method can only be considered safe if it preserves the invariant and prevents the caller from making inconsistent changes to internal data. The importance of preserving invariants cannot be overstated. Not long ago, a customer who used the LDAP interface to update their ADS directory reported an issue with one of our products. Occasionally the application became sluggish and consumed a lot of CPU cycles for no apparent reason. After lengthy investigations, we discovered that the customer inadvertently corrupted the directory by making an ADS group a child of its own. We fixed the issue by adding specific runtime checks to our application, but wouldn’t it be safer if the LDAP API didn’t allow you to corrupt the directory in the first place? The Windows administration tools don’t allow this, but since the LDAP interface does, applications still need to watch out for infinite recursions in the group hierarchy.

The invariant must be preserved even when methods fail. In the absence of explicit transaction support, all API calls are assumed atomic. When a call fails, no noticeable side effects are expected.

Special care must be taken when storing references to client side objects internally, as well as when returning internal object references to the client. The client code can unexpectedly modify these objects at any time, creating an invisible and particularly unsafe dependency between the client code (which we ignore) and the internal API implementation (which the client ignores). On the other hand, it is safe to store and return references to immutable objects.

If the object is mutable, it is a great deal safer to make defensive copies before storing or returning it rather than relying on the caller to do it for us. The submit() method in the example below makes defensive copies of jobs before placing them into its asynchronous execution queue, which makes it hard to misuse:

JobManager    jobManager  = ...; //initializing
Job           job = jobManager.createJob(new QueryJob());      

//adding parameters to the job
job.addParameter("query.sql", "select * from users");
job.addParameter("query.dal.connection", "hr_db");      

jobManager.submit(job); //submitting a COPY of the job to the queue      

job.addParameter("query.sql", "select * from locations"); //it is safe!
jobManager.submit(job) //submitting a SECOND job!

For the same reason, we should also avoid methods with “out” or “in-out” parameters in APIs, since they directly modify objects declared in client code. Such parameters frequently force the caller to make defensive copies of the objects prior to the method call. The .Net Socket.Select() method usage pattern shown bellow made Michi Henning frustrated enough to complain about it in his “API Design Matters“:

ArrayList readList = ...;   // Creating sockets to monitor for reading
ArrayList writeList = ...;  // Creating sockets to monitor for writing
ArrayList errorList;        // Sockets to monitor for errors.

while(!done) {

    SocketList readReady  = readList.Clone();  //making defensive copy
    SocketList writeReady = writeList.Clone(); //making defensive copy
    SocketList errorList  = readList.Clone();  //making defensive copy

    Socket.Select(readReady, writeReady, errorList, 10000);
         // readReady, writeReady, errorList were modified!
    …
}

Finally, APIs should be safe to use in multi-threaded code. Sidestepping the issue with a “this API is not thread safe” comment is no longer acceptable. APIs should be either fully re-entrant (all public methods are safe to call from multiple threads), or each thread should be able to construct its own instances to call. Making all methods thread safe may not be the best option if the API maintains state because deadlocks and race conditions are often difficult to avoid. In addition, performance may be reduced waiting for access to shared data. A combination of re-entrant methods and individual object instances may be needed for larger APIs, as exemplified by the Java Messaging Service (JMS) API, where ConnectionFactory and Connection support concurrent access, while Session does not.

Conclusion

Safety has long been neglected in programming in favor of expressive power and performance. Programmers were considered professionals, expected to be competent enough to avoid traps, and smart enough to figure out the causes of obscure failures. Programming languages like C or C++ are inherently unsafe because they permit direct memory access. Any C API call – no matter how carefully designed – may fail if memory is corrupted. However, the popularity and wide scale adoption of Java and .Net clearly signals a change. It appears that developers are demanding safer programming environments. Let’s join this emerging trend by making our APIs safer to use!

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.

Specifying behavior

In the paper “Six Learning Barriers in End-User Programming Systems”, Andrew J. Ko and his colleagues show that programmers make numerous assumptions when working with unfamiliar APIs, over three-quarters of them about API behavior. While programmers can directly examine type definitions and method signatures, they need to infer behavior from method and parameter names. It is not entirely surprising that many such assumptions turn out to be incorrect. Ko’s paper documents a total of 130 cases when programmers failed to complete the assigned task. In 36 of those cases, the programmers did not succeed in making the API call at all. In a further 38 cases, they were unable to understand why the call behaved differently than expected and what to do to correct it. In another 25 cases, they were unable to successfully combine two or more method calls to solve the problem.

Why self-documenting APIs are rare

Under-specified behavior causes serious usability issues in numerous APIs. Many developers honestly believe in self-documenting APIs, but as we will show, fully self-documenting APIs are an ideal towards we should aim, rather than a result we can realistically expect to achieve. Despite our very best efforts, subtle and unintuitive behavior is present in most APIs.

Even in the seemingly clear-cut cases, figuring out the precise behavior without additional help can be unexpectedly daunting. Take the TeamsIdentifier class shown below as an example:

//Uniquely identifies an entity.
class TeamsIdentifier {

   //Constructs an identifier from a string.
   TeamsIdentifier(String id) {...}

   //Returns the id as a String.
   java.lang.String asString() {...}

   //Convenience method to return this id as an array.
   TeamsIdentifier[] asTeamsIdArray() {...}

   // Returns a copy of the object.
   java.lang.Object clone() {...}

   //Checks if two ids are equal.
   boolean equalsId(TeamsIdentifier id) {...}

   // Intended for hibernate use only.
   java.lang.String getTeamsId() {...}

   boolean equals(java.lang.Object o) {...}
   int hashCode() {...}
   void setTeamsId(java.lang.String id) {...}

   //Returns a string representation of the id.
   java.lang.String toString() {...}
}

It looks straightforward enough, you say. Let’s see if you can answer, in total confidence, the following questions:

Expression True or False?
TeamsIdentifier id1 = new TeamsIdentifier(“name”)
TeamsIdentifier id2 = new TeamsIdentifier(“Name”)
id1.equals(id2)
   ?
id1.equalsId(id2)
   ?
id1.toString().equals(“name”)
   ?
id1.getTeamsId().equals(“name”)
   ?
TeamsIdentifier id = new TeamsIdentifier(“a.b.c”)
id.asTeamsIdArray().length == 3
   ?
TeamsIdentifier id = new TeamsIdentifier(“a:b:c”)
id.asTeamsIdArray().length == 3
   ?

Knowing that AssetIdentifier and UserIdentifier both extend TeamsIdentifier, can you answer, again in total confidence, the questions below?

Expression True or False?
AssetIdentifier assetId = new AssetIdentifier(“Donald”)
UserIdentifier userId = new UserIdentifier(“Donald”)
assetId.equals(userId)
   ?
assetId.equalsId(userId)
   ?
assetId.toString().equals(userId.getTeamsId())
   ?

Of course, we can make sensible assumptions about what the correct behavior should be, but we have to honestly admit that we don’t really know. For that we either need to test the API or look at the implementation. Looking at the implementation is rarely a practical option. Learning by trial and error is time consuming and it doesn’t tell us which observed behavior is by design as opposed to merely accidental. For example, if we get the same AssetIdentifier object back every time, we might incorrectly assume that we can write id1 == id2 instead of id1.equals(id2). Our program works correctly only until the next version of the API comes out.

We provide a huge service to our users when we remove guesswork from API usage by properly documenting behavior.

Using code for specifying behavior

Code is more concise and precise than words. It is difficult to think of a good reason why not to use code for specifying API behavior. We are documenting for developers, who should welcome, and have no problem understanding code. The above tables document the behavior of TeamsIdentifier and its derived classes when we enter the appropriate True or False values into the second column. You probably noticed that the code in the first column is similar to what we would write for unit tests. In the case of APIs, unit tests are twice as useful because they also document the expected behavior. Some developers call these code snippets assertions, while those familiar with the work of Professor Bertrand Mayer call this particular method of specifying behavior Design by Contract. Starting with version 4.0, the .Net Framework natively supports design by contract, while third-party tools exist for many other programming languages.

No matter what we call it or what tool we use, we should precisely specify API behavior using code.

Indicating stateless, accessor and mutator methods

The existence of observable internal state is a primary cause of unintuitive behavior, since it allows a method call to modify the result of the next (seemingly unrelated) call. A stateful algorithm controls access rights in multi-user systems. Is it possible to discover, from studying the API alone, how moving a document into a different folder affects its access rights? Isn’t it true that this depends not only on the security settings assigned to the document itself and those of the destination folder, but also on the security settings of its parent folder and recursively up to the root folder? Doesn’t it also depend on the user’s assigned roles, group memberships and perhaps on the security model currently in use? All these settings may be accessible via the API, but they alone won’t tell us how the access control algorithm actually works.

Realizing that state prevents us from designing self-documenting APIs, we could be tempted to stick to stateless APIs. While this isn’t always possible, it is an excellent idea to isolate the impact of internal state to the smallest possible part of APIs. We should have as many stateless methods as possible, since their behavior only depends on the parameter values. In object-oriented environments we should also favor immutable objects, which have state that cannot be changed once the objects are created. Fixed state is obviously less predictable than no state, but more predictable than evolving state.

Where we cannot avoid modifiable state, we should group the affected methods into two distinct categories: accessors, which can only read the state, and mutators, which can also change it. Accessors are like gauges on a control panel, and mutators are like switches and buttons. The accessors produce the same result when called a second or third time in a row, while mutators may produce a different result every time. Inserting a call to an accessor into the middle of an existing program is safe, while inserting a mutator may change the behavior of the subsequent API calls, breaking the program’s logic.

We must explicitly tell callers if a method is stateless, an accessor, or a mutator to help them use it correctly. We cannot rely on them guessing correctly or on naming conventions alone. We won’t be able to start all accessor names with “get” or “is” – show() or print() are accessors, as are many other, less obviously named methods. Because mutators are the most challenging, it is a good idea to keep their number to an absolute minimum and pay careful attention to their design.

Using strong invariants

Not all mutators are equally problematic. The stronger the invariant, the more predictable and intuitive the behavior becomes. The invariant is a set of statements (assertions) about behavior, which always hold true, regardless of state. It is essentially guaranteed, predictable behavior. We will illustrate this with an API, which helps us cover a geometrical shape with a triangular mesh as shown in the figure below:

Triangular mesh

Triangular mesh

Depending on our design, some or all of the following statements may be true after each API call:

  1. The whole geometric area is fully covered with the mesh
  2. All triangles in the mesh are regular (the triangle area is not null, no two nodes overlap each other, the three nodes don’t lie on the same straight line, etc.)
  3. There are no unconnected nodes
  4. No two triangles overlap each other
  5. Every node lies either inside or on the boundary of the geometric shape
  6. Every edge lies either inside or on the boundary of the geometric shape

The simplest API we can imagine, which requires us to insert and connect nodes directly, cannot guarantee any of this and would be rather difficult to use (remember, you cannot see the mesh when programming with an API!). We intuitively know that an API, which could guarantee all of the above invariants, would be much easier to use, but is such an API feasible? While it is not easy to figure them out, such mutators exist, and they are known as the Delaunay mesh refinement operators. Here are four of them:

Triangle split – splits a triangle into three smaller ones by adding a new node in the middle

Edge split – replaces two adjacent triangles with four smaller ones by splitting the common edge into two halves

Edge flip – changes the shape of two adjacent triangles by flipping the shared edge to the other diagonal of the bounding rectangle

Node nudge – changes the shape of the connected triangles by repositioning a node inside the polygon defined by the neighboring nodes

Delauney mesh refinement operators

Delauney mesh refinement operators

Notice how simple it is to describe what each method does? To see the big difference this design makes, try to describe how to correctly refine a mesh by inserting and (re)connecting nodes, and then do it again using the Delaunay operators. Which is easier?

Great APIs have strong invariants, but as we just saw, this doesn’t happen by itself, it requires careful design.

Using weak preconditions

Weak preconditions help callers just like strong invariants. If invariants are constraints on the API designer, preconditions are constraints on the caller: conditions which should be met for the call to succeed. From the caller perspective, the invariants should be strong, and the preconditions weak. In an ideal world, all API calls would succeed and produce correct results for all possible arguments. In the real world, this is either impossible or it conflicts with other design requirements. The trick is to stay as close to the ideal solution as possible.

For example, one of our APIs limits the length of string method parameters to less than 255 characters for efficient database storage and better performance. On the other hand, it would be easier to use without these limitations. Web Services APIs, in general, are infamous for taking complex data structures as arguments, yet they only work when these data structures are appropriately constructed. The documentation rarely states the preconditions explicitly, leading to backbreaking trial-and-error style programming.

To sum it up, weak preconditions (or no preconditions) are better than strong ones, and documented preconditions are far preferable to undocumented ones.

Conclusion

Observable state is just one of the many reasons why self-documenting APIs are a largely unreachable ideal. Reentrancy, performance characteristics, extensibility via inheritance, the use of callbacks, caching, clustering and distributed state can all lead to complex, unintuitive behavior. While careful design using strong invariants and weak preconditions can make API behavior more predictable, behavior still needs to be explicitly specified. The recommended way of specifying behavior is with code in the form of unit tests, assertions or contracts.

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.

Follow

Get every new post delivered to your Inbox.

Join 29 other followers