Considering the perspective of the caller

Regular software design produces mediocre APIs because focus on implementation hurts APIs in many different ways. We will illustrate how with an internal Java API to avoid singling out any well-known public APIs. We immediately notice the complexity. The class ContentInstance, for example, appears 5 levels deep in the class hierarchy, and implements 4 additional interfaces (not counting the standard Serializable interface):

com.company.service.javabean
Class ContentInstance

java.lang.Object
  extended by com.company.service.common.DataObject
      extended by com.company.service.javabean.ManagedObject
          extended by com.company.service.javabean.ExtensibleObject
              extended by com.company.service.javabean.ContentItem
			extended by com.company.service.javabean.ContentInstance

All Implemented Interfaces:
    IAttributedObject, IChannelAssociate, IPersistable, IRelatedAttribute, java.io.Serializable

ContentInstance itself defines (or overrides) 33 methods, which is reasonable, but it also inherits a whopping 79 methods from ManagedObject, 8 methods from ExtensibleObject and 7 methods from DataObject for a grand total of over one hundred methods. That’s a lot of methods in a single class! We agree that complex problems require complex solutions and we should have no problem seeing similar complexity in a (hidden) implementation. In APIs, however, we greatly value simplicity. We don’t like spending time browsing through dozens of classes and hundreds of methods. We like it when ContentInstance is always ContentInstance and not ManagedObject or IChannelAssociate depending on the context, which tends to happen a lot when using such complex inheritance hierarchies. We are glad that someone else did the implementation work for us, but we don’t feel the need to understand how they did it. We focus on what we are implementing when using APIs, and frankly, we don’t have much time for anything else. The better the API manages to hide implementation details from us, the more we appreciate it.

Excessive abstraction is another problem which arises from implementation-focused API design. DataObject, ManagedObject, ExtensibleObject, ContentItem and ContentInstance are all pure design abstractions with no corresponding real-world objects or concepts we could immediately relate to. What’s the difference between DataObject and ManagedObject or between ContentItem and ContentInstance? We need to understand the whole API before we can understand its parts, a daunting task with a large API. We are happy to acknowledge that no complex problem can be solved without powerful abstractions. On the other hand, we need to confess that we find it difficult to understand someone else’s abstract concepts, because for this we must think like that other person. We wish the other person thought more like us instead, in familiar concepts like Document, Folder, Project or User.

Complex and counter-intuitive usage patterns are a third annoyance of implementation-focused design.  Read a programmer’s recollection of trying to figure out how to create a new instance of the ContentInstance class: “At first, I didn’t expect that ContentInstance can be instantiated because I found no constructor and no factory method in the class definition. Only after further investigation did I discover the newInstance() factory method ContentInstance inherits from the abstract super class ManagedObject. I was confused by the abstract base class declaring a factory method while the concrete class did not. Eventually, I learned from the documentation that using the ContentInstance and ContentType classes is similar to using Object and Class in the Java reflection API. The correct way of instantiating a ContentInstance was by calling ContentType.newInstance(), the inherited static newInstance() method proving to be a bit of a red herring. While the analogy with the Java Reflection API certainly helped, I started wondering if writing a program using this API would be just as awkward as writing an entire Java program using the reflection API…”

There are many other signs of implementation-focused design in the API. An out-of-process call is evident when IManagedObjectRef.getManagedObject() method throws a java.rmi.RemoteException. The object-relational mapping layer (Castor) is revealed when the class AttributeData inherits from org.exolab.castor.jdo.TimeStampable. Internal caching is obvious from methods like ContentType.clearCache(), while the underlying database schema is visible and accessible through methods like AttributeDefinitionData.getColumn(). With so many implementation details spilling out, we are left to wonder which part of our code will break when we upgrade to the next version of the API.

Designed for use versus designed to implement

The above issues may seem hard to avoid, but in practice they are not. While doing API usability tests at Microsoft, Jeffrey Stylos and his colleagues discovered a surprisingly easy way to do it. It happened almost by accident: on a few occasions, they asked the developers to solve simple programming tasks without giving them any specific APIs to use, instead they allowed them to make up the APIs they wanted to use. They were surprised by what they saw: when asked to send a simple text message using an unspecified messaging interface, all the developers wrote:

TextMessage msg = new TextMessage();

and not a single one of them wrote

MessageFactory factory = (MessageFactory) DirContext.lookup(“MessageFactory”);
TextMessage msg = factory.createTextMessage();

Given the opportunity to design their own graphics API, none of the developers wanted to write

Image.draw(false);

instead, they wanted to use two distinct methods, similar to these:

Image.overlay();  //draw over previous image
Image.draw();       //erase previous image before drawing

Boolean method parameters hardly ever figured in any of the APIs which developers were asking for. None of the developers thought they would need to do complex initialization steps; instead they assumed that the API will work right out-of-the box. Few of them expected that they will be required to extend classes, to implement interfaces, or to catch and handle exceptions. None of them wrote more than a few lines of code for the basic scenarios they were asked to implement. They just assumed that the API will take care of the details and hide the complexity from them.

The APIs the developers designed for themselves to use and someone else to implement were thus very different from the APIs they would design for themselves to implement and someone else to use. The conclusion is that the APIs we design are heavily influenced by our point of view. Let’s call these the caller’s point of view and the implementer’s point of view. Since the API is implemented only once but used many times, it should be clear that the caller’s point of view is dominant in API design.

Write client code first

So how do we design APIs from the caller point of view? By doing the exact same thing the developers were asked to do in the above experiment: writing the client code first. Not just once, but separately for every core usage scenario we want the API to cover. As we do this, repeating API usage patterns will emerge, as well as the types and methods which we need to provide. It helps when the developer writing these usage scenarios is not the same as the one implementing the API, to prevent any accidental “implementation bias”. It also helps if more than one person contributes code scenarios, so that personal preferences and programming style won’t have an undue influence on the API.

It is very important to point out that we are advocating writing real code for solving real problems as use cases, not pretend or throw-away code. The written code should be constantly updated and maintained as the API evolves and it should work correctly when the API is finally implemented. We don’t consider this wasted time, as this code can be reused for samples in the API documentation and as part of the API test suite. We should be skeptical about any API for which code samples are not readily available.

If an application has a graphical user interface, it is very common practice to model the API after the GUI, with method calls corresponding roughly to user actions, method parameters to user input, and results to what is displayed on the screen. This correctly reflects the user’s perspective and has nothing to do with the implementation, right? Well, the caller (programmer) and the user are not the same; they have drastically different needs. Issues with such APIs include:  insistence to log in with user name and password even when writing code for  unsupervised batch processes, excessive dependence on exception handling (the result of reusing existing input validation and error reporting logic), over-abundance of basic data types in method signatures (especially the string data type), data structures that have the same fields as forms displayed on the user’s screen, and so on. Such APIs are perhaps useful for developing alternative GUIs, but are less suitable for other scenarios.

The true test of our commitment to the “consider the caller perspective” guideline comes when we need to provide programmatic access to existing functionality. The implementation already exists. What are we going to do? Start coding scenarios and design an API from scratch? Or do we succumb to temptation, document the implementation we already have and call it an API? After all, any other API will require bridging, will introduce new bugs and may cause some performance problems. Why waste time on it?

If so many embarrassing (for the designer) and irritating (for the user) API design mistakes can be easily avoided by considering the viewpoint of the caller and writing the client code first, why is this not a common practice? Honestly, we don’t know the answer. The necessary time and effort certainly plays a role. Ultimately, just like in the case of User Experience Design, API Design is either part of an organization’s culture or it isn’t. One thing is for sure: considering the viewpoint of the caller is an essential part of any disciplined API design process.

This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.