API Design meets Protocol Design

In my previous post I showed that software interfaces used in distributed applications have to satisfy two conditions:

1. should be remotely accessible
2. should be suitable for use in high latency environments which permit partial failures, have no shared state and no reliable synchronization mechanism

I also argued that technology alone – no matter which technology you choose – only helps you with the first requirement. The concepts, principles, and methods of protocol design can help you meet the second condition.

I’d understand if you were a bit surprised. Protocol design is indeed applicable because API and protocol are two alternative – but equally correct – ways of describing software interfaces. Wikipedia defines API as “… a particular set of rules and specifications that software programs can follow to communicate with each other” and protocol as “… a formal description of digital message formats and the rules for exchanging those messages in or between computing systems and in telecommunications.” We certainly tend to focus more on programming language constructs like methods with parameters when describing APIs and on processes exchanging messages when describing protocols, but ultimately these concepts describe, from a different point of view, how programs communicate with each other.

Consider the familiar real-time stock quote service as an example. We can describe it equally well by showing its API in a programming language (here, in Java):

Listing 1: Java client library interface

/**
* My sample real time stock quotes service
**/
public interface StockQuotes {

   /**
   * Returns a stock quote
   * @param ticker is the symbol of the stock
   * @return the current price of the stocks
   * @throws RemoteException if there is a communication error
   **/
   public float getQuote(String ticker) throws RemoteException;
}

or by showing the messages (requests) it accepts and replies it provides (in our case, a simple RESTful protocol)

Listing 2: REST request and reply

GET /quote/?ticker=AAPL HTTP/1.1
Host: StockQuotes.com
Accept: application/json

returns

HTTP/1.1 200 OK
Date: Fri, 17 Feb 2012 16:59:59 GMT
Content-Type: application/json
Content-Length: 29

{“ticker”:”AAPL”,
“price”:512.68}

The API description feels more natural when working via a client library while the description of the HTTP messages is useful for accessing the service from exotic environments (say Adobe Flash or SAP’s ABAP) for which no client library is provided. The same functionality could be also described as a SOAP Web Service with the following WSDL:

Listing 3: WSDL description

<?xml version="1.0"?>
<definitions name="StockQuoteServiceDefinitions">
   <!-- Namespace declarations omitted for brevity -->
   <message name="GetQuoteRequest">
      <part name="ticker" type="xsd:string"/>
   </message>
   <message name="GetQuoteResponse">
      <part name="price" type="xsd:float"/>
   </message>
   <portType name="StockQuotePortType">
      <operation name="getQuote">
         <input message="tns:GetQuoteRequest"/>
         <output message="tns:GetQuoteResponse"/>
      </operation>
   </portType>
   <binding name="DefaultBinding" type="tns:StockQuotePortType">
      <!-- Details of document literal binding style omitted for brevity -->
   </binding>
   <service name="StockQuoteService">
      <documentation>My sample real time stock quote service</documentation>
      <port name="StockQuotePort" binding="tns:DefaultBinding">
         <soap:address location="http://www.StockQuotes.com/quote/">
      </port>
   </service>
</definitions>

Frameworks like Microsoft’s WCF or Java’s JAX-WS can automatically generate a client library (called a binding) from this description. The generated code looks just like a regular API to the caller, but implements its functionality by making XML (SOAP) requests over HTTP to a server. It would be pointless to debate whether this WSDL describes an API or a protocol. To support automatically generated client bindings it has to describe both.

You might wonder: if API and protocol are equally appropriate for describing software interfaces, why is it that expressions like “Web Services API”, “SOAP API”, “RESTful API”, “Web API”, “Hypermedia API”, or  “Mobile API” are commonplace while few people ever talk about protocols? I don’t claim to know the answer. I can only tell you why I also frequently use the word API when I actually mean protocol.

I’ve noticed that the term protocol is very closely associated with ubiquitous low-level transport protocols like TCP/IP and I’m often concerned that it is not immediately obvious that I’m talking about high-level application protocols like SMTP, HTTP, Atom or LDAP. Since the latter are used just like APIs to build client applications for accessing email, browsing the web, reading news, or looking up someone’s phone number in a corporate directory, I simply call them APIs. It is technically correct and unlikely to be misunderstood. It is certainly more convenient than spelling out “application protocol” every time.

I also sometimes worry that developers may think I want them to write a ton of code, parse complex messages, and handle complex interaction patterns. If they had a college course on protocols, they were probably thought about message framing, byte stuffing, and many other low level details. This is not what I typically mean. Protocols are rarely built from scratch. They are layered on top of existing protocols, extend existing protocols, or constrain (further specify) existing protocols. Plus we have all the various tools. I use the word API in casual conversations to convey a subtle “this is not as hard as you might think” message.

Just to be clear, I’m not saying that calling your remote interface an API is a mistake. It is not. My point is that thinking of it as an application protocol is helpful, especially when you are designing it. It is helpful because it forces you to take the differences between local and remote communication seriously.

When I think in terms of requests and replies instead of function calls, it is hard for me to ignore latency or the possibility of partial failures. When I think in terms of data sent as messages instead of objects, I have no expectations of shared state. When I think in terms of independent processes, I know that there are no reliable methods of synchronizing them. When I think in a language-neutral way, it is a lot less likely that I assume language features which may not be available to some clients or may not work remotely.

The protocol viewpoint also helps me maintain realistic – some may even say skeptical – expectations about tools and technologies I’m using. I rely on them to take care of low-level protocol details and automatically generate repetitive, boilerplate code. I accept, however, that pretty much everything else is my responsibility.

In my next post I’ll talk about how I apply protocol design concepts, principles, and methods to the concrete case of RESTful design and how this helps me understand – and chose from – the many contradictory advice that are out there.

This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.