Anticipating evolution

API designers need to resolve an apparent paradox: how to keep APIs virtually unchanged yet respond to ever-changing customer requirements. It is more intricate a skill than simply applying specific API evolution techniques. It can be compared to a chess master’s ability to anticipate several upcoming moves of a game. Just like beginner chess players, we start by learning the specific API evolution techniques, but we become true experts when we are able to plan ahead for at least a couple of API releases. We are more likely to design long-lasting, successful APIs if we master this skill.

Let’s start with the fundamental rule of API evolution: existing clients must work with a new product release without any changes, not even a recompilation. While breaking changes can be tolerated in internal code, they are prohibited in public APIs. We must either limit ourselves to binary compatible changes or keep the old API unchanged while introducing a new API in parallel, a method called API versioning.

Maintaining backwards compatibility

Backwards compatible changes are preferable because clients can upgrade smoothly and without any human intervention, taking advantage of new features at their convenience. Conversely, API versioning demands an explicit decision to upgrade because code changes are required. Clients frequently choose to defer upgrades, requiring a long period of support for multiple API versions. We should plan to evolve APIs primarily through backwards compatible changes. We should avoid API versioning if possible.

Anticipating evolution means choosing designs which allow the largest number of backwards compatible changes. For example, C++ developers know that adding a field to a C++ class changes its size and breaks binary compatibility with client code into which size was hard-coded by the compiler. Similarly, adding a virtual method modifies the virtual method table, causing clients to call wrong virtual functions (see Listing 1). Because the need for new fields and methods is likely to arise, smart designers move all fields and virtual methods into a hidden implementation class (see Listing 3), leaving only public methods and a single private pointer in the public class (see Listing 2):

Listing 1: Original API class design is hard to evolve

#include <vector>  //exposed direct dependency on STL
#include "Node.h"  //exposed implementation class Node
class OriginalClass {

public:
	int PublicMethod(...);

protected:
	std::vector<Node> children; 

	// Adding a field modifies the size, breaks compatibility
	int count; 

	// Adding a method modifies the vtable, breaks compatibility
	virtual void ProtectedMethod(...);
};

Listing 2: New API class design using the Façade pattern

class ImplementationClass; //declares unknown implementation class

class FacadeClass {

public:
	int PublicMethod(...); 

private:
	ImplementationClass *implementation; //size of a pointer
};

Listing 3: The implementation details are never exposed to the client

#include <vector>  //OK, client code never includes it
#include "Node.h"  //OK, client code never includes it

class ImplementationClass {

public:
	int PublicMethod(...);

protected:
	std::vector<Node> children; 

	//OK, client never instanciates direcly
	int count; 

	//OK, the client has no direct accesses to the vtable
	virtual void ProtectedMethod(...);
};

Binary compatible changes are different depending on platform. Adding a private field or a virtual method is a breaking change in C++, but a backwards compatible change in Java. As one of our teams recently discovered, extending SOAP Web Services by adding an optional field is a compatible change in JAX-WS (Java) but a breaking change in .Net. Providing lists of compatible changes for each platform is outside the scope of this document; this information can be found on the Internet. For example, the Java Language Specification states the binary compatibility requirements and Eclipse.org gives practical advice on maintaining binary compatibility in Java. The KDE TechBase is a good starting point for developers interested in C++ binary compatibility.

While we are comparing platforms, we should mention that standard C is preferable to C++ for API development. Unlike C, C++ does not have a standard Application Binary Interface (ABI). As a result, evolving multi-platform C++ APIs while maintaining binary compatibility can be particularly challenging.

Keeping APIs small and hiding implementation details help maintain backwards compatibility. The less we expose to the clients, the better. Unfortunately, compatibility requirements also extend to implementation details inadvertently leaked into the API. If this happens, we cannot modify the implementation without using API versioning. Carefully hiding implementation details prevents this problem.

We can break backwards compatibility (without modifying method signatures) by changing the behavior. For example, if a method always returned a valid object and it is modified so that it may also return null, we can reasonably expect that some clients will fail. Maintaining the functional compatibility of APIs is a crucial requirement, one that requires even more care and planning than maintaining binary compatibility.

The only backwards compatible behavior changes are weakened preconditions or strengthened postconditions. Think of it as a contractual agreement. Preconditions specify what we ask from the client. We may ask for less, but not more. Postconditions specify what we agreed to provide. We may provide more, but not less. For example, exceptions are preconditions (we expect clients to handle them). It is not allowed to throw new exceptions from existing methods. If a method is an accessor, a part of its postcondition is a guarantee that the method does not change internal state. We cannot convert accessors into mutators without breaking the clients. The invariant is part of the method’s postcondition and should only be strengthened.

API behavior changes are likely to go undetected since developers working with implementation code often do not realize the full impact of their modifications. When we talked about specifying behavior, we already noted the importance of explicitly stating the preconditions, postconditions and invariants, as well as providing automated tests for detecting inadvertent modifications. Now we see that those same practices also help maintain functional compatibility as the API evolves.

SPIs (Service Provider Interfaces) evolve quite differently from APIs because responsibilities of the client and the SPI implementation are often reversed. APIs provide functionality to clients, while SPIs define frameworks into which clients integrate. Clients usually call methods defined in APIs, but often implement methods defined in SPIs. We can add a new method to an interface without breaking APIs, but not without breaking SPIs. The way pre- and postconditions can evolve is often reversed in SPIs. We can strengthen preconditions (this is what the SPI guarantees) and weaken postconditions (this is what we ask from the client to provide) without breaking clients. The differences between APIs and SPIs are not always clear. Adding simple callback interfaces will not turn APIs into SPIs, but callbacks evolve like SPI interfaces.

Surprisingly, we need to worry less about source compatibility, which requires that clients compile without code changes. While binary and source compatibility do not fully overlap, all but a few binary compatible changes are also source compatible. Examples of exceptions are adding a class to a package or a method to a class in Java. These are binary compatible changes, but if the client imports the whole package and also references a class with the same name from another package, compilation fails due to name collision. If a derived class declares a method with the same name as a method added to the base class, we have a similar problem. Source incompatibility issues are rare with binary-compatible APIs and require few changes in client code.

If we focus too much on source compatibility, we increase the risk of breaking binary compatibility since not all source compatible changes are binary compatible. For example, if we change a parameter type from HashMap (derived type) to Map (base type), the client code still compiles. However, when attempting to run an old client, the Java runtime looks for the old method signature and it cannot find it. The risk of breaking binary compatibility is real because during their day-to-day work, developers are more concerned about breaking the build than about maintaining binary compatibility.

Versioning

API versioning cannot be completely avoided. Some unanticipated requirements are impossible to implement using backwards compatible changes. Software technologies we depend on do not always evolve in a backwards compatible fashion (just ask any Visual Basic developer). Also, API quality may also degrade over time if our design choices are restricted to backwards compatible changes. From time to time, we need to make major changes in order to upgrade, restructure, or improve APIs. Versioning is a legitimate method of evolving APIs, but it needs to be used sparingly since it demands more work from both clients and API developers.

Anticipating evolution in the case of explicit versioning means ensuring that an incompatible API version is also a major API version. We should deliberately plan for it to avoid being forced by unexpected compatibility issues. The upgrade effort must be made worthwhile for clients by including valuable new functionality. We should also use this opportunity to make all breaking changes needed to ensure smooth backwards compatible evolution over the several following releases.

API versions must coexist at runtime. How we accomplish this is platform-dependent. Where available, we should use the built-in versioning capabilities; .Net assemblies have them and so does OSGi in Java, although OSGi is not officially part of the Java platform. If there is no built-in versioning support, the two API versions should reside in different namespaces, to permit the same type and method names in both versions. The old version keeps the original namespace while the new version has a namespace with an added version identifier. The API versions should also be packaged into separate dynamic link libraries, assemblies, or archives. Since C does not support namespaces, separate DLLs are needed to keep the same method names. We should make sure we change the service end point (URL) when versioning Web Services APIs, since all traffic goes through the same HTTP port. We should also change the XML namespace used in the WSDL. This ensures that client stubs generated from different WSDL versions can coexist with each other, each in its namespace.

It is often advantageous to re-implement the old API version using the new one. Keeping two distinct implementations means code bloat and increased maintenance effort for years. If the new API version is functionally equivalent to the old one, implementing a thin adaptor layer should not require much coding and testing. As an added benefit, the old API can take advantage of some of the improvements in the new code, such as bug fixes and performance optimizations.

Conclusion

Designing for evolution can be challenging and time consuming. It adds additional constraints to API design which frequently conflict with other design requirements. It is essentially a “pay now versus pay later” alternative. We can spend some effort up front designing easy-to-evolve APIs or we can spend more effort later when we need to evolve the API. Nobody can reasonably predict how an API is likely to evolve; hence nobody can claim with authority that one approach is better than the other. It is thought provoking, however, that nobody has yet come forward saying they regretted making APIs easier to evolve.

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.

Making it safe

Being safe means avoiding the risk of pain, injury, or material loss. A safety feature is a design element added to prevent inadvertent misuse of dangerous equipment. For example, one pin of the North American electric plug is intentionally wider to prevent incorrect insertion into a socket. But it was Toyota who first generalized the principle of poka-yoke (“mistake avoidance”), making it an essential part of its world-renowned manufacturing process. When similar principles of preventing, avoiding, or correcting human errors are applied to API design, the number of software defects is reduced and programmer productivity improves. Rico Mariani calls this the “pit of success”:

The Pit of Success: in stark contrast to a summit, a peak, or a journey across a desert to find victory through many trials and surprises, we want our customers to simply fall into winning practices by using our platform and frameworks. To the extent that we make it easy to get into trouble we fail.

Preventing unsafe use

Engineers place all dangerous equipment and materials – high voltage, extreme temperature, or poisonous chemicals – safely behind locked doors or inside sealed casings. Programming languages offer controlled access to classes and methods, but time and again we forget to utilize it. We leave public implementation classes in the API package. We forget to declare methods users shouldn’t call as private. We rarely disallow class construction, and seldom declare classes we don’t want callers to extend as final. We declare public interfaces even when we cannot safely accept implementations other than our own. These oversights are the equivalent of leaving the boiler room unlocked. When inadvertent access to implementation details is possible, accidents are likely to happen.

Our next line of defense is type checking. In a nutshell, type checking attempts to catch programming mistakes at the language level, either at compile time in statically typed languages, or at run time in dynamically typed languages. If interested in the details of what type checking can or cannot do for you in various languages, you should read Chris Smith’s excellent “What to know before debating type systems”. For various theoretical and practical reasons, type checking cannot catch all usage errors. It would be ideal if every statically typed API call which compiles executed safely, but present-day compilers are just not sophisticated enough to make it a reality. However, this does not mean that we should not take advantage of type checks where we can. We may be stating the obvious, yet we often see APIs which are not as type safe as they could be. The ObjectOutputStream class from the Java I/O library declares the

final void writeObject(Object obj) throws IOException

method which throws an exception if the argument is not Serializable. The alternative method signature

public final void writeObject(Serializable obj) throws IOException

could turn this runtime verification into a compile time check.

Every time a method only works for a small subset of all possible parameter values we can make it safer by introducing a more restrictive (read: safer) parameter type. Especially string, integer, or map parameter types deserve close examination because we often use these versatile types unsafely in programming. We take advantage of the fact that practically every other type can be converted into a string or represented as a map, and integers can be many more things than just numbers. This may be reasonable or even necessary in implementation code where we often need to call low-level library functions and where we control both caller and callee. APIs are, yet again, special. API safety is very important and we need to consider design trade-offs accordingly.

When evaluating design trade-offs it helps to understand that we are advocating replacing method preconditions with type invariants. This moves all safety-related program logic into a single location, the new type implementation, and relies on automatic type checking to ensure API safety everywhere else. If it removes strong and complex preconditions from multiple methods it is more likely to be worth the effort and additional complexity. For example, we recommend passing URLs as URL objects instead of strings. Many programming languages offer a built-in URL type; precisely because the rules governing what strings are valid URLs are complicated. The obvious trade-off is that callers need to construct an URL object when the URL is available as a string.

Weighing type safety against complexity is a lot like comparing apples and oranges: we must rely on our intuition, use common sense, and get lots of user feedback.  It is worth remembering that API complexity is measured from the perspective of the caller. It is difficult to tell how much the introduction of a custom type increases complexity without writing code for the use cases. Some use cases may become more complex while others may stay the same or even become simpler. In the case of the URL object, handling string URLs is more complex, but returning to a previously visited URL is roughly the same if we keep URL objects in the history list. Using URL objects result in simpler use cases for clients that build URLs from fragments or validate URLs independently from accessing the resource they refer to.

As a third and final line of defense – since type checking alone cannot always guarantee safe execution – all remaining preconditions need to be verified at run time. Very, very rarely performance considerations may dictate that we forgo such runtime checks in low-level APIs, but such cases are the exceptions. In most cases, returning incorrect results, failing with obscure internal errors, or corrupting persisted data is unacceptable API behavior. Errors resulting from incorrect usage (violated preconditions) should be clearly differentiated from those caused by internal problems and should contain messages clearly describing the mistake made by the caller. That a call caused an internal SQL error is not considered a helpful error message.

We should be particularly careful when providing classes for extension because inheritance breaks encapsulation. What does this mean? Protected methods are not a problem. Their safety can be ensured the same way as for public methods. Much bigger issues arise when we allow derived classes to override methods. Overriding is risky because callers may observe inconsistent state from within the method they override (known as the “fragile base class problem”) or may make inconsistent updates (known as the “broken contract problem”). In other words, calling otherwise safe public or protected methods from within overridden methods may be unsafe. There is no language mechanism to prevent access to public and protected methods from within overridden methods, so we often need to add additional runtime checks as illustrated below:

public Job {

   private cancelling = false;

   public void cancel() {
      ...
      cancelling =  true;
      onCancel();
      cancelling = false;
      ...
    }

    //Override this to provide custom cleanup when cancelling
    protected void onCancel() {
    }

    public void execute() {
      if(cancelling) throw IllegalStateException(“Forbidden call to
         execute() from onCancel()”);
      ...
    }
}

It is generally safer to avoid designing for class extension if it is possible. Unfortunately, simple callbacks may also expose similar safety issues, though only public methods are accessible from callbacks. In the example above, the runtime check is still needed after we make onCancel() a callback, since execute() is a public method.

Preventing data corruption

A method can only be considered safe if it preserves the invariant and prevents the caller from making inconsistent changes to internal data. The importance of preserving invariants cannot be overstated. Not long ago, a customer who used the LDAP interface to update their ADS directory reported an issue with one of our products. Occasionally the application became sluggish and consumed a lot of CPU cycles for no apparent reason. After lengthy investigations, we discovered that the customer inadvertently corrupted the directory by making an ADS group a child of its own. We fixed the issue by adding specific runtime checks to our application, but wouldn’t it be safer if the LDAP API didn’t allow you to corrupt the directory in the first place? The Windows administration tools don’t allow this, but since the LDAP interface does, applications still need to watch out for infinite recursions in the group hierarchy.

The invariant must be preserved even when methods fail. In the absence of explicit transaction support, all API calls are assumed atomic. When a call fails, no noticeable side effects are expected.

Special care must be taken when storing references to client side objects internally, as well as when returning internal object references to the client. The client code can unexpectedly modify these objects at any time, creating an invisible and particularly unsafe dependency between the client code (which we ignore) and the internal API implementation (which the client ignores). On the other hand, it is safe to store and return references to immutable objects.

If the object is mutable, it is a great deal safer to make defensive copies before storing or returning it rather than relying on the caller to do it for us. The submit() method in the example below makes defensive copies of jobs before placing them into its asynchronous execution queue, which makes it hard to misuse:

JobManager    jobManager  = ...; //initializing
Job           job = jobManager.createJob(new QueryJob());      

//adding parameters to the job
job.addParameter("query.sql", "select * from users");
job.addParameter("query.dal.connection", "hr_db");      

jobManager.submit(job); //submitting a COPY of the job to the queue      

job.addParameter("query.sql", "select * from locations"); //it is safe!
jobManager.submit(job) //submitting a SECOND job!

For the same reason, we should also avoid methods with “out” or “in-out” parameters in APIs, since they directly modify objects declared in client code. Such parameters frequently force the caller to make defensive copies of the objects prior to the method call. The .Net Socket.Select() method usage pattern shown bellow made Michi Henning frustrated enough to complain about it in his “API Design Matters“:

ArrayList readList = ...;   // Creating sockets to monitor for reading
ArrayList writeList = ...;  // Creating sockets to monitor for writing
ArrayList errorList;        // Sockets to monitor for errors.

while(!done) {

    SocketList readReady  = readList.Clone();  //making defensive copy
    SocketList writeReady = writeList.Clone(); //making defensive copy
    SocketList errorList  = readList.Clone();  //making defensive copy

    Socket.Select(readReady, writeReady, errorList, 10000);
         // readReady, writeReady, errorList were modified!
    …
}

Finally, APIs should be safe to use in multi-threaded code. Sidestepping the issue with a “this API is not thread safe” comment is no longer acceptable. APIs should be either fully re-entrant (all public methods are safe to call from multiple threads), or each thread should be able to construct its own instances to call. Making all methods thread safe may not be the best option if the API maintains state because deadlocks and race conditions are often difficult to avoid. In addition, performance may be reduced waiting for access to shared data. A combination of re-entrant methods and individual object instances may be needed for larger APIs, as exemplified by the Java Messaging Service (JMS) API, where ConnectionFactory and Connection support concurrent access, while Session does not.

Conclusion

Safety has long been neglected in programming in favor of expressive power and performance. Programmers were considered professionals, expected to be competent enough to avoid traps, and smart enough to figure out the causes of obscure failures. Programming languages like C or C++ are inherently unsafe because they permit direct memory access. Any C API call – no matter how carefully designed – may fail if memory is corrupted. However, the popularity and wide scale adoption of Java and .Net clearly signals a change. It appears that developers are demanding safer programming environments. Let’s join this emerging trend by making our APIs safer to use!

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 Canada License.