I write a lot of APIs for a lot of different organisations. I also spend time working with developers with a wide range of skills sharing thoughts, techniques and practices. There’s always something new to learn and always a lively discussion to be had, as with so many areas of technology. I was asked to document some of my guiding principles and techniques for developing an API quickly and efficiently for use on a site that is supporting a data hack.
It was supposed to be a fairly straightforward couple of hours writing, but like so many things it grew and grew until it reached this stage some days and lots of effort later. It’s not exhaustive, it’s hardly extensive, but it might provide a useful starting point for someone, so I’m sharing.
Software development is part art part science. Like all good science, the techniques that we are discussing here have been tried and tested over multiple projects and have demonstrated a proven ability to meet the stated aims. Like all good art these techniques are far from the only way of achieving these aims.
What is an API?
An API defines a contract that can be relied upon to allow for data and / or services to be integrated with the program that we are developing.
There is no implied technology or location specified in an API and in that regard it can be considered synonymous with a software development kit (SDK), Application Development Framework and so on.
For our purposes, it is useful to be more prescriptive in the definition of an API. For this article we will take an API to specifically define a mechanism through which a consumer can access data or services hosted in a remote location.
Why would we want one?
An API says to the development community that you are open for business. There are many contentious assertions made in relation to the Internet to the extent that it can be difficult to get people to agree on anything. There are two assertions that seem to be universally acceptable:
- The Internet is constantly changing, morphing from one state to another without stopping.
- The rate at which the change is taking place is accelerating.
It’s impossible to keep up with the Internet. If Google, Microsoft and Apple can’t keep up then you probably can’t either.
You can improve your chances of continued success through collaboration. You can’t solve every challenge on the Internet and you risk becoming increasingly irrelevant as competitors or new entrants inexorably erode your market position. Presenting an API creates opportunities for partnership. Rather than exposing yourself to the risk of substitution, the API allows you to be part of the solution. An API positions you as an expert in a particular field; it is a statement that asserts ‘This is what we’re really good at’ and allows collaborators to build on your success and skills.
How do we build them?
The best APIs are the ones that we both share with external users and use ourselves. By using our own APIs we keep a vested interest in ensuring that they maintain the best qualities in terms of:
* Integration
* Consistency
* Stability
* Performance and Scalability
* Availability
Integration
APIs are a means for integration. When we’re developing our API we ant to make sure that the means to integrate is open to the widest possible audience. The API itself should not limit who or what can connect to it.
Consistency
You need to think about your API in the same way that you think about the user interface to your program. You are presenting the user with a mechanism that allows them to achieve a goal. We spend a lot of time designing interfaces that are clean, simple and efficient. In many regards the elegance of in interface is measured by both its simplicity and its consistency. Users are able to quickly develop a mental model of the interface that allows them to achieve their goals without having to second-guess every action.
The design of your API deserves the same level of diligence. While the data being passed around in the form of messages naturally changes, the mechanisms should be consistent. Once the developer knows how to get a collection of data or a single element of data, that same approach should work for all the other pieces of data – they shouldn’t have to learn new techniques for each operation.
Stability
Stability exists on a number of levels in regard to APIs. The first and most obvious is that the service being presented through the API is available when a call is made to the API. Those that crash unpredictably won’t attract users; actually neither do those that crash predictably.
The second and slightly subtler definition of stability relates to the definition of the API itself. An API that changes dramatically on a regular basis will quickly fall out of favour with the developers. The reliability of an API is often directly attributed to its volatility. If your API needs to change often questions will be asked about whether the service that it is providing can be relied upon. Questions will also be raised in relation to your own stability, skills, competence and mental faculties, which can be particularly challenging if your own development team is consuming your API.
Performance and Scalability
You’ve built a gorgeous API, its consistent, reliable, open, standards based and so on but only allows for five users at a time before it grinds to a halt. Net result? You only get five users. Building an API for scale is a matter of following some simple practices that ensure that when you’re suddenly sitting on top of the most popular API on the Internet that you can respond in a manner that ensures you don’t witness the greatest exodus since Moses parted the Red Sea.
Availability
Less to do with the code you write, but of clear importance when deciding how and where to host your API, is how available your API is to its consumers. Beautiful, consistent reliable, fast but only online for two hours on a Wednesday is equally not a desirable experience.
The availability of your API is a function of a number of elements including:
Hosting
Your API will have hosting requirements. It’s easy to insist that everything be hosted in the cloud, but the practicalities of this need to be considered when you’re starting, not when you’re deploying. If you have an API that relies heavily on some internal data systems, can you still offer the service on the cloud? If you can’t then what are your contingency plans for ensuring that sufficient capacity is available for traffic spikes? Where you host may not be entirely in your control. The specifications for your hosting are and need to be defined at the beginning.
Language / Platform
Your API’s availability is also determined by the platform you’re running it on and the language you have chosen to develop it in. Consider for a moment the Microsoft Windows platform: once a month Microsoft will release patches and updates to the platform. Occasionally these will be critical security patches that need to be applied immediately. Patching requires downtime and downtime impacts your availability.
Similarly language choice can have a material impact on the availability of your service. Our friends in the C/ C++ community have always taken something of an imperious tone to developers in ‘managed’ languages (seriously automatic reference counting and garbage collection – not the root of all evil). This recently lost some of its veneer when previously undiscovered holes were found in the wildly popular Open SSL libraries. Nothing will impact your availability like having to take an entire system down due to security concerns.
Approach
Armed as we now are with a firm resolve tempered with a healthy dose of reality / paranoia we look to begin our challenge of developing an API. Before we finally start to write the code let’s make sure we leave no stone unturned by thinking about how we’re going to approach the problem.
Outside In or Inside Out
The outside in / inside out question has numerous facets to it including structure of the team, nature of the technology, degree of integration with legacy or other systems required, time and delivery constraints and on and on… Sometimes it’s just a matter of taste or personal preference.
Lets qualify what we’re talking about: outside in says you define your API first and then build in to complete the offering. My more erudite colleagues will call this ‘Contract First’ or ‘Design by Contract’ but for our purposes it means I’m going to decide what you can do and how we’re going to pass information between us first, and then figure out how I’m going to make all that work.
Inside out in contrast says you build your core service and logic and finally wrap it in an API.
Personally I favour outside in. Here’s my reasoning, for better or worse. I see the API as an interface that is no less important than a visual interface as I’ve previously asserted. In my mind it demands the same level of thinking and design as a user interface. It is customer facing and should be customer focused. It’s a contract that should not be changed without significant thought and consideration. How that API gets implemented to deliver the service it exposes is of no concern to the consumers of the API so long as it meets their needs in relation to integration, consistency, stability and all those other virtues we’ve already discussed. With a nod to the Wizard of Oz, we don’t want anyone looking behind the curtain to see the magic.
Seek Collaboration
An API is a means of collaboration. Whether it’s a separation of duties and functionality for a development team in a single organisation, or a means to distribute your development efforts and teams globally the API is the gathering point for those efforts. In the interests of allowing teams to maximise their ability to develop in parallel, in my mind anyway, it’s important to get a shape on that gathering point. Once we’ve all agreed on what that looks like, you can race on ahead developing the client that uses the API while I work back into the core making it all work. Where we have an issue we come back together, adapt and move forward again.
For me it’s an indication of the mind-set of the API designer. If I’m open to collaboration early, and willing to consider alternatives to my ‘one true way’ then I’m immediately declaring myself open to conversation. In this way my API is a clear reflection of my core ethos of engaging and working together to build the best solution.
Conversely, I can decide to fix all of my own problems first, craft an innovative, technically perfect, glorious, awe inspiring service layer which I then wrap in an API that you should use to benefit from my magnificence.
In one mode I’m asking you what works for you, in the other I’m telling you what you need. Once again there’s no silver bullet, no completely correct path. Where the platform you’re offering is so leading edge that the only way to build understanding is by leading out on the API then absolutely do that, in the majority of cases though I would argue that an early engagement with your customers – those that will consume the API – will lead to a better API.
Regardless of what approach you take, you can be sure that any number of people will explain to you in some detail why you made the wrong choice. The important point is to pick an approach that you can defend and then start. The alternative guarantees that you’ll never release.
Bows and Arrows 
Leave a Reply