IX-API: Design notes and recollections
It’s been my great pleasure this year to work on one of the most interesting and challenging projects of my career — the international collaboration project between LINX, AMS-IX and DE-CIX to create and implement the IX-API specification.
In brief, the IX-API project is intended to provide a common, consistent, and easy-to-use API specification to order large-scale network connectivity from IXPs programatically, without the risks and delays of human-to-human interactions. If you’re not familiar with this area of technology, you could do worse than to consider this “AWS for Networks”.
Spoiler: It turns out that that’s quite difficult.
There are a number of reasons why trying to map and control the complexities of real-world networking through a restful API is a tricky process.
Restful APIs, after all, are neat, orthogonal things. They’re built of discrete, unrelated entities, with neat identities, and a clear, clean, Create-Retrieve-Update-Delete cycle.
Networks aren’t. They’re built of complex, interacting parts, which have to be assembled in specific ways and particular orders. You can’t necessarily just Create, Update or Delete an entity without creating or preparing other related entities first. For example if, in IX-API parlance, you want to create a Network Service Configuration, you’re going to need to know about its Network Service, Connection and Contacts. And to know about any of those entities, you’re going to need to know exactly what each of them is.
It turns out that one of the problems of this neat, technical and exact field of technology, is that we’re* a little inconsistent about what we call things. Words like “Port”, “Connection”, “Service” and “Device” get thrown around with great frequency, with meanings depending heavily on context and/or on shared expertise. That real-world imprecision doesn’t work too well for APIs, where each entity has to refer to a very exact type of Something.
To work around this, the project defined a very specific set of Entities. This happened well before I joined it, so I can’t claim too much credit, but what I did contribute here was the IX-API Glossary of Entities — a set of definition of each entity type, mapped to common usage and expanded definitions, to help both the networking layman such as myself, and the networking journeyman who needs to pin down exactly how each term is being used. I don’t think I’m giving away too many trade secrets when I reveal that I initially created that document as a mapping table in the LINX wiki, linking the “official” IX-API terms with internal LINX naming and database references. I’d encourage any IX aiming to implement the IX-API to do exactly the same thing.
To summarise the above issue, in terms that won’t surprise any developer, “Naming Things Is Hard”.
Having nailed down what each Thing is, however, the next step was to define their interactions. Again, a lot (albeit not all) of this happened before I joined the project, so I don’t claim a lot of the original credit. However, I was able to draw out and help codify some of the design decisions we took, and tried to stick to, in the development of the API specification.
- The API should be as easy as possible to implement, both as a provider and consumer, to minimise the barriers to entry.
- The API would not attempt to enforce best networking practice. If it was technically possible to run a network in a given way, even if would have other engineers scratching their heads, the API should allow it.
- Allowing for different IXPs to have different rules and configurations, the API should never fail to perform a task due to a specific IXP’s business rules, without a fair chance for the API client to discover those rules and requirements beforehand. Surprises were to be frowned upon.
- More generally, behaviours should be consistent within the API. REST compatibility should be maximised, and different endpoints should not have inconsistent behaviours or expectations.
- The API would define, and the partner IXPs implement, a specific, minimal use case in the first instance. Reaching a functional commercial level with this minimal use case would then give real-world feedback to inform further development.
These design goals each had specific impacts on the work done:
In terms of ease of implementation, we chose to define each API endpoint with minimal complete data. In other words, if you called a GET on a given entity, you would receive just enough data to explore the API to define that entity’s place in it. So, a “Network Service Config” (or Port Unit in LINX parlance) would come back with its speed, name etc, but not the names of it connection, owner etc — these would come back as IDs. If you then needed to find out details of these further relationships, you could fetch the given IDs from the appropriate endpoints. In some ways this is the opposite of a GraphQL mentality — you don’t get to ask the API server for multiple possible relationships in one call, so the API server’s work is simplified. The trade-off here is, of course, that you might have to do multiple calls to fully document an entity — but while this presents an increase in work, it does not present an increase in complexity. In the IX-API case, convenience does not trump simplicity.
There is however one exception to this “simplicity at all costs” mantra — because entities might return a list of IDs of a given type — eg a Network Service Config might have multiple addresses, each identified by IDs, one API call is allowed to present a list of IDs of a single type to return. This is consistent over all endpoints, and for my money represents a practical trade off for minimal server-side complexity.
As regards Network Best Practice, we recognised that different IXPs — or other providers of an IX-API server — might have different needs, resource levels, and technical designs. We also didn’t believe that three IXPs could or should define what was, or was not “OK” in network design. So you can have a Network Service Config with one, none, or a thousand addresses, but you can’t have an IPv4 address with 48 characters.
To facilitate these differing practices and requirements, areas known to be different between partner IXPs are documented by the servers themselves — so, for example when looking up a Network Feature (typically a Route Server that a port can be configured to use), the available modes and address classes, and the required contact information, are all specified in a way that the client can parse and present to the end user.
Strict REST compliance also proved tricky. The main problem here was that REST interfaces imply that once an item is created with an ID, that ID remains constant, always referring to exactly one thing, and when that ID-addressed entity is updated, it continues to refer to that same thing, which is modified instantly. Sadly, the real world is not that neat or orthogonal. If, for example, a customer requests a speed update on a port, this may well require the substitution of faster hardware, which cannot happen instantly. So a port has an ongoing state, a desired state, and a “pending” status, which all still have to be represented by the single, unchanged ID that the API originally provided. In particular, the “old” and “new” entities might well be considered within the IX’s management system as different entities, with differing IDs, in which case a “meta identity” needs to be persisted across the backend entities, with its relationship with those backend entities determining the state of the frontend entity. While I did briefly contemplate that an update could create a new entity and a new ID, I’m grateful to those who shot that particular idea down — it would have been grossly unrestful and therefore too “surprising” to consumers of the API. In this case, the API server really does have to swallow and support the complexity.
Summary: Mapping new, external data models onto internal, legacy ones is also difficult. To use the terms of “No Silver Bullet”, this is essential rather than accidental complexity; avoiding the complexity would be the wrong thing to do here.
On the last of the above points, I believe the decision on scope was also important, and correct. The “Use Case A” defined for initial delivery was quite limited, so one of the key questions I’ve seen asked about IX-API since its launch has been “Why didn’t you implement ‘X’”, where ‘X’ is typically pricing info. I have a few answers to this, all of which should be taken as my own and not as a policy of the IX-API group (not least because, since my relocation to the USA, I’m no longer involved in the project!)
- “No plan ever survives contact with the enemy”. IX-API is a very new idea, breaking a lot of new ground to perform a function that’s never really been available in the networking industry. The only way we were going to learn what worked was to implement it to the best of our judgement and make it live, and see what did, and did not work.
- “YAGNI”. By limiting what we specified and built, we avoided speculative design of parts of the API that might be proven to be invalid. Such a specification could well have created parts of the API which were legacy before even being released, and increased the weight of maintenance to no good end.
- “Build the ground floor first”. If there are multiple ways just to build a network, there are far more ways to charge for it. The pricing data for any instance of any type of service would depend on any amount of internal business logic specific to each IXP, which would in turn depend on the technical API being correct. In my mind it’s perfectly valid for a pricing API to be deferred — and personally I’m not even sure it would be that widely used. I don’t think, for example, that AWS’s pricing API is particularly heavily used — most projects I’ve worked on have estimated prices from human-readable data, and used alarms to monitor them, but never made decisions in code based on this answer.
- “The story continues”. IX-API is an evolving project. There are multiple phases and huge expansions planned. The initial IX-API partners did just enough to prove that the project worked; there is absolutely no belief that even the scoping of what is possible is complete.
Beyond the technical complexities, it’s also worth touching on the organisational difficulties of a project this size, executed in multiple nations simultaneously. We benefitted enormously by having some excellent engineers and supportive pilot consumers on all sides in this project, and we worked hard on communication, but this was still a huge, difficult and time-limited project. English was, by necessity, the primary language of the project, but even with the Dutch and German partners putting the English to shame linguistically, there was always the risk of subtleties being lost in a second language. Once I joined the project as the only British developer who could speak German, that dynamic changed a little. While most discussion was still in English, the ability to seek clarification and find nuances of design in another language occasionally proved invaluable. I’ve written elsewhere on the risks of the UK’s monoglot culture, but I would encourage any company embarking on an international project of this type: don’t make your partners do all the work linguistically. Find developers, engineers or project people who can reciprocate and have the discussions in the partner languages too. Not only does it improve the feeling of community in the project, it can save some pain too.
Finally, I want to re-iterate my thanks to the IXPs, partners and engineers on this project, and to encourage potential users and consumers of such APIs to get involved with the IX-API project.
Footnotes
- When I refer the networking industry as “we”, I’m probably stretching a little. I’m a software engineer with systems administration and lightweight networking experience. By no stretch of the imagination am I a Network Engineer.
- If you’re unfamiliar with IXPs or networking in general, the EUROIX video makes a great explainer.
Personal Note
As noted above, I’ve just relocated to the USA — to Boston, MA to be precise — and I’m looking for work. If the above has interested you, or you’re looking for a senior or multilingual (human or machine languages) software engineer, check out my portfolio and CV links at https://parsingphase.dev
