API Governance Powered by Machine Learning

Running a successful cloud software business is hard. Growing it is even harder. These days just making a brilliant software product is not good enough. The product must interconnect with other systems to be successful. Software businesses must transform platform businesses so they can form partnerships with other companies. Building and managing Application Programming Interface (APIs) platforms becomes an important part of growing a platform business because it is the language for integration. As the number of partners increases, keeping API usage monitored and governed becomes tricky. Losing governance can be quite risky. This article explores how a simple machine learning (ML) algorithm called cluster analysis can help you to build an autonomous system that detects shifts and pivots of API usage and makes it harder for integration partners to go rogue.

A hidden problem in all API platforms

If your company runs an API platform for some time, you probably have some form of monitoring in place to ensure the platform is stable. Whether it is a logging system that triggers certain alarms or a periodic usage report that operations engineers look at. It is easy to find out if a big usage problem such as a big spike of calls indicating issues at their end. But what if your partners silently changed their way of using the API because they had made a newer integration or pivoted on their use cases? These changes typically go undetected until some events in the platform forces it to take a closer look. 

Now this exposes a problem, someone can completely pivot their use case of your API from what they have originally proposed and verified on. In my experience, some of these pivots can cause lengthy investigation, long engagements, tough negotiation and potentially lead to shutting down on API usage. For example, a conversation partner of a CRM system started to use the API to churn customers off instead of onboarding them. A security addon that starts to use user data to sell security products and becomes a lead generation tool for other software vendors. An invoice tool started to use information they gathered to provide lending products to end-users, which really should be in a commercial relationship with the platform. 

API Governance – what it looks like

A poorly-governed platform will rely on the end user’s reporting, partner’s self-reporting or, in some cases, just waiting for incidents to happen to react passively. 

A well-governed API platform requires all partners to be transparent about their current use cases of the integration and any significant changes in the future. The ways to make sure platforms know about changes right now are usually quite heavy handed: 

1. Engineers to audit partner’s API usage periodically
2. Accounts / partner managers constantly check in with partners
3. Require all partners to report pivot of use cases of APIs

As you can see, if you want to keep a platform player honest and competing fairly, you will need plenty of human experts to enforce the rules. Is there a way to automate this process and make this more proactive than passive? 

First the experts, then the expert systems

What we want here is an AI expert system that helps to replace our human experts. But we need to take a closer look at what experts do and how they do it.

When an API partner causes an issue, who do you speak to first in your organisation? The developer relations team (if you have one), the support team, the API engineers, or the partner managers/sales? Whoever it may be, the first thing they will look for is what & how APIs they are called now, compared to the past to see if there is anything different then come up with analysis and hypotheses of what they are doing. So what kind of problem & problem solving technique is this?

Is it an anomaly detection problem? This is a really mature field already well implemented in signal processing, finance, manufacturing, networking, data mining and cyber security. It is the identification of rare items, events or observations that raise suspicions. It might be an anomaly-detection problem in the microscopic context. But if we take a more holistic look at the whole busy ecosystem again, we will find the real problem is that the usage profile changed from one type of partner to another. Clustering analysis in machine learning has been maturely tested and applied to problems such as customer segmentation and buying behaviour analysis.

Finding the API black sheep? Or putting similar ones together? 

Clustering analysis is a type of unsupervised classification that is used in machine learning to tell how similar or distant the datasets are from each other. We can use this technique to classify the use cases of our API partner and detect shifts in behaviours. 

To do this, we don’t need to inspect every single API call payload. We only need the high level information. Most platforms have logging systems to tell which partner called what API endpoint (including what method) and how many times they called. I call these the metadata of APIs, because they are general information of the API calls, not the exact content of them. 

Unsupervised learning does not mean that there is no human involvement required. The models or inputs still need to be finely tuned by human experts to make this work. 

An example of the process

Let’s pretend we are a CRM platform software, with a lot of partners connecting to us via 6 APIs methods. So for one partner, for a period of time – perhaps one month – you will get similar data to Table 1. 

Table 1 – Monthly API call count of partners

If we standardised the numbers calls to 1 (by using max-min method below), to eliminate the noise created by the volume of API call numbers, a single partner’s API usage profile looks like Picture 1. 

Picture 1 – Standardised API Usage Profile of Partner A

Clustering analysis in machine learning finds out how similar the profiles are for partners, and groups them into categories. 

Step 1 – Analyse in 2D 

Visualisation is a great tool to test the classification. As a first step, look at only two methods that represent two different use cases. This effectively reduces the complexity to two dimensions. For example, POST Contact and POST Invoice are two very different methods. The human expert should look at the categorisation and tweak the type of clustering ML algorithm used; or, change the number of clusters as inputs. Do this until the prediction is close to what makes sense. In Picture 2, we see that  2-dimensional analysis suggests there can be four categories of use cases. 

Picture 2 – 2D analysis suggests there are 4 categories of integrations

Step 2 – Analyse in 3D 

In the second step, we further increase the complexity to 3D to verify the model. Adding the POST Inventory call data into the set to visualise in 3D. Repeat the process, ask humans to categorise them, and use the ML clustering algorithm to do the same. Tweak it to align both. It is okay if the categorisation is different from the previous step; with more information added, they are expected to change. Picture 3 suggests there are now 5 categories of integrations. 

Picture 3 – 3D analysis suggests there are 5 categories of integrations

More steps – Analyse in higher dimensions 

Beyond 3D we can’t really visualise it. But we can still gradually add in more dimensions and tweak the ML clustering analysis model. As the dimension increases the categorisation will get more accurate. 

The End Result

As an end result you will get a list of category IDs (or group IDs in some libraries) to suggest which integration has similar use cases to the other ones.

Table 2 – possible output of a clustering analysis

Doing this on an ongoing basis. With a small server querying the API logging system, the system can automatically detect large shifts and pivots in the use cases of API partners. 

You can use this trigger to automate some flows. You can, for example, send out emails to your partners to enquire about the shifts in behaviour. Or you can use this opportunity to proactively up-sell your premium features in the APIs. 

Summary

Detecting changes in API use case pivots is labour intensive and requires human expertise. Using clustering analysis machine learning algorithms is a great way to increase the governance of API platforms at scale.