DevOps teams rush to deliver their applications to production. Data & information spreads fast in modern organizations. Teams push hard to eliminate manual processes and replace them with automated scripts. One of the most powerful ways to make this a reality is to use APIs for every application (component) which talks to each other. In this post, I will highlight why APIs can’t be missed when it comes to DevOps.
What are APIs?
API stands for Application Programming Interface. An API lets you communicate with an application based on machine-readable code (for example JSON or XML). JSON and XML are structured in a predefined and standardized format. An API endpoint that acts as a receiver can then make sure every consumer uses the API in the same way. Use APIs when you have (a lot of) repeated processes that need to execute multiple times and every time in the same manner.
Some examples of when to use an API over manual processes are:
- Deploy Cloud resources (applications and other infra components) programmatically. For example to provision test environments.
- Onboard a user to an application: send the personal details of the intended user to the API which handles the request. A predefined set of response codes is sent back to the client.
- Use APIs to fetch a list of vulnerabilities for a certain application. Don’t click and point in a Graphical User Interface but use the API to get this data and process it. Base on this, you can, for example, generate a top 10 new vulnerabilities every week.
Nowadays, modern programming languages and other technologies use REST or SOAP-based APIs. REST almost always uses HTTP whereas SOAP is a protocol over HTTP. REST becomes very popular nowadays, most cloud providers use REST. Find out the differences (and similarities) of REST and SOAP at the website of restfullapi.
Organizations that practice an API-first strategy help everyone to treat APIs as first-class citizens. Especially, developer teams benefit from it to automate every stage in their application development lifecycle. Manual processes which exist and deviate across departments and other developer teams become a thing of the past. API first helps to enable automation, save costs, operate more efficiently, and become more consistent in everything you do. Without APIs, frequent deployments are not an option. In DevOps, APIs are an absolute necessity to push frequent releases. The business really needs it otherwise becomes out of work soon.
API first benefits
One of the most powerful benefits of API first is that development teams can work in parallel. Considering development capacity as scarce and time to market as crucial, this is a very important benefit. APIs require contracts between services. The contract enforces both creators (producers) of APIs and consumers on technical standards. Having multiple versions of an API speeds up the roll-out of new features for fast performing teams whereas less mature teams can still use the older versions of the API.
Another benefit of API first is testing. DevOps teams can test (their) APIs by creating mock methods. This mimics an API from a functional perspective. Teams can already write tests while the actual implementation of the API is still in progress. Early feedback flows from the consumers to the producers. This can be done automatically by digging into the logging and monitoring data of the API itself or using regular sprint reviews. Once the full implementation of the logic behind the API is finished, the mock function is replaced by the actual business implementation.
Relationship of APIs and DevOps
Why do companies focus on APIs when it comes to DevOps? Simple: because otherwise, you can’t properly practice it in your organization. Since DevOps is all about reducing manual efforts and automating, you can’t leave out using APIs in your daily operations. Data between applications needs to be exchanged using APIs, tool vendors require the usage of APIs to integrate with them. Besides this, every cloud provider mandates the usage of an API in some sort of way to automate whatever you want to do in the cloud. Every CI/CD pipeline requires APIs to integrate with the tools which are called in those pipelines.
Without APIs, things become hard (if not impossible) to automate, difficult to increase efficiency and maintain consistent processes. Manual processes can’t be repeated in a similar way every time they are executed. So they should be avoided as much as possible.
Let’s focus a bit more on these aspects given the following examples below:
In a data-first based organization, data is the most important asset (next to human resources). Applications collect and generate a huge amount of data. This data needs to be interpreted (consumed) by other teams to extract useful information out of it. Reports are useful assets to make this happen. Consumers do not manually login to the reporting app which aggregates this data. They do not search for the needle in the haystack themselves. Instead, they write a script or a fully-fledged application that calls the APIs of the producer application to fetch the data they need. It’s obvious they do not need to talk to the reporting team to gather and handle the data manually. They only need to adhere to the API contract(s) to fetch the data they need. And no more than that.
DevOps teams design, build, deploy and run their own Cloud resources. APIs can help to set up the correct amount of cloud resources based on what is needed at a specific point in time. For example: when you connect your cloud infrastructure pipelines to your monitoring solution, you can trigger a CI/CD pipeline to provision another Virtual Machine to spread the traffic in case of high loads. There is no manual intervention of the DevOps teams anymore. If the previous situation happens during off-hours, an automatic, API-driven architecture is much faster and cheaper compared to having someone on-call.
APIs help to automate recurring tasks. Often, developers and testers have to run things more than twice. A lazy but highly efficient IT professional uses the phrase: if you have to do a task twice, automate it.
APIs help to automate stuff and make processes repeatable. Consider someone finds a bug. Being able to reproduce the scenario exactly as it happened, helps to pinpoint the actual problem. Less human efforts create reliable and predictable results. Thus the quality of your IT product improves.
Business continuity and disaster recovery
Business continuity is one of the core principles for a company to survive a disaster. Have you ever followed a disaster recovery plan which includes a list of steps to recreate the situation before the interruption happened? If the majority of these steps is a manual process, it takes long before your systems are up again.
It can even hamper a proper recovery since in the stress to get things back online, you can make (silly) mistakes. Since everyone might be stressed, the results of a mistake can be more problematic than ever. Standardization on disaster recovery and the setup of replicated machines that utilize APIs help to keep downtime and data loss to a minimum.
Developers who are familiar with Object-Oriented programming languages such as Java understand why it is so important to structure their code in a proper way. High cohesion and loosely coupling are important design principles.
Simply said, high cohesion means that each feature is specialized in a single task and does that task very well. Loose coupling means no tight dependencies between different pieces of source code such as Java classes.
Both design principles also apply when it comes to the usage of APIs: every API endpoint should be dedicated to a single feature and provide appropriate support for DevOps teams to use it. Loosely coupling of (cloud) applications that expose APIs endpoints help to make sure they can be maintained separately. Interfaces between different applications need to be maintained independently of each other. This way developers are much more flexible to release new versions of their applications. There is no interference of other applications.
From a developers’ perspective, APIs are as important as any other aspect of an application. API first underscores this.
APIs for business and IT leads
Business leaders are advised to embrace an API first strategy and even push this as the only way of working for applications to interact with each other. For new applications, this is a must-have. Sometimes, legacy applications don’t support standard APIs. Business leaders need to think of ways to modernize those applications: refactor or rebuild from scratch to fully support the new way of working. Even when adding a proper API interface to a legacy application can help to increase the quality of such products. An API helps to access data through the application in a standardized way. It protects the data itself from improper use.
Access control in confluence
Confluence is a tool for DevOps teams to document their IT products, procedures, share architecture diagrams, etc. A lot of organizations use it.
Suppose the Product Owners or IT leads control access rights to a certain space (section) in Confluence. A space admin controls who has access to this space. Some spaces have pages that are internal (team-specific) spaces. Access control can be controlled using the GUI of confluence. However, it would be much better to control it by using the API of confluence. The first team which uses the Confluence API to control access can create a set of REST calls and store it in a public Git repository. Other teams can then re-use that to control their own space. If you want to revoke access to private pages: run a CI/CD pipeline to remove the user in charge from the access control list automatically.
Without using an API for this case, you would need to main access controls based on every single page in Confluence. You don’t have a clear overview since every page is treated on its own. Audits become much easier to get a clear overview of who can access which page.
Rolling out the strategy
Once business leaders are convinced of the benefits of an API first strategy, it’s time to make things practical. The following steps help to implement the strategy and promote this across all of the departments of the organization.
Key services and persons of interest
Answer the question: which key services should require an API. Focus on the most lucrative applications first. For example: estimate how many consumers will use the API, how many integration points the application will have, etc. Take your time to do this. Without this business tactic in place, it’s no use to proceed.
Stakeholder management becomes important to get support. You need everyone within the organization since you want to promote API first. Identify key stakeholders and strong leaders to define and promote the vision. Link it to practical examples and use cases to strengthen your message.
Build an API portfolio to support the entire organization. Don’t focus on specific departments only. Sometimes organizations are so big, different departments don’t even know from each other they are working on the same thing. Force every team which creates a new API to register it in the portfolio. Otherwise, your portfolio is not complete. Add functional descriptions and potential use cases, otherwise, you just have a dump-list that cannot be understood by the business representatives.
Successful companies such as PayPal have created an API portal by developers, for developers. This portal is the one-stop-shop for anything you want to know about APIs within the organization. From a business point of view, create a dashboard to monitor the usage of all APIs. Popular APIs which get a lot of traffic might give an indication of the quality of it or the number of integration points. This helps developers filter out what they need and it helps to conform to the principle of reuse.
Be consistent with API contracts. This becomes more important by the increased number of APIs you intend to create. Document all APIs and make sure developers can find them. Let them work consistently and standardize on naming conventions, endpoint names, etc. Enforce the usage of standards from the beginning.
Based on the previous point, create a style guide that is to be followed by the teams which create APIs for their services. A style guide includes: how to handle versioning, error handling, API status codes, etc. Swagger can help to define the style guide. Versioning also includes the versioning scheme as well as how to handle backward-compatible interfaces.
The same message as before applies here: automate as much as you can. For example, generate API-based documentation (to speed up and to have it aligned with the actual implementation), style validation (don’t forget the quality gates in your CI/CD pipelines), API mocking methods.
APIs help DevOps practitioners to speed up automation and delivery of business features. It has clear benefits over traditional ways of data integration (patterns). When using cloud technology, APIs are a must-have. Strong API governance is important as well as buy-in from the senior management. To implement an API first strategy, focus on key persons of interest, technical aspects as well as governance-related challenges. I hope this article helped you to adopt an API first approach in your organization to further improve your DevOps initiatives.