How to Avoid this Type of Technical Debt: Your Unmanaged 3rd Party APIs

Sometimes it feels like an organization's backend is like the wild-wild-west of APIs. Or the dark side of the moon.

As businesses grow, integrating with SaaS applications using APIs becomes an essential part of their application development. But companies fail to understand that consuming APIs requires continuous management. We’ve talked to hundreds of tech leaders that tackle continuous issues with their consumed APIs post-integration. They implement caching, throttling, load balancing or other mechanisms trying to build up resilience. 

Here are some of the most common statements we’ve compiled. Blink once if you agree, or blink twice if you don't:

•  “It’s a one time thing. We’ve solved it once, so it won’t be a continuous issue.”
•  “Our problem is unique to our business needs, it can’t be duplicated for future integrations.”
• “API breaking changes are inevitable. What can we do about it beforehand?”

Overtime, these optimization or maintenance needs become a can kicked down the road, and to put  plainly - external API consumption, when left unmanaged, can lead to various technical debts that can weigh down the efficiency of engineering operations.

In this post, I will break down why organizations should avoid neglecting this technical debt. Then, we'll show how to reduce this debt to improve API performance and, as a result, your productivity.

What Happens When We Fail to Plan for the Day After Integration

1. Recurring maintenance 

Engineering teams tend to have an “integrate now, maintain later” approach to consuming third party APIs.

But in short, it’s the maintenance work that keeps piling up in every sprint and release. Issues like caching, rate limits, load balancing, throttling and others creep up in the outgoing traffic due to unmanaged integrations and data consumption. Some issues can’t be expected or prepared for, while others are easy to control. But the work engineers are doing to maintain the integrations seems to be never ending, while becoming more complex  and exponentially growing.

Take for example scenario #1 of a hyper-growth company: when companies are consuming APIs on behalf of their clients, they need to manage multiple services and apply business logic on several instances with the same configurations, and they need to build resilience mechanisms to continue delivering their own SLAs to clients. This is when management of consumed APIs turns into urgent prioritized Jira tickets. 

2. Unexpected behavior in production 

Seamless API functionality is dependent on a successful third-party integration. This relies deeply on business requirements and logic, and is performed in a way that ensures responses are handled, tested, debugged, and deployed in production. 

But then you might start getting feedback from CS/ business stakeholders on the API performance. This is where theory (development) meets reality (production). At this point, those external APIs are running as part of your product. Now that those APIs are consumed in large volumes by your customers or those APIs have downtime/ breaking changes/ unexpected behavior/ exceeding costs - this is when you and your dev team are becoming aware of the changes that need to be made. Which brings us to…

Symptoms of Third-party API Maintenance Technical Debts

1. Beware of downtime, folks. 

First, APIs can frequently suffer from downtime, which can cause interruptions to your app's functionality. When a third-party API becomes unavailable, it can render your integration unusable and may even affect other areas of your product, depending on the nature of your connections. As your number of integrations grows, the likelihood of experiencing downtime on a regular basis increases.

2. Tokens, lots of tokens. 

Second, there’s the need to manage API tokens. This includes storing tokens securely to avoid cyber risks and compromise, allocating the token quota across the different consumers/ services are tedious tasks, renewing expired tokens, managing access control and assigning to different users, revoking tokens when they are not in use, rotating them for security means and overall monitoring.

And when authentication methods change, this means you are no longer compatible with these APIs. In case of business critical applications, this could even break down your systems and operations. Not to mention the misconfigurations that cyber attackers could exploit.

3. All the Breaking Changes.

Lastly, API breaking changes can also have a significant impact on the technical debt of your organization. Whenever an API undergoes a breaking change, it forces developers to modify the code of all the applications that rely on that API. Any other dependencies or components that rely on that API may also need to be updated, causing a ripple effect throughout the entire system.

This accumulation of work required to fix API breaking changes is a form of technical debt that can increase over time, especially if your organization relies heavily on APIs or has a large number of applications that use them. Failing to address this technical debt can lead to increased development costs, decreased system reliability, and slower time-to-market for new features.

How to Reduce & Avoid the Technical Debt of Consumed Third Party APIs

1. Think Long Term Solution not Maintenance 

Whether you’re a startup, getting started, or an organization already scaling - the APIs your business consumes need to be managed on the go. Maintenance is a hassle, and adds redundant overhead for small teams or falls through the cracks.

At Lunar, we believe efficient long term solutions or enterprise-grade features should include:

• Routing between API providers alternatives
• Cost control and ownership of API consumption
• Low latency
• Fail-safe mechanisms
• PII obfuscation
• Supporting different environments
• Storing API tokens securelyAPI resilience and fallbacks implementations 

2. Map and Catalog All Your External APIs and API Endpoints in Use

It's common practice, but not all scaling companies have an internal catalog or portal to gain visibility into all the APIs they’re consuming across the organization and the data points, issues, or expected breaking changes. 

A unified catalog of all external APIs and API endpoints in use enables tracking their maintenance and ensuring no technical debt is being accumulated. While this is a common practice for internal APIs, it usually does not take place for external ones.

3. Decouple your API Maintenance from Your App Code

Here’s a known pro tip - there really is no reason to build the same code again from scratch with each new API integration.

API maintenance needs to be streamlined as much as possible and that’s why it deserves its own resources. Ensure API maintenance becomes its own separate part of the developer workflow, and not a part of code writing, or crunch time, since it could easily get deprioritized or just become partially managed.

To support this process, ensure API third-party policies are configurable, so developers can easily control them. In addition, create standardized configuration of API maintenance, so developers know what they need to look like and can easily fix them.

4. Simulate in pre-prod and create alerts

This part is more about checking whether the fix you've created actually does what it needs, instead of deploying it blindly to production. Companies large and veteran enough have already built a simulation environment to do so.

When simulating, opt to create a health score for your consumption, based on your own needs and business logic. Finally, create alerts on future API breaking changes, so you can update the health score. By tracking the health score and fixing recurring issues you can prevent future technical complexities.

Examples:

• If you are implementing API caching, simulate the capability on production traffic - measure the cache hit ratio before pushing to production.
• If you’re implementing a throttling policy, make sure to measure the impact of the throttling overtime, before releasing to production.
• If you want to obfuscate PIIs in API calls, make sure to see that all sensitive data has actually been obfuscated before going to production.

5. Autonomous and Automated - Prepared for Any Scenario

Ensure you have resilience policies in place that can backup each and every API you consume. This is made up of three parts:

• Circuit breakers - By using circuit breakers in your development, you can find, predict and prevent failures that could require troubleshooting and minimize the accompanied technical debt.
• However, since you can’t prevent all failures and you can’t control external API providers, make sure you have a fallback policy and backup plans, like using another provider. For example, if you’re using an external API for payments processing, create an alternative payment option if Stripe’s availability or latency exceeds a certain threshold and responses take too long.
• Finally, have a prioritization plan in place. At some point, you will exceed the number of API calls you have set. Then you need to decide how to prioritize the most valuable API calls, which ones to drop and which ones should be sent during off-peak hours, like the weekend.
  

Takeaways and conclusions 

Finding a robust solution to help with all the management heavy lifting is not easy, but we’ve been building Lunar exactly with these issues and solutions in mind. 

If you have the time and expertise to think about long term solutions and management, to test and simulate, to decouple maintenance from your code, and to monitor costs and optimize on business logic - that’s great. But as most companies don’t - Lunar is the best solution to give robust support for all these problem areas. 

Managing tech debt is really the last thing you should worry about, so managing integrations while being minded to optimal consumption should be the strategy to opt for.

***

To learn more and chat about API consumption issues, get in touch at info@lunar.dev

Follow our journey on Twitter or LI.