Why We're Building Lunar

Introduction

Let's take it from the top before we dive in - API consumption is a huge vortex that’s costing and causing too many companies, just too many issues in production. So we set out on a long ideation mission, and started building a new paradigm and tool to fix that. But to truly understand why we started Lunar, we need to lay down these rules that guided us:

1. We were not trying to solve a single problem. We wanted to look at an endless problem space (pun intended)

It’s not about building caching to reduce API consumption cost, nor is it to apply better visibility to your cluster’s out-going traffic. It’s the over-laying understanding that many companies run into a span of problems with 3rd party APIs, and that looking at the outgoing API traffic as a whole, will allow us to better understand the problem modern cloud-native companies are dealing with.

2. New kids on the devtools block…

Don’t get me wrong. We’re techies, and we love digging deeper into hard technical problems and sharing our thoughts on the latest in the industry. But as 2 founders that spent years of their career in the cyber security industry, we’re joining the cool kids on the block, and bringing best practices from the security industry, which is what helped us to map the problems and tackle them from a different angle.

3. We tried refuting our thesis more than we tried proving it. 

We felt many of the API consumption and integration problems first hand. We have all been there at some point. We all solved them one way or the other. But many months into our research we started connecting the dots and understand that there’s a bigger problem at stake than just us trying to implement a caching mechanism for VirusTotal or throttling our Okta API calls. For a long time I thought that we are solving an (important) ad-hoc problem, no reason to think that an infrastructure of some sort is needed here. I mean, it’s part of a “pay-for-play” when you integrate with 3rd party APIs right?

Looking back at what we’ve learned in over a year, we managed to get invaluable industry insights that led us to build a  thoughtful and elaborate thesis on the need to unify, manage, control and optimize the ever-growing usage of 3rd party APIs. Along with the ever growing usage of the API economy, it only made sense that a different type of approach should be taken.

4. Fourth rule of Lunar - DO NOT talk about Lunar.

Just kidding, I got carried away with this one..

So now that we went through the rules we’ve been following since the day we embarked on this journey, here’s that part of the blog post when I talk about us as founders.

‍

Our Background

Roy and I go way back… all the way back to first grade. This was  a time of cringey hairstyles and questionable fashion choices. But despite all that, our friendship has stood the test of time and has only grown stronger over the years. We've been through so much together, from childhood to colleagues; until we decided to embark on a new adventure and start the Lunar mission.

Our Strategy So Far

At Lunar, our strategy orbits our vision of providing a platform for Unified Management of API Traffic. With that in mind, we’re throwing into the orbit maximum flexibility and developer autonomy to define policies based on specific business logic. Being mindful of these goals, we started planning how our very early start-up is going to look like.. 

Starting with a prototype.

“If a picture is worth a thousand words, a prototype is worth a thousand meetings” 

Before doing the heavy lifting, recruiting all of our amazing R&D core team and coding ourselves to sleep - we needed the indication that our thesis is right. And in order to do that as fast as possible and show value to design partners, we decided to start with a prototype. 

As the old saying goes - “Even our prototyping took 2 phases”

1. Phase 1 - Flip an API Gateway (been there tried that)

We had to see it for ourselves - was it really as easy as some suggested? We took the open-source API Gateway (shout out to KrakenD) and wired the outgoing API traffic through. We battled this prototype for quite some time, until reaching the inevitable conclusion that though API Gateways are great for API producers for managing their inbound traffic, that’s not the case with outgoing API traffic. Another solution needed to be built.

‍

2. Phase 2 - Build it, Map the problems, Investigate.

Second phase of our prototyping was where our Cyber Security background really kicked it and we were actually getting down to the heart of the matter. It wasn’t about making our MVP as scalable and robust as possible for the design partners. That phase was a crucial and iterative process in mapping customer needs, performance specifications and avoiding major development detours and pitfalls. 

During this phase we’ve also implemented internal visibility capabilities, on top of Grafana, so that we could analyze the API traffic of our design partners, and research the root causes for consumption issues. I believe this decision paid off, because as we now move to our closed-beta version, we gained so much knowledge and specific expertise that made our architecture endurable and robust for production environments from the very beginning.

Allocate time for proper research.

“If we knew what we were doing it wouldn’t be called research.” 

- Albert Einstein

At this point we’re working out of our offices in Tel-Aviv, 8 people strong, with two office dogs that have to take turns coming in, but we love them as they are. 

We encourage our team to dedicate time to research. As a company that values professionalism and data-driven decision making, we believe it's crucial to be convinced of our strategic decisions from both the R&D and product side. We spent a lot of time conducting deep research and analysis, taking our aggregated knowledge from the prototype phase. A few practices worth mentioning are:

1. Research individually Vs. researching in pairs

We did both. Most of the research tasks started individually, and afterwards peer-pairing to help extract and articulate the fresh knowledge from one another.

2. All-hands meeting

This is where the final decision took place. It was a well thought forum, where we mapped down all our options, baffled on the pros and cons of each direction and convinced the forum regarding the road to go down.

3. Aggregating data points from customer calls and surveys

Because we documented most of our customer calls (that’s over 100 calls during the early days) from the very beginning, we had a decent data-set to run analytics and gain insights that guided us in our data driven approach. And in order to increase our reach behind our network of friends and colleagues, we ran surveys to establish a more holistic understanding of our customer’s needs.

Hit the bench(mark)

“Benchmark has got a lot of playability. You can sit with your family and friends, and you'll all have an opinion about what the answer is. What's amazing is when some of the answers are revealed, it creates even more discussion.”
— Paddy McGuinness

Finally, we decided to benchmark our performance for each phase of the development. As a platform that sits in the critical data path of companies, we knew there was a lot of responsibility on our shoulders. We’ve built procedures and internal toolset to measure our system, even before the official release, to ensure our users that:

1. Lunar acts as an external layer and adds as minimal latency as possible to the outgoing traffic. Another guiding rule of thumb - do not mess things up with a customer.
2. The platform supports hundreds of concurrent HTTP connections to withhold high volumes of traffic, positioning ourselves with the market leaders of OSS proxies.
3. Memory consumption will remain low and will not stress our customer’s environment whatsoever.
4. Implementing fail-safe mechanisms, to make sure the customer's system continuity will remain. That means that if Lunar goes down, the API call will not be lost and it will be made directly to the API provider.

Moving forward, we plan to scale our company by continuing to innovate, adapt to market trends, and focus on providing exceptional customer service. We believe that our strategic approach to launching the company, coupled with our commitment to research, benchmarking, and customer satisfaction, will ensure that Lunar remains at the forefront of the API traffic management market.

Challenges

As developers, Roy, Lunar’s CTO, and myself,  set out on this journey knowing from our previous professional experience how the API market was an area of focus for many in the industry. We believed that the management of  consumed  3rd party APIs was an underserved market, but it took us a long time to come to that realization.

 Along the way, we faced several challenges that required us to dig deep and find creative solutions. One of the biggest challenges was market education. We had to convince potential customers that a unified management system for outgoing  API traffic was essential, especially as companies grow and their production environments become more complex. Ah yes, that good ol' Build Vs. Buy dilemma!

At times, this is still a  tricky task. While many developers and R&D teams we spoke with agreed that they had built some form of API middleware to monitor and control outgoing API calls, some were hesitant to move to a more elegant and cost-beneficial  solution vs. a homegrown one. However, we continued to advocate for the benefits of a centralized management system, highlighting the maximum flexibility it provides in defining policies based on specific business logic.

From the very beginning, we were also asked to address concerns around security and performance. We knew that we could not become a bottleneck for  API consumption, nor should we store PIIs, and we had to ensure that we did not impact latency. We set out to build mechanisms that make our solution secure by design, efficient, and effective - by design as developers would want and enjoy. 

What’s our vision

Talking with so many amazing developers, R&D leaders and cloud infrastructure gurus over the past 1.5 years, ranging from different verticals and company stages, has shaped our way of thinking. We grow to the understanding that: 

1.  It’s easy to integrate. It’s a headache to maintain.

1. Every BE encounters it - it’s not a problem to start working with 3rd party APIs. That’s the beauty of the API economy and its mission statement. And there are many tools out there to help developers do so. But problems  occur after integration. Once going to production, now developers are experiencing problems that they didn’t tackle during integration. 
2. The real issues start at a certain scale  - Consuming at scale (hitting those rate limits much?), API breaking changes, anomalies on consumption patterns, rotating API tokens and the list goes on and on. There’s no working with external APIs without understanding that it comes with the ongoing costs of maintaining those cardinal components in your production environment.

2. Though every company consumes their external APIs differently, they’re building the same type of API middleware 

1. Throttled by email APIs? Many companies will implement their own client-side throttling logic to avoid rate limits
2. Expensive  API call for KYC APIs? You’re not the only one building a Redis caching to save on API calls
3. Cool time increases exponentially with Microsoft365 APIs? How about coding the retry logic based on your special needs

That was an insightful thing to learn. From that understanding we realized we need to build the platform, with the commonly known middleware plugins, but also having customers having the ability to build and configure their own type of plugins. Our customers knows best how to configure the right middleware according to their business logic. We’re here to help them do it as fast and smart as possible.

‍

Shining a light on the dark side of APIs, and beyond. 

Sharing our building journey.

This story starts in our very own galaxy with a very tired engineering team,  facing 3rd party encounters and API headaches… alone. 

As we progress in our journey, we’re happy to share some lessons that we hope can help other entrepreneurs facing similar challenges at these stages: 

• Education first.  Be prepared to educate the market about your solution and its value.
• Lend your ears more than your code. Always be willing to listen to your users and early adopters and adapt your approach to meet their specific needs. 
• Security is non-negotiable. Do not compromise on security and performance, but be creative in finding solutions that meet the highest standards while still delivering maximum value to your users.

Lunar’s manifesto.

• We believe that there’s a need for a new category in the API landscape.
• There are API gateways to manage and and enforce inbound API traffic for the APIs you build as there are service meshes to manage and enforce the interconnectivity between services. That’s why there’s a need for an Edge Proxy - to gain unified visibility, controls and optimization on outgoing API traffic. 
• We pride ourselves on designing a modular product that both developers and managers can enjoy, one that works as an additional layer to the infrastructure.

And that’s what Lunar is all about!

To conclude, the mission has just begun. I’m excited to see where the journey will take us and am expecting great things ahead. While knowing that building a company is a roller-coaster, best to have your friends with you for the ride. 

Stay tuned for more to come!

‍