Service meshes and API gateways

API’s or application programming interfaces are a set of rules and protocols that allow two software applications to communicate with each other. As applications grow in size, so does the number of API’s too. without the right tools and infrastructure, managing these API’s can quickly become a challenge.

With the rise of API’s, API management platforms were for many years the primary technology to implement modern API technology, mainly driven by the implementation of API gateways, acting as a central server that sits between (public) clients and (private) backend applications.

Sketching the history, evolution and trends of API gateways is a subject of another blog series. But let’s say first came the API gateways, and much later, sparked by the rise of microservice architectures and container platforms, the service mesh sprang into life as a new product category and, sadly, largely ignored by the APIM vendors. Both are working primarily on the network service-to-service communication layer.

At first nobody understood how this new pattern played with the API gateway pattern, and to this day, many people are still confused by it.

What you usually see or hear is that API gateways are for north-south traffic and service meshes are for east-west traffic. Although in principle correct, it is not entirely accurate.

So, in this blog I well attempt to illustrate the difference between an API gateway and a service mesh and compare some different architectures for end-to-end API workload management.

Service Mesh

So, from the previous blog we learned that a service mesh is a dedicated infrastructure layer that manages service-to-service network communication within a distributed system. Its primary capbilities are:

• Traffic management
• Observability
• Security
• Reliability patterns

We also learned that before service meshes, development teams usually had to implement the service mesh use-cases in the application code themselves. This meant that different teams would be re-implementing the same functionality over and over again, creating fragmentation and security risks in managing network connectivity.

API Gateway

An API gateway act as a central server between clients and backend services and, from a business perspective, has two primary use cases:

API’s as a product

This is all about packaging an API as a product or business service that other developers, partners or teams will consume. In this particular case, the API gateway usually encapsulate common requirements that handles all the requests coming from the client application like authentication, rate limiting, developer on boarding, client application governance or monetization. In short: it’s all about enterprise API strategy and life cycle management.

API service connectivity and load balancing

The second use case is about enforcing networking policies to connect, secure, encrypt and protect network traffic between the external clients and the API gateway. This is where the overlap with service mesh capabilities starts to emerge.

Since an API gateway is, in essence, a reverse proxy, it also acts as a load balancer and by extension can handle some traffic management like http routing, proxy request to different versions of your Api’s, fault injections and so on, just like a service mesh who has basically the same type of networking capabilities.

An API gateway is therefore an API management tool that sits at the edge of a network, managing external client requests and routing them to appropriate backend services. Instead of clients directly calling backend services, they send their request to the API gateway. The gateway then processes these requests, enforces some security and routs them to the appropriate backend service.

• Its primary responsibilities include:
• Request routing
• Authentication and authorization
• Protocol translation
• Traffic management for external access
• Enforcing request and response policies
• Resilience
  • rate limiting
  • error handling
  • circuit breaking
• Observability

The API gateway pattern describes an additional hop in the network that every request will have to go through in order to consume the underlying APIs.

In this context, some people are calling the API gateway a centralized deployment and a service mesh a decentralised deployment.

Since service meshes are usually deployed on container platforms like Kubernetes, it is perhaps useful to bring some container platform capabilities into view for a more complete picture. So, we have to add two extra products into the equation: The Kubernetes Ingress controller and the Gateway API

Kubernetes ingress controller

Simply stated, a Kubernetes ingress controller is also a type of network gateway and it might come as no surprise when I say there are some overlapping capabilities with both an Api gateway and a service mesh regarding network concerns and http traffic.

Briefly, a Kubernetes ingress controller:

• Manages external access to services within a cluster

• Provides basic routing and SSL termination (like an API gateway)

• manages only http traffic (L7). This is perhaps the biggest difference, as API gateways and service meshes can usually handle also traffic at the TCP/UDP level (L4) or more application protocols then only http(s).

So, an ingress controller mainly serves as an entry endpoint into a Kubernetes cluster. A cluster reverse proxy server so to speak. If you don’t have a Kubernetes platform, there’s no real use for a Kubernetes ingress controller.

When you have a Kubernetes cluster and an API gateway but no service mesh, and all your backend service are implemented on Kubernetes, the API gateway usually forwards all the requests to the ingress controller, who forwards them to the correct internal backend services within the Kubernetes controller.

Gateway API

The Gateway API in Kubernetes is implemented to provide more expressive and extensible ways to manage network traffic then the original ingress controller. It is, therefore, like the ingress controller, part of the Kubernetes eco system.

The Gateway API provides a vendor-neutral API for traffic routing and management and enhances the basic capabilities of an ingress controller. The decision to use a Gateway API or an ingress controller depends on your specific use case and needs. You can always start with the standard ingress controller and when you needs are becoming more complex, you can switch to the Gateway API.

You can use them both in parallel, using the ingress controller for the basic use cases and the Gateway API for the more complex use case like advanced traffic routing or authorisation uses cases for example.

When a service mesh comes into the picture, the Gateway API becomes almost a necessity nowadays. Via the GAMMA initiative, new service mesh use cases are being built into the gateway API, so it makes sense to use a Gateway API when deploying a service mesh on a Kubernetes cluster.

Observe that the Gateway Api and an Api gateway are rather two different beasts with different use cases and different roles. The gateway API leans into the service mesh uses case and capabilities and is thus more focused on internal network traffic than an API gateway.

When you have a Kubernetes cluster, an API gateway and a service mesh, with your backend services implemented on Kubernetes, in a typical scenario, the API gateway forwards all the requests to the Gateway API, who forwards them to the correct internal backend service within the Kubernetes controller.

Again: if you don’t use Kubernetes clusters, the topic of the gateway API is a mood point.

No service mesh, no Kubernetes cluster

This is the simplest setup, most microservice architectures start this way. Nice and simply, nothing too complex and still manageable with manual operations. Routing configuration to the backend services is done with static configuration on the API gateway itself. Service discovery is not (yet) necessary.

No service mesh but a Kubernetes cluster

When the number of microservices reaches a certain threshold, manual management becomes a burden and organisations tend to switch to a container platform, with Kubernetes as the de facto default option to handle some operational management issues.

In this setup, the API gateway forwards all backend requests to the Kubernetes ingress controller. Service discovery is a built-in capability of Kubernetes, so when using the ingress controller, automatic service discovery is covered.

Service mesh and Kubernetes controller

At a certain scale, even default Kubernetes operations won’t suffice. This is the point where introducing a service mesh makes sense. The service mesh takes over some networking operational management tasks from the Kubernetes cluster. When multiple Kubernetes clusters are used, the case for a service mesh becomes even more prominent.

In combination with the API gateway, the Kubernetes cluster and the service mesh are providing a robust, secure and observable end-to-end API infrastructure platform.

In this setup the standard Kubernetes ingress controller is replaced with the Gateway API and the API gateway is routing all request to the Kubernetes Gateway API.

Service mesh but no Kubernetes controller

The last one is a bit the odd one out. Sometimes organisations using microservice architectures are running these backend services without using a container platform. The services are usually deployed on virtual machines or bare metal. But when the number of backend services reaches a certain number, the manual operational management becomes a burden, and a service mesh is introduced to deal with this kind of complexities. But since Kubernetes doesn’t come into play in this scenario, an alternative solution for service discovery that Kubernetes is offering out-of-the-box, has to be implemented.

This is usually implemented in some well-known services like Eureka or Consul. Note that this has nothing to do with a service mesh implementation. The API gateway simply needs to figure out programmatically where to send the requests to. Service meshes don’t solve the problem of service discovery.

Summary

When an organisation starts offering API products to its clients and/or suppliers, a need for managing these API’s emerges. If that API integration platform is build on a comprehensive microservice platform, you might consider using an API gateway for external interface management and a service mesh for internal service-to-service communication.

The recommendation is to start with an API gateway for basic external interface management and introduce a service mesh at a later time as your backend architecture becomes more complex or as your information security needs more stringent security measures.

If you have (a need for) Kubernetes-specific routing, choose the Kubernetes Gateway API, especially in combination with a service mesh.

The landscape of service networking is rapidly evolving. While service meshes and API gateways have distinct origins, they are converging to provide more comprehensive, flexible, and powerful networking solutions for backend service architectures.

Both API gateways and a service meshes have distinct roles in the broader domain of API management. Knowing why and when to apply API gateways and service meshes makes your API management more robust, secure and observable.