Analyzing Polyglot Microservices

As a member of the platform engineering team that supports and powers Capital One’s flagship mobile products, part of my job is to chase new technologies and paradigms, testing the boundaries of our evolving system. A key part of innovation for us is to conduct iterative experiments with the current architectural design, as well as explore how emerging technologies could fit into it. This requires an inquisitive mindset and an environment in which one can safely fail, all with the aim of delivering value. For my team at Capital One, it’s not only possible to run rolling experiments, it’s our mandate to explore.

One topic that has piqued our curiosity recently is a platform architecture that supports the development of polyglot microservices and the prospect of how it could allow our mobile teams to deliver features more efficiently.

Why Polyglot Microservices?

The essence of a polyglot architecture is to delegate the decision over which technology stack and programming languages to use to the service developers. There are numerous big technology companies that have evolved to support a polyglot microservices architecture, like Google, eBay, Twitter, and Amazon. Like Capital One, these companies operate on a vast scale of many products and many teams. Before I move forward with a polyglot architecture thought experiment, it’s important to take pause and understand what the need for pursuing a multi-language microservice ecosystem could be within a given enterprise.

Enterprise Needs

As the variety of computing devices continues to expand, and business demands become more complex, the services that contribute to those capabilities must stay ahead of that growth. The era of migrating from a monolithic to a microservices-based system has largely passed and nowadays, hailing the virtues of a microservice architecture is mostly unnecessary. Instead, we are in an age that’s focused on extending the stable power of microservices within a distributed system. Whether it’s enhancing the infrastructure that allows intelligent communication between services within a service mesh, distributed tracing of requests for monitoring, or advanced traffic management with approaches like canary rollouts.

Role of Platform Teams

As part of the platform team, it’s our task to analyze, design, and build out the foundation and necessary tools for the wider mobile organization. And as a team that supports the development of microservices that respond to the entirety of our incoming mobile traffic, we do not have an “ad hoc” development mindset, we take a structured and more formal approach.

When a team like ours is in the role of supporting multiple groups across many lines of business, our speed to develop and deliver a customized user experience reaches a ceiling.We support engineering teams that write in multiple client languages, runtimes, and ecosystems so our aim is to build solutions that work for everyone.

There is an obvious opportunity to hire more backend engineers and scale up the organization and delivery. But with polyglot microservices, we wanted to explore what it would mean to lower the activation energy of our existing mobile developers to build and deliver their own microservices.

Potential Advantages of Polyglot Microservices

Let’s take a look at the potential benefits that a polyglot microservice architecture could provide.

Developer Creativity

Currently, our microservices and libraries are centered around Java-based tools, like Spring Boot, Spring Cloud, and a heavy dose of the Netflix OSS stack. This should come as no surprise since the Java ecosystem has evolved to adopt microservices architecture in remarkably innovative ways. But not all mobile engineers are equally well versed in Java. With polyglot microservices, even if the entire microservice ecosystem is built using Java, an iOS developer who writes in Swift is no longer limited by their lack of Java experience.

When you allow developers to pick the language of their choice, we as the platform team provide a set of guidelines on production readiness requirements and the scaffolding of a base microservice to ensure these standards are met. In addition to that, we provide the proper knowledge transfer about the overall system to ensure that an engineer has all the tools required to participate in building services. For example, let’s say a mobile developer identifies a beneficial user experience that could be more efficiently provided for by a microservice than extensive UI logic. Now with polyglot microservices, they would now have the tools to close that feedback loop themselves instead of only viewing the system from a single perspective.

Therefore, the potential for developer creativity and out-of-the-box problem-solving exponentially expands when the governance over which language to develop in is loosened. Put another way, it inspires innovation, a sense of ownership within an engineer, and the tools to act on the two.

Autonomy

From an organizational and process standpoint, it can be frustrating to constantly depend on another team in order to deliver new business intent. In case the entirety of the backend is written in Python, yet a team desires to write their microservices in Kotlin, the autonomy to reach their internal goals is restricted as they cannot directly contribute to the backend system.

In order for teams to regain their autonomy, they must know how the rest of the system functions as well as be able to meaningfully contribute to it. There is the potential to empower teams by providing core functionality within the scaffold such as application logging, health checks, and an environment to deploy so that they can focus on writing the business logic.

Time to Market

When you lower barriers in engineering, the rate of efficiently delivering business solutions tends to increase. When teams can focus on tools they are familiar with, you remove mental overhead so they can focus on value-added work, as opposed to figuring out the fundamentals. So, instead of worrying about how to containerize their application, how to implement circuit breaker patterns, or how events should be logged, the engineer can focus on the impact to the business intent.

Moreover, when the microservices are standardized across languages, it’s much easier to extend the logic across multiple platforms and infrastructures, while also creating a consistent pattern for deployment and operations. Again, engineers can continue to stay in context, there is a much higher level of reusability and maintainability, and they can expand their understanding of the larger system they are working with.

Potential Disadvantages of Polyglot Microservices

Now, to review the potential pitfalls of a polyglot architecture.

Complexity

Without a doubt, in a system that supports polyglot microservices, complexity abounds. No engineering organization, enterprise or otherwise, is looking to add unscreened resources to a system. One obvious source of complexity is coordination and consensus between teams. In order to control this complexity, this necessitates activities like a deeper understanding of the common libraries and dependencies used within the microservices, a well-defined purpose to the development of every new service, as well as the fundamental loose coupling of services (e.g. aligned directly to the business problem you’re trying to solve).

We predict that there needs to be a delicate balance between providing autonomy to teams, whilst also providing strong guardrails for them to achieve their development goals. Checklists and thorough documentation will be paramount to manage the added complexity.

Scaling Issues

I would be remiss to talk about microservices without mentioning containers at least once. With the isolation that containers provide, it is much easier to build microservices with multiple technologies. This has immensely influenced the approach to application design and deployment patterns over the past year.

However, there are certainly some costs of ownership and maintenance to this architectural pattern that are worth considering. For one, in the process of standardizing services in multiple languages, one has to solve the same problem across all those languages. This can be cumbersome and does not even account for the need to update frameworks, as well as any underlying dependencies. Of course, as a team with a large responsibility to serve our client traffic, this always entails testing the impact of these changes to our existing implementations.

The Future is Bright

As more and more client logic gets extended to the server, it’s important to provide a strong foundation so teams can stay ahead of the curve in implementing new technologies and trends. One direct manner to achieve that is to provide teams with the autonomy to develop in the language of their choice while maintaining operational parity with existing services. That is, to not only aim to allow many languages, but for each of them to have the same level of monitoring, tracing, resiliency, etc. as the incumbent technology stack.

Polyglot microservices have some interesting use cases, and we see them being specifically beneficial for the mobile teams that we support, and ultimately to our end users. As experimentation is foundational to my role, it’s incredibly empowering for myself and my team to research various architectural approaches and attempt to solve problems with it.