Microservices Test Environments: How to Enable Collaboration and Speed at Scale

Organizations usually adopt a microservices architecture because they can break code into smaller pieces that can be managed independently. As we’ve discussed elsewhere, microservices architectures offer improved scalability, fewer dependencies between teams, and faster time to market. While too many dependencies between teams can slow progress, teams still need to be able to collaborate, especially when it comes to API development and software testing. The reality is that the microservices model doesn’t facilitate collaboration, a challenge that often becomes increasingly clear as organizations grow.

Here, we’ll describe the benefits of using a microservices architecture, along with the reasons why it can be difficult to realize those benefits at scale. We’ll also discuss forward-thinking ways to enable rapid feature development and smart resource sharing in a collaborative development and testing environment moving forward.

Why Organizations Adopt a Microservices Architecture

The problem with monoliths

In recent years, many organizations have adopted a microservices architecture. This shift is largely the result of the many limitations of a monolithic code base, which can become too unwieldy to manage.

In a monolith, developers can’t work independently and instead remain highly dependent on others, which makes it difficult to make and deploy changes quickly. While unit testing is possible, end-user behavior can only be tested as a whole using a waterfall approach instead of on an ongoing basis as development occurs. No matter where the bottleneck is within the code base, developers have to replicate all the code vs. simply dealing with the affected parts of the code. When one thing breaks, everything breaks. This limited scalability slows the software development release process, which gets very expensive. As changes accumulate, it becomes harder and harder to test results because software regressions tend to increase in number, size, and severity.

Running the entire code base as one unit constrains scalability. In response, organizations that use monolithic architectures can scale vertically by using more CPUs, more RAM, and bigger machines, or they can scale horizontally by adding more machines.

The benefits of microservices

One key reason organizations move to microservices is scalability. Code can be broken into smaller parts that can be developed, tested, deployed, and updated independently. Say, for instance, that there are 100 instances of one high-traffic service vs. 5 instances of another service. Developers can take advantage of fine-grain scalability on a per-service level rather than adopting a brute-force scalability approach.

With a monolithic code base, a problem with one part of the code can affect or even break the entire application. With microservices, if one service has a memory leak, for example, only that service is greatly impacted, reducing troubleshooting turnaround times and making the entire application much more reliable, resilient, and fault-tolerant.

In addition to operational scalability and reliability, microservices offer production efficiencies. In terms of delivery, developers can share changes regularly without affecting stable parts of the code, giving them the autonomy and flexibility to determine their own release cadence. Making smaller changes independently and more frequently, perhaps even daily, reduces risk, leads to faster deployment, and lends itself to a more cost-effective, agile model of delivery.

The Downsides of a Microservices Model

Dependencies

Though in principle microservices enable a faster shift to production, the reality is that the actual software delivery speed may not be as fast as anticipated once the number of services crosses a specific threshold. When an organization has roughly 20 or more microservices, the benefits of independent development and shipment to production can start to erode. It’s at that time that dependencies become a real issue. Though the whole point of microservices is to reduce dependencies on other teams, dependencies never disappear entirely. Instead, the dependences shift to well-defined APIs.

Simply put, microservices rely on each other. Each independent unit of code talks to other units of code through well-defined APIs that make external calls to other services. As a result, the number of dependencies increases over time. While unit testing is fairly straightforward, the bulk of the testing shifts to the boundaries since the service needs to talk to other microservices external to it. Because system behavior is so dynamic, the burden shifts to the interaction between microservices, making it more challenging to test small pieces of each microservice. A great deal of time is commonly spent debugging production incidents, especially at scale.

Collaboration

This problem can also impact code development itself. Because developers constantly have to define APIs that other teams are consuming, collaboration is essential. However, one piece of code may not necessarily play well with others, and it can be hard for developers to see the whole picture because they’re isolated in one set of services. Developers often won’t know what does or doesn’t work until fairly late in the game.

There are other operational considerations to take into account as well. Unlike a simple monolithic code base, organizations can have tens or hundreds of microservices, each using its own database, code base, and language. At scale, the complexity can be daunting.

With so many moving pieces in a microservices architecture, testing and quality assurance can cause significant friction. As the number of microservices increases, it becomes exceedingly difficult to access a high-fidelity testing environment, especially at an earlier stage in the development process. What ends up happening is that integration issues that should have been discovered earlier are instead discovered much later in the process. It’s expensive to debug and fix issues found in the staging environment. Further, as issues are discovered at this late stage, the environment itself becomes unstable. Every time an issue is uncovered, the environment breaks and can’t be used until the code is rolled back. Worse, it’s not always clear which commit caused the environment to break or what to roll back.

Testing is one of the biggest difficulties with the microservices model. For example, regression testing is typically done through automated testing as part of the CI pipeline. Because a CI environment isn’t a true representation of a real-world production environment, it’s hard for testers to catch changes that may have regressed existing functionality. New software and feature testing can also pose problems, as new features often touch the UI, front-end tier, API tier, or multiple other microservices on the backend.

Ultimately, errors discovered late in the development cycle require rework and slow the process down, wasting valuable developer time.

How Organizations Can Reclaim the Benefits of Collaboration

Solutions

Microservices allow independent development of individual services, but new feature development requires collaboration across service boundaries throughout the development lifecycle. Because each microservice is being developed independently, the most common place where all the pieces of a new feature can be deployed together and tested for compatibility as a whole is the staging environment. The staging environment is shared by all teams, meaning that it can’t support parallel development and testing.

In response, some organizations use a brute-force approach to scaling, where they achieve parallel feature development by cloning multiple copies of an environment. Duplicating the staging environment creates a pool of environments that can be used for parallel development. The benefits include higher fidelity testing, shorter feature loops, and moderate scalability.

To avoid bottlenecks completely, organizations and devops teams might duplicate a staging environment for every feature. If there are 50 microservices, feature testing would involve all 50, which would require substantial operational overhead and become cost prohibitive.

The Signadot Sandbox Solution

Signadot has a novel microservices testing solution that allows developers to see the effects of their changes in real time. Signadot is a Kubernetes-native platform that enables organizations to spin up hundreds or thousands of lightweight environments, called Signadot Sandboxes.

Developers can use Sandboxes to collaborate on testing features end to end before merging, which allows for fast iterations on APIs without impacting other developers. Unlike a traditional environment setup, Signadot Sandboxes use application-level multi-tenancy to share resources. Multi-tenancy creates a large number of high-fidelity environments that can exist in parallel.

To be clear, multi-tenancy doesn’t duplicate resources blindly; instead, this environment shares resources between environments in a strategic way and achieves isolation at the application level, isolating test requests by labeling and routing request traffic. Using this dynamic routing of test requests, teams can create thousands of these Sandbox environments within one physical Kubernetes cluster without the infrastructure cost of duplicating physical environments or the operational burden. Because isolation is built into these environments, multiple developers can test their services at the same time without affecting each other’s work.

For example, let’s say one service is producing a new API and another service is consuming that API. Both services need to be able to communicate even before code is merged to the main branch. A developer can make a code change in the dev branch, and the Sandbox will deploy the change into a Sandbox environment. Using dynamic routing requests, developers can then independently make changes and consume each other’s changes quickly.

Another benefit of a Sandbox environment is that developers operate on a single baseline cluster (much like a staging environment), which is continuously being updated as part of the CI/CD process. Developers will know their testing is valid because they will always work against the latest versions of all dependencies.

Before Signadot: Developers need to commit their changes to their main branch, deploy to staging, and only then be able to test that their feature works end to end. They have to follow this process every time they make a change to each of their services.

• Not scalable
• Long and slow feedback loop
• Regressions introduced into the staging environment cause other teams to be blocked

After Signadot: Developers can commit changes to their feature branches and test their changes end to end using Sandboxes.

• Scalable
• Very fast feedback cycles
• Independent testing of their feature

Collaboration Is Back -- and Better Than Ever

While microservices architectures bring a great many benefits, they also have drawbacks, such as difficulties with collaboration and testing. Understanding viable ways to overcome these challenges is key to an organization’s ability to optimize a microservices architecture.

Many leading tech firms such as Uber and Lyft use a multi-tenancy model to provide test environments at scale that meet their microservices testing needs. For most organizations, however, implementing and maintaining an effective and reliable solution isn’t affordable or practical. For this reason, Signadot’s innovative, Kubernetes-native implementation of this model is ideal for growing organizations that have 20 or more microservices.

Developers often struggle to collaborate because they’re siloed in independent development environments. However, when everyone’s developing on the same staging cluster using Signadot Sandboxes, it’s easy to collaborate, resulting in a highly beneficial, end-to-end testing environment for each feature. With Signadot, developers can continue to work independently while collaborating on API production and consumption -- all without breaking other developers’ work. And it all happens pre-merge with fast feedback loops.

Signadot’s cost-effective, scalable, multi-tenancy Sandbox model enables smart resource sharing, greater collaboration, and streamlined testing. To learn more, check out how DoorDash used Signadot to build a safe, highly reliable multi-tenancy solution or reach out!

‍