Key points
- You can scale specific parts of your application independently to match demand, which helps save on costs and server resources.
- If one part of your system fails, the rest of the application can usually keep running, preventing a total system outage.
- Managing many small services is complex, so you need automated tools to handle starting, stopping, and connecting them reliably.
- Because services are spread out and restart frequently, you need centralized monitoring tools to track performance and find errors quickly.
- This approach is ideal for large, complex projects, but the extra setup and maintenance make it unnecessary for most smaller businesses.
- You must store important data separately from your services and use automated systems to back up that data and report errors.
A microservices-based architecture benefits apps that need to be highly scalable, resilient, and available, as well as benefiting use cases where latency is an issue, allowing for components to be distributed nearby to end-users. Microservices aren’t just for hosting consumer and SaaS apps, and can also be deployed for internal infrastructure and tools where reliability is a priority.
This guide explains how to scale microservices and how it differs from traditional app architectures, and the impact this has on how IT teams need to manage, monitor, and maintain microservices-based systems.
What are microservices?
Microservices (or a microservices-based architecture) is a way of developing apps using independent component services, each encapsulating a specific functionality. The components are loosely-coupled, communicating using APIs (usually over HTTP) so that if one service fails, the others can keep working. Microservices are usually deployed for client-server hosted apps and services, not apps that run locally on your devices (though they can be containerized for this purpose).
Because each component service is independent, they can be individually scaled: for example, a simple e-commerce app may be split into components for processing payments, sending emails, and displaying web pages. If there was a sudden spike in traffic (but no one was actually checking out their carts to trigger payments and emails), only the web server microservice would need to be scaled. This is resource-efficient and cost-effective, while ensuring there is always capacity to meet demand.
Microservices and monolithic architectures
Microservices are an alternative to a monolithic architecture, where everything is in a single codebase, typically hosted on a single powerful server.
The primary benefit of a microservices-based approach is scalability: to scale a monolithic app, you can either increase the resources available to it (i.e., more memory, CPU, known as vertical scaling), or run another instance of the app and spread load across them (horizontal scaling).
Scaling monoliths can be highly inefficient depending on the size and nature of your app, as it may mean assigning resources for an app where only a small component of it is under heavy load. In microservices, because each component can be horizontally scaled, overall resource usage can be optimized.
High availability is also addressed by microservices: in a monolith, if the app crashes, it all crashes. Conversely, if an individual microservice crashes, the others can keep running while it recovers (for example, if your email sending microservice crashes, customers can keep shopping and may just experience a delay in receiving an order update, rather than not being able to shop at all).
How microservices change system interactions
When microservices scale horizontally, containers or instances are created and destroyed to meet demand. This means anything in local memory or storage is lost. Part of building a scalable microservices-based app is handling this with persistent storage, databases, and ways to share state (such as key-value stores).
Microservices also rely heavily on network performance: components must be in constant communication, with low latency and no interruption.
New failure patterns in distributed microservices-based systems
While one of the benefits of microservices is that components can keep running when others fail, this is not always feasible, depending on your use case and the design choices made during development.
Components will inevitably rely on the availability of others, and some operations must be performed in order. This means that failures need to cascade, and error handling needs to handle partial system outages, not just full failures.
Scaling works differently in distributed environments (and this has side effects)
Scaling, combined with isolated components that can be updated individually (so long as their APIs are consistent), results in nodes that are frequently created and destroyed as resources are allocated based on demand. In some cases, a container or instance for a particular service may only live long enough to serve a single request.
This greatly impacts visibility, as the state of each component is lost when it is destroyed, including error logs needed to diagnose issues.
Coordination challenges across services
Microservices must remain coordinated and available to each other, and dependencies between services accounted for in configurations. This involves the use of tools like Kubernetes, or leveraging the native features of cloud platforms like Azure and AWS, to manage containers and virtual machines.
When latency or connection issues occur, microservices-based architectures can become unreliable or unavailable.
Performance depends on microservices’ purpose and relationships
In a microservices-based architecture, performance can be prioritized per component. For example, a web server should respond as quickly as possible so that end-users get the best experience, whereas an email notification server can take a few seconds to fire off a notification without impacting operations.
In some cases, a microservice may actually be an external API (e.g., a communication API for sending SMS), which would have different performance expectations than a service hosted in the same datacenter as your app.
This makes performance optimization more complex, and more than just a matter of raw numbers.
How complexity increases over time in microservices-based systems
While seemingly more complex, microservices can actually simplify IT operations when leveraged appropriately: it’s often easier to troubleshoot a single service in isolation than an entire monolithic app-stack.
While there are initial setup complexities, orchestration and automation make deployment of microservices-based apps increasingly simple for DevOps teams. CI/CD tools and infrastructure-as-code tools like Terraform remove much of the legwork and risks of misconfiguration.
Microservices performance and security monitoring differences and challenges
The key to the success of any IT system is visibility and a proactive approach to solving problems. IT and DevOps teams that are constantly ‘putting out fires’ cannot provide stable, sustainable IT services. If you are hosting microservices-based apps, you must be able to monitor and troubleshoot virtual machines and containers.
Containers as a service (CaaS) can also be used to abstract the direct management of infrastructure. These services offer streamlined interfaces for managing containerized apps, and take care of many of the configuration, maintenance, and security overheads associated with managing container orchestration platforms.
Backups are another challenge for microservices: running instances should not contain long-lived data that requires backing up, whereas key-value stores, databases, and object storage used to persist data may all need a suitable backup solution that allows for roll-back or full recovery, with immutability to protect against malicious threats.
When microservices deliver the most value
Microservices-based apps deliver the most value when:
- Components require independent scaling
- Applications are large and complex
- Development teams need to work independently
- Continuous change is required
For end-users and business stakeholders, the value of microservices is in reliability, speed, and cost optimization. For tech teams, it’s consistency and ready scalability.
That said, you should not rush off to ask your developers to move your existing code to a microservices-based architecture. Most small to medium-sized businesses are unlikely to benefit from the additional work required to rebuild established codebases and the operational overheads of maintaining distributed systems.
However, if you are designing and implementing a new system, you should assess whether there are long-term operational benefits to using a microservices-based approach, especially if you have plans to scale to serving a large number of users.
Oversight and performance for your entire IT infrastructure
Gaining insights into the operations of ephemeral, auto-scaling infrastructure can be a challenge. While many tools allow for error handling from within containers and spot instances, these are often developer-focused.
For IT teams that manage the day-to-day operation of microservices-based apps, you should look for error reporting tools that work with your codebase and allow for integration with a unified IT platform. This centralizes alerts from your microservices apps, and puts them alongside your endpoint, security, and network monitoring notifications.
You can also set up application performance monitoring to benchmark individual microservices and overall performance, and use helpdesk automation to escalate issues or cost anomalies to the right technician for immediate diagnosis.
