Performance-Aware Microservices Architecture Live Planning and Scaling

Microservices architecture (MSA), with containerization and Cloud provisioning, is now a standard for software deployment, driven by automation and DevOps practices. While improving modularity, flexibility and maintainability, MSA presents challenges in non-functional quality aspects, including security, performance, and reliability. This paper addresses the open problem of adaptive, real-time capacity planning for performance-driven scaling of microservices, focusing on the MSA API gateway pattern. A queuing-network-based methodology is developed to estimate the microservice replicas per workload by explicitly modeling infrastructure and interactions. This approach enables accurate, flexible infrastructure scaling capacity planning and design-time analysis of system properties. A 3-step methodology including benchmaring, modeling, and deployment is proposed and applied to a real-world case study on an API gateway MSA to demonstrate its effectiveness.