Scalability is a system’s ability to handle growing demand. On AWS, the main tool for it is EC2 Auto Scaling, which adds and removes virtual machines (EC2 instances) according to rules you set. Most AWS applications scale out, adding more instances behind a load balancer, rather than scaling up to one bigger machine.
Scale up or scale out
There are two ways to add capacity:
- Vertical scaling (scale up): Move to a bigger instance type. This usually means stopping and restarting the instance, and it has a ceiling, because there is only so large a single machine can get.
- Horizontal scaling (scale out): Run more instances behind a load balancer, such as an Application Load Balancer. This can grow further, and losing one instance does not take the whole application down.
Auto Scaling is built for horizontal scaling.
The three capacity settings
An Auto Scaling group (ASG) has three numbers:
- Minimum: The group never runs fewer instances than this.
- Maximum: The group never runs more instances than this. Always set it. It is your main cost cap.
- Desired: The count the group is aiming for right now. Scaling policies change it, but it always stays within the minimum and maximum.
Launch templates, not launch configurations
An ASG launches instances from a launch template, which records the machine image, instance type, key pair, security groups and other settings. AWS has deprecated the older launch configurations, and new accounts can no longer create them, so use launch templates for new work.
Scaling policies
Five policy types cover most cases:
- Target tracking: Keeps a metric near a target value, such as average CPU at 50%. AWS creates and manages the CloudWatch alarms behind it, so do not edit or delete them.
- Step scaling: Adds or removes different amounts depending on how far a metric is past a threshold.
- Simple scaling: Makes one adjustment per alarm and waits for a cooldown before acting again.
- Scheduled actions: Change capacity at fixed times, for example before a known traffic peak.
- Predictive scaling: Forecasts demand from history (it needs at least 24 hours of data and uses up to 14 days) and schedules capacity ahead of the demand it forecasts for the next 48 hours.
Target tracking is the simplest starting point for most web applications.
Choosing a metric
Predefined target-tracking metrics include average CPU utilization, average network in and out, and ALB request count per target. CPU is common, but it is not always the right signal:
- Web applications often scale better on request count per target.
- Queue-based workers often scale better on queue depth, such as messages waiting in Amazon SQS.
- CPU-light, memory-heavy workloads may need a custom metric.
Pick a metric that tracks the work the instances do.
How CloudWatch drives scaling
Amazon CloudWatch collects the metrics that policies use. EC2 basic monitoring sends data every 5 minutes at no extra charge. Detailed monitoring sends data every minute and costs extra, which lets the group react faster.
Alarms need several consecutive breaching data points before they fire. As a rough guide, scale-out can trigger after about three one-minute data points above a threshold, and scale-in after about fifteen below it. These values are approximate, so check the current defaults in the AWS documentation.
Cooldowns and health checks
Two settings prevent the group from thrashing:
- Cooldown: The time after a scaling action before another one can start. Simple scaling uses a default of 300 seconds.
- Health check grace period: The time a new instance gets before health checks begin. The console default is 300 seconds. If your application needs longer to start, raise it. Otherwise a healthy instance can be marked unhealthy and replaced again and again.
Spreading instances across availability zones
An ASG can span several availability zones and rebalances instances across them. When it removes instances, the default termination policy first balances the count across zones, then removes the oldest launch template or configuration, then picks at random. Spreading instances across at least two zones protects the application if one zone has a problem. See cloud regions and availability zones explained.
A practical setup
- Create a launch template with the machine image, instance type and a startup script that installs and starts your application.
- Create an Auto Scaling group across subnets in at least two availability zones. Set the minimum, desired and maximum counts.
- Attach an Application Load Balancer and a target group, and turn on load balancer health checks.
- Add a target-tracking policy. For example, keep average CPU near 50%.
- Test it. Generate load, then watch the scale-out and scale-in events in the group’s activity history.
- Set a budget alert in AWS Budgets before you rely on the group.
Design the application to be stateless. If session data lives on one instance, a user can lose their session when the group replaces that instance. Store session data in a shared store instead.
Costs and limits
EC2 Auto Scaling itself is free. You pay for the instances it launches and for CloudWatch features such as detailed monitoring. Set a sensible maximum and a budget alert to avoid unexpected AWS charges. Without them, a traffic spike, a bug or a bot can scale costs up quickly.
Beyond EC2
Application Auto Scaling extends the same idea to other services, including ECS services, DynamoDB tables, Aurora replicas, Spot Fleets and Lambda provisioned concurrency.
Common misconceptions
- Auto Scaling is instant. New instances need time to boot and warm up, often several minutes.
- Scalability and elasticity are the same. Elasticity is automatic adjustment to demand. Scalability is the ability to grow.
- CPU is always the right metric. Choose the metric that matches the work.
- Desired capacity can go outside the limits. It cannot. The minimum and maximum bound it.
- Auto Scaling makes any app scalable. An app that keeps session state on one server still needs redesign.
Frequently asked questions
Does AWS Auto Scaling cost extra?
EC2 Auto Scaling is free. You pay for the EC2 instances it runs and for any CloudWatch features you enable, such as detailed monitoring.
Should I scale up or scale out?
Scale out for most web applications, because it can grow further and tolerates the loss of one instance. Scale up when the application cannot run across several instances, or for a short-term boost.
Why does Auto Scaling keep replacing my instances?
The most common cause is a health check grace period that is shorter than the application’s startup time. Increase the grace period, and check that your health check endpoint responds once the application is ready. For the broader idea of elastic capacity, read what is elasticity in cloud computing?