AWS CodeDeploy at Scale: The Expert's Guide to Traffic Shifting, Lifecycle Hooks, and Rollback
A deep, production-grade walkthrough of how AWS CodeDeploy actually behaves once you move past a single in-place EC2 deployment — blue/green traffic shifting mechanics across EC2, Lambda, and ECS, lifecycle event hook ordering, alarm-driven automatic rollback, and the failure modes that only surface once a deployment is touching live production traffic.
If you have already pushed a revision through a basic in-place EC2 deployment, you know the demo story. What almost nobody tells you is what happens once a bad deployment starts throwing errors under real traffic, your alarm-based rollback fires halfway through a linear traffic shift, and you need to understand exactly which lifecycle hook ran, which didn’t, and why the old and new environments were briefly serving traffic simultaneously. This guide skips the introductory tour entirely and goes straight into how CodeDeploy orchestrates deployments internally across its three compute platforms, and how production teams design deployment strategies that fail safely.
AAdvanced Core Concepts
We skip what a deployment or an application revision is in general. Instead, we look at the mechanics that only matter once you are choosing a deployment strategy deliberately: the three compute platforms, in-place versus blue/green deployment types, and how deployment configurations actually control blast radius.
Three Compute Platforms, Three Different Deployment Mechanics
CodeDeploy supports EC2/On-Premises, AWS Lambda, and Amazon ECS as distinct compute platforms, and each one implements deployment fundamentally differently under the same conceptual umbrella. EC2/On-Premises deployments install a new application revision onto existing instances (in-place) or provision a parallel fleet (blue/green) using an agent running on each instance. Lambda deployments shift invocation traffic between function versions using weighted aliases, with no agent or instance concept at all. ECS deployments shift traffic between two parallel task sets behind a load balancer, again with no agent involved. Because the underlying mechanics differ so much, deployment configurations, lifecycle hooks, and even what “rollback” means are platform-specific, not universal concepts you can reason about identically across all three.
Think of the three compute platforms as three different kinds of stage changeovers. EC2 in-place is like renovating a shop while it stays open — customers keep shopping while workers update sections one at a time. EC2 and ECS blue/green is like building an entirely new shop next door, and only redirecting customers to it once it’s fully ready. Lambda traffic shifting is like gradually redirecting a percentage of phone calls to a new call center team, dialing the percentage up over time rather than switching everyone at once.
In-Place Versus Blue/Green: What Actually Changes
An in-place deployment updates the application on the currently running compute — instances are taken out of load balancer rotation, updated, validated, and returned to service, in batches. A blue/green deployment instead provisions or designates an entirely separate, parallel environment (a new Auto Scaling group, a new ECS task set, or a new Lambda alias target), deploys the new revision there, and only then shifts traffic over — leaving the original environment intact and immediately available for a fast rollback simply by shifting traffic back, rather than needing to redeploy the previous revision.
Blue/green rollback is fast precisely because the old environment was never destroyed — it is still running, just not receiving traffic. In-place rollback, by contrast, requires redeploying the previous revision onto the same instances, which takes as long as any other deployment. Teams that need genuinely fast rollback should not assume in-place deployments give them that property.
Deployment Configurations: Controlling Blast Radius
A deployment configuration defines how quickly and how much of your fleet or traffic is updated at once. For EC2/On-Premises, this means minimum healthy host percentages or counts (OneAtATime, HalfAtATime, AllAtOnce, or a custom percentage). For Lambda and ECS, it means traffic-shifting patterns — Canary (a small percentage shifts first, then the rest after a defined wait), Linear (traffic shifts in equal increments at fixed intervals), or AllAtOnce. The deployment configuration is the single biggest lever controlling how much production traffic is exposed to a bad revision before an operator or an automated rollback trigger can react.
BInternal Working
What happens inside AWS’s infrastructure and, for EC2, inside the CodeDeploy agent itself, during a deployment.
The CodeDeploy Agent Polling Model
For EC2/On-Premises deployments, a CodeDeploy agent runs on each target instance and does not receive pushed commands from the CodeDeploy service. Instead, it polls the CodeDeploy control plane for pending deployment work, and when it finds an assigned deployment, it pulls the application revision from S3 or GitHub, and executes the lifecycle event hooks defined in the application specification file locally, on that instance, reporting status back to the control plane after each event. This pull-based model is why an agent that has stopped running or lost network connectivity fails silently from CodeDeploy’s perspective until a health check or timeout surfaces the problem, rather than CodeDeploy immediately knowing the instance is unreachable.
flowchart LR
CD[CodeDeploy Control Plane] -->|deployment assigned| AGENT[CodeDeploy Agent on Instance]
AGENT -->|pull revision| S3REV[(S3 / GitHub Revision)]
AGENT --> BI[BeforeInstall]
BI --> INSTALL[Install]
INSTALL --> AI[AfterInstall]
AI --> AS[ApplicationStart]
AS --> VS[ValidateService]
VS -->|status report| CD
Traffic Shifting for Lambda and ECS
Lambda traffic shifting works by adjusting the weighted routing configuration on a function alias, incrementally moving invocation percentage from the old version to the new one according to the canary or linear pattern, with CodeDeploy itself managing the alias weight updates on a timer. ECS blue/green works by having two parallel task sets running behind a load balancer’s target groups, with CodeDeploy incrementally shifting the load balancer’s routing weight from the original task set to the new one, and only terminating the original task set after the deployment is confirmed successful (or immediately restoring full weight to it on rollback).
CData Flow & Lifecycle
Tracing a deployment’s complete life, from a triggered revision to a confirmed, stable rollout.
Revision Registered
A new application revision (a bundle referencing the application specification and, for EC2, the application files) is registered from S3 or GitHub.
Deployment Created
A deployment is created against a deployment group, specifying the deployment configuration that governs rollout pace and blast radius.
Lifecycle Hooks Executed
Platform-specific lifecycle events run in strict order — for EC2 this includes BeforeInstall, Install, AfterInstall, ApplicationStart, and ValidateService.
Traffic Shifted or Instances Rotated
For blue/green or traffic-shifting platforms, traffic moves from old to new according to the configured pattern; for in-place, instances are updated and returned to load balancer rotation in batches.
Validated
Health checks, CloudWatch alarms, or a custom validation hook confirm the new revision is behaving correctly under real or shifted traffic.
Completed or Rolled Back
The deployment either completes successfully, or an automatic rollback trigger reverts traffic and instance state to the previous known-good configuration.
Why Lifecycle Hook Ordering Is Non-Negotiable
Each lifecycle event exists for a specific purpose the platform enforces strictly — ApplicationStop always runs before BeforeInstall on a redeployment to the same instance, ensuring the old process is stopped before new files are installed, and ValidateService always runs last, after ApplicationStart, so that validation logic checks a genuinely running new version rather than a partially started one. Skipping or misusing a hook (running validation logic inside AfterInstall instead of ValidateService, for instance) can mean validation runs before the application is actually accepting traffic, producing false-positive successful deployments.
DAdvantages, Disadvantages & Trade-offs
Advantages
- Unified deployment orchestration across EC2, Lambda, and ECS reduces the need for separate bespoke deployment tooling per compute platform.
- Native alarm-based automatic rollback ties deployment safety directly to real production health signals rather than a fixed timer.
- Blue/green deployments provide near-instant rollback by simply reverting traffic weight, without redeploying anything.
- Deep CodePipeline integration lets deployment become a single, consistent stage across many services and pipelines.
Disadvantages
- EC2 deployments depend on an agent running correctly on every instance, adding an operational dependency that must itself be monitored and kept updated.
- Lifecycle hook semantics and available deployment configurations differ meaningfully across the three compute platforms, so expertise on one platform does not fully transfer to another.
- Blue/green deployments on EC2 or ECS require provisioning parallel capacity, at least temporarily doubling compute cost during the deployment window.
- Alarm-based rollback is only as good as the alarms configured — a deployment can complete “successfully” while silently degrading a metric nobody is watching.
The Central Trade-off: Deployment Safety Versus Speed and Cost
Every safety mechanism CodeDeploy offers — canary shifting, linear shifting, blue/green parallel capacity, alarm-based rollback — trades deployment speed or infrastructure cost for reduced blast radius when something goes wrong. AllAtOnce deployments are the fastest and cheapest but expose one hundred percent of traffic to a bad revision immediately; canary and blue/green strategies are slower and, for blue/green, temporarily more expensive, but they bound how much of production is ever affected by a failed deployment. Choosing a strategy is fundamentally a decision about how much risk a given service can tolerate, not a purely technical preference.
EPerformance & Scalability
How deployment configuration choices interact with fleet size and traffic volume at real production scale.
Batch Size Versus Deployment Duration
Smaller batch sizes (OneAtATime, or a low custom minimum healthy percentage) take proportionally longer to complete a full fleet rollout, since only a small fraction of instances update at once, but they also limit how many instances can be simultaneously affected by a bad revision. Very large fleets deploying with a conservative batch size can take a meaningfully long time to fully roll out, which is a deliberate trade production teams accept for safety on critical services, while less critical or very large low-risk fleets often use a more aggressive percentage to keep deployment windows shorter.
Traffic-Shifting Wait Intervals
Canary and linear configurations for Lambda and ECS include a wait interval between traffic percentage increases, giving time for metrics and alarms to reflect the new version’s actual behavior before shifting further. Setting this interval too short risks shifting most traffic before a genuine problem has had time to manifest in your monitoring; setting it unnecessarily long extends total deployment time without meaningfully improving safety once alarms have had adequate time to evaluate.
A deployment configuration that worked fine at a small fleet size can behave very differently at ten times the instance count — the same percentage-based minimum healthy host setting translates to a much larger absolute number of instances being updated simultaneously as fleets grow, which is worth revisiting rather than assuming a configuration remains appropriately conservative forever.
FHigh Availability & Reliability
CodeDeploy’s control plane runs as managed, multi-AZ AWS infrastructure with no servers for you to operate directly. Reliability of the deployment mechanism itself — hook execution, traffic shifting, status tracking — is AWS’s responsibility, but reliability of the deployment outcome depends heavily on decisions you make: deployment configuration choice, alarm coverage, and lifecycle hook correctness.
Automatic Rollback Triggers
CodeDeploy supports automatic rollback on deployment failure (a lifecycle hook returning a failure status) and, separately, automatic rollback triggered by one or more specified CloudWatch alarms entering an alarm state during the deployment window. These are complementary, not redundant: a deployment can complete every lifecycle hook successfully while a business metric like error rate or latency still degrades under real traffic, which is exactly the scenario alarm-based rollback exists to catch that hook-based rollback cannot.
Reliability in Practice: Meaningful Health Checks in ValidateService
A ValidateService hook that only checks whether a process is running, without verifying it can actually serve a real request successfully, provides a false sense of safety. Production-grade validation hooks typically make an actual synthetic request against the newly deployed version and check for a genuinely correct response before reporting success.
GSecurity
IAM: Service Role Scope
CodeDeploy uses a dedicated IAM service role to perform deployment actions on your behalf — registering instances, updating load balancer target group weights, shifting Lambda alias traffic, or updating ECS task sets — and this role’s permissions should be scoped tightly to exactly the resources a given application’s deployment group needs to touch, rather than granted broad account-wide deployment permissions that would let one application’s pipeline accidentally affect another’s infrastructure.
Revision Source Integrity
Because application revisions are pulled from S3 or GitHub by the CodeDeploy agent or service, controlling who can write to the source S3 bucket or push to the linked GitHub repository is a direct security boundary — anyone able to place a malicious revision bundle in that location can have it deployed to production infrastructure, making S3 bucket policies and GitHub repository access controls a meaningful part of the deployment security surface, not just an incidental detail.
Instance-Level Permissions for the EC2 Agent
The CodeDeploy agent running on an EC2 instance operates under that instance’s own IAM instance profile, which must be granted permission to read from the revision source and report status back to CodeDeploy. Over-broad instance profile permissions here are a common oversight, since the instance profile is frequently reused across many purposes beyond just deployment, inheriting more access than the deployment function alone requires.
HMonitoring, Logging & Metrics
The signals that actually predict a bad deployment before it becomes a customer-facing incident.
Deployment Status Events
CodeDeploy emits status events (Created, InProgress, Succeeded, Failed) per deployment, and per lifecycle event, which should feed directly into deployment dashboards rather than being checked manually.
Configured CloudWatch Alarms
The alarms attached to a deployment group’s automatic rollback configuration are the real-time production health signals that can halt a bad rollout mid-flight — their coverage quality directly determines rollback effectiveness.
CodeDeploy Agent Status (EC2)
An instance with a stopped or unreachable agent will silently fail to receive deployments, a distinct failure mode from an application-level deployment failure that is easy to misdiagnose without checking agent status directly.
CloudTrail API Events
Every deployment creation, configuration change, and rollback action is logged to CloudTrail, providing an audit trail of who triggered or configured a given production deployment.
Notifications and Human-in-the-Loop Awareness
CodeDeploy can publish deployment lifecycle events to Amazon SNS, commonly wired into chat notification channels so that engineers are aware a deployment is in progress, succeeded, or triggered an automatic rollback, without needing to actively poll the console — a small but meaningful reliability practice for organizations deploying frequently across many services.
IDeployment & Cloud Integration
CodeDeploy is most commonly one stage inside a CodePipeline definition, receiving a build artifact from a preceding CodeBuild stage and deploying it to a target deployment group, with pipeline-level approval gates optionally inserted before a production deployment stage runs.
flowchart LR
CB[CodeBuild - Build Artifact] --> APPROVAL[Manual Approval Gate]
APPROVAL --> CDDEPLOY[CodeDeploy Stage]
CDDEPLOY --> BLUE[Blue Environment - Current]
CDDEPLOY --> GREEN[Green Environment - New]
BLUE -.traffic before.-> ALB[Load Balancer / Target Group]
GREEN -.traffic after shift.-> ALB
ALB --> USERS[End Users]
Load Balancer Integration for Blue/Green
Blue/green deployments on EC2 and ECS rely on Application Load Balancer or Network Load Balancer target groups as the actual traffic-shifting mechanism — CodeDeploy does not move traffic by any means other than updating which target group (or which weighted target group split) the load balancer is routing to, which is why correct target group and listener configuration is a prerequisite for blue/green deployments to function at all, not an optional enhancement.
JDesign Patterns & Anti-Patterns
Context
A customer-facing Lambda or ECS service needs deployment safety proportional to the business impact of a bad revision reaching production traffic.
Decision
Use a canary traffic-shifting configuration paired with automatic rollback triggers tied to error rate and latency alarms, rather than an AllAtOnce configuration with no alarm coverage.
Consequence
A bad revision is exposed to only a small percentage of traffic before an automatic rollback reverts it, at the cost of a longer total deployment window.
Context
Teams place health-check or smoke-test logic inside AfterInstall instead of ValidateService, often because it was convenient during initial setup.
Problem
AfterInstall runs before ApplicationStart, meaning validation logic executes before the new application version is actually running and able to serve requests.
Consequence
False-positive successful deployments where a genuinely broken application still reports as validated and healthy.
Pattern: Reused Deployment Groups Per Environment
Maintaining separate, clearly named deployment groups per environment (staging, production) rather than a single deployment group with environment logic embedded in scripts keeps blast radius, permissions, and rollback configuration cleanly isolated per environment.
KBest Practices & Common Mistakes
Best Practices
- Attach real production health alarms to automatic rollback configuration for any customer-facing service.
- Put genuine, request-level validation logic in ValidateService, not an earlier hook.
- Scope each application’s CodeDeploy service role tightly to only the resources its own deployment group touches.
- Revisit deployment configuration batch sizes as fleet size grows rather than assuming a percentage stays appropriately conservative.
- Monitor CodeDeploy agent health on EC2 fleets separately from application-level health.
Common Mistakes
- Assuming AllAtOnce is safe for critical customer-facing services simply because it is the fastest option.
- Placing validation logic in the wrong lifecycle hook, producing false-positive successful deployments.
- Leaving automatic rollback unconfigured, relying entirely on manual observation to catch a bad deployment.
- Granting an overly broad CodeDeploy service role that can affect infrastructure beyond the intended application.
- Not accounting for the temporary doubled compute cost of blue/green deployments in capacity or budget planning.
LReal-World & Industry Examples
Serverless APIs — Canary Deployments for Lambda
Teams operating customer-facing Lambda-backed APIs commonly use canary traffic shifting paired with error-rate alarms, so a regression in a new function version is caught and rolled back automatically after only a small fraction of API calls have been affected.
Container Platforms — ECS Blue/Green for Zero-Downtime Releases
Organizations running ECS services commonly use blue/green deployments specifically to guarantee zero-downtime releases, since the old task set remains fully running and able to instantly reabsorb traffic if the new task set fails validation.
Traditional EC2 Fleets — Conservative In-Place Rollouts
Organizations with large, long-lived EC2 fleets commonly use conservative in-place deployment configurations (a low custom minimum healthy percentage) for critical services, accepting a longer deployment window in exchange for keeping the vast majority of the fleet serving traffic at any given moment during rollout.
“A rollback strategy you haven’t tested is a rollback strategy you don’t actually have — blue/green only pays off if the old environment was truly left ready to take traffic back.”
MFrequently Asked Questions
NSummary and Key Takeaways
What to Remember
- The three compute platforms deploy fundamentally differently. Expertise on one does not automatically transfer to another.
- Blue/green rollback is fast because the old environment never stops running; in-place rollback takes as long as a normal deployment.
- Deployment configuration is the primary blast-radius control — batch size and traffic-shift pattern determine how much of production is exposed to a bad revision.
- Lifecycle hook ordering is strict and purposeful; validation logic belongs in the hook designed for it, not wherever is convenient.
- Automatic rollback needs real alarms attached to it — hook success alone cannot catch a metric-level production regression.
- The load balancer, not CodeDeploy itself, performs the actual traffic shift for blue/green EC2 and ECS deployments.
- Deployment safety and deployment speed trade off directly. Choosing a strategy is a risk decision, not a purely technical one.