AWS CodePipeline — The Assembly Line for Shipping Software

AWS CodePipeline — The Assembly Line for Shipping Software

A deep, chapter-by-chapter walkthrough of AWS CodePipeline — how it is built, how a code change actually flows from commit to production, and how to run that flow reliably at scale.

Imagine a factory conveyor belt that carries a product through a sequence of inspection stations — one station checks the paint, another tests the wiring, another packs it into a box, and only a fully passed product reaches the loading dock. If any station rejects the product, the belt stops right there instead of shipping a broken item. AWS CodePipeline is that conveyor belt for software: it takes a code change, moves it automatically through build, test, and approval stations, and only lets it reach production once every station has signed off. This tutorial goes chapter by chapter through the intermediate-level machinery of AWS CodePipeline: its architecture, its internal behavior, its failure modes, and the decisions that separate a dependable, fast-moving delivery pipeline from a fragile one nobody trusts.

1Core Concepts, Refreshed

Before going deep into CodePipeline itself, a few continuous delivery concepts need to be sharp at an intermediate level.

Stages, Actions, and Artifacts

A pipeline is made of stages that run in order — for example Source, Build, Test, and Deploy. Each stage contains one or more actions, the actual units of work like “pull code from a repository” or “run a build project.” Artifacts are the files passed from one action to the next, such as source code moving from the Source stage into the Build stage, or a compiled package moving from Build into Deploy.

Continuous Integration vs. Continuous Delivery vs. Continuous Deployment

Continuous integration means automatically building and testing every code change as it is merged. Continuous delivery extends that by automatically preparing a release that is always ready to deploy, typically pausing for a human approval before production. Continuous deployment goes one step further and removes that manual gate entirely, pushing every passing change straight to production without a person clicking anything.

Simple Analogy

Continuous integration is like a chef tasting every dish as it’s cooked. Continuous delivery is having the finished, plated dish ready at the pass, waiting for the head chef’s nod before it goes to the table. Continuous deployment is skipping that nod and sending the dish straight out the moment it’s plated.

Component

Pipeline

The full, ordered workflow of stages that a change moves through from source to deployment.

Component

Stage

A named phase (like Build or Deploy) containing one or more actions that run together.

Component

Action

A single unit of work inside a stage, such as invoking a build project or a deployment target.

Component

Artifact

The files handed off between stages, stored in an S3 artifact bucket behind the scenes.

2Architecture & Components

CodePipeline itself is an orchestrator — it does not build or test code directly, but coordinates other services that do.

A typical pipeline starts with a Source stage connected to a repository such as CodeCommit, GitHub, Bitbucket, or an S3 bucket, watching for new commits or uploaded objects. From there, a Build stage commonly invokes AWS CodeBuild to compile code and run automated tests, though CodePipeline can also call third-party build tools through partner actions. A Deploy stage then pushes the built artifact out using a target service like AWS CodeDeploy, Amazon ECS, AWS Elastic Beanstalk, or CloudFormation.

graph LR
    A[Source Stage - CodeCommit/GitHub] --> B[Build Stage - CodeBuild]
    B --> C[Test Stage - Automated Tests]
    C --> D[Manual Approval Stage]
    D --> E[Deploy Stage - CodeDeploy/ECS/CloudFormation]
    F[(S3 Artifact Bucket)] -.stores artifacts.-> A
    F -.stores artifacts.-> B
    F -.stores artifacts.-> C
        
FIG 1 — A pipeline moves artifacts through stages, backed by an S3 artifact store behind the scenes

Every artifact produced by one stage is stored in an Amazon S3 bucket that CodePipeline manages as the handoff point between stages — this is why a Build stage doesn’t need to know anything about how the Deploy stage will use its output; it simply writes an artifact, and CodePipeline makes it available to whatever comes next.

V1 Pipeline Type

The original pipeline structure with fixed stage and action limits. Still supported, and adequate for straightforward pipelines.

V2 Pipeline Type

Adds higher limits, variables that can be referenced across stages, and pay-per-execution pricing. Best for more complex or higher-volume delivery workflows.

3Internal Working

Understanding how CodePipeline actually triggers and sequences work removes a lot of the mystery around “why did my pipeline run twice” or “why is it stuck.”

A pipeline execution begins when its source is triggered — typically a webhook event from a Git provider, or an Amazon EventBridge rule watching for a new commit or a new object in S3. CodePipeline then walks through stages strictly in order: every action inside a stage must succeed before the next stage begins, and if any action fails, the entire pipeline execution stops at that point rather than continuing forward with a broken artifact.

Simple Analogy

CodePipeline behaves like a relay race where each runner (stage) must fully finish and hand off the baton (artifact) before the next runner starts. If a runner trips and drops the baton, the race stops there — nobody downstream keeps running with an imaginary baton.

Only one execution of a given pipeline typically runs its stages at a time in the default configuration, meaning a second commit arriving while a pipeline is already mid-run will queue behind it rather than racing it through the same stages simultaneously — though superseding behavior can let a newer execution skip ahead of a stale, already-outdated one in later stages.

Manual approval actions pause the pipeline entirely, sending a notification (often through Amazon SNS) and waiting indefinitely — or until a configured timeout — for a human to approve or reject before the pipeline proceeds or stops.

!
Common Misconception

A pipeline stage passing does not mean the deployed application is actually healthy in production — it means the specific checks configured in that stage passed, which is only as thorough as the tests and validations you actually built into it.

4Data Flow & Lifecycle

A code change’s journey through CodePipeline follows a consistent lifecycle from commit to running in production.

1

Trigger

A commit, pull request merge, or new artifact upload triggers a new pipeline execution.

2

Source Retrieval

CodePipeline pulls the exact code revision and stores it as the initial artifact.

3

Build & Test

The code is compiled, unit-tested, and often scanned for security or quality issues, producing a new artifact.

4

Approval (optional)

A human reviewer approves or rejects the release before it reaches sensitive environments.

5

Deployment

The final artifact is deployed to staging, then production, often using a gradual rollout strategy.

Many pipelines repeat the Build-through-Deploy sequence twice — once against a staging environment, and again against production — so the exact same artifact that passed staging is the one promoted forward, rather than rebuilding from source a second time and risking a subtly different result.

EnvironmentPurposeTypical Gate Before It
DevelopmentFast feedback for engineersAutomated unit tests
StagingRealistic pre-production validationIntegration tests, security scans
ProductionLive traffic serving real usersManual approval, canary metrics

5Advantages, Disadvantages & Trade-offs

Choosing CodePipeline over a third-party CI/CD platform, or over fully manual deployment, involves real trade-offs.

Advantages

  • No CI/CD server infrastructure to provision, patch, or scale yourself
  • Deep native integration with CodeBuild, CodeDeploy, CloudFormation, ECS, and Lambda
  • Fine-grained IAM control over who can trigger, approve, or modify pipelines
  • Built-in support for manual approval gates and cross-account deployments
  • Pay-per-execution pricing on the V2 pipeline type suits variable workloads

Disadvantages / Trade-offs

  • Less rich plugin ecosystem than some dedicated third-party CI/CD platforms
  • Complex pipeline logic (dynamic fan-out, advanced conditionals) can be harder to express than in general-purpose scripting-based CI tools
  • Debugging a stuck or failed execution sometimes requires digging through multiple linked service consoles
  • Cross-region or cross-account setups add IAM and artifact-bucket configuration overhead
“A pipeline you don’t trust gets bypassed the first time it’s inconvenient.”

6Performance & Scalability

Pipeline speed is mostly determined by how the stages are structured, not by CodePipeline’s own overhead.

Actions within the same stage can run in parallel if they don’t depend on each other’s output, which is a common way to speed up a Build stage — for example running unit tests and a security scan side by side instead of one after the other. Sequential stages, by contrast, are inherently serial, since each depends on the artifact the previous stage produced.

Simple Analogy

Parallel actions are like two chefs working on different parts of the same dish at once. Sequential stages are like courses in a meal — you can’t plate dessert before the main course has actually been cooked and served.

For CodeBuild-backed stages, choosing a larger compute instance type reduces build time for CPU-heavy compilation or test suites, and enabling build caching avoids re-downloading dependencies on every single run. At the pipeline level, splitting one enormous pipeline into smaller, independently triggered pipelines per service is a common scaling pattern for large organizations with many microservices.

Parallel
Actions inside one stage can run concurrently
Serial
Stages always run strictly in order
Per-service
Pipelines scale by splitting per microservice
i
Tip

Move slow, less critical checks (like extended integration suites) to run in parallel with faster required checks, rather than stacking every check sequentially and stretching total pipeline time unnecessarily.

7High Availability & Reliability

A delivery pipeline needs to be reliable in two directions — CodePipeline staying available, and deployments themselves not breaking production.

As a fully managed, multi-Availability-Zone service, CodePipeline’s own control plane does not require you to manage redundant infrastructure. The reliability work that falls to engineering teams is designing safe deployment strategies: rolling deployments replace instances gradually, blue/green deployments stand up a fully separate new environment before switching traffic over, and canary deployments send a small percentage of traffic to the new version before a full rollout.

sequenceDiagram
    participant P as Pipeline
    participant D as Deploy Action
    participant Prod as Production Fleet
    P->>D: Trigger deployment
    D->>Prod: Deploy to 10% of instances (canary)
    Prod-->>D: Health metrics
    alt Metrics healthy
        D->>Prod: Continue rollout to 100%
    else Metrics degraded
        D->>Prod: Roll back automatically
    end
        
FIG 2 — A canary deployment rolls back automatically if health metrics degrade
Strategy

Rolling Deployment

Replaces instances a few at a time, keeping some capacity always serving traffic.

Strategy

Blue/Green Deployment

Stands up a full parallel environment and switches traffic over once it’s verified healthy.

Strategy

Canary Deployment

Sends a small slice of traffic to the new version before committing to a full rollout.

Strategy

Automatic Rollback

Reverts to the previous known-good version automatically when health checks fail post-deploy.

8Security

A pipeline that can deploy to production is, by definition, a powerful and sensitive piece of infrastructure that deserves careful access control.

IAM Roles and Permissions

Each pipeline uses an IAM service role that determines exactly what it is allowed to touch — which source repositories, which build projects, and which deployment targets. Scoping this role narrowly means a compromised or misconfigured pipeline cannot reach far beyond what it actually needs to do its job.

Cross-account Deployments

Larger organizations often run a central pipeline account that deploys into separate development, staging, and production AWS accounts, using cross-account IAM roles rather than one shared account holding every environment — limiting the blast radius if any single account is compromised.

Secrets and Approval Gates

Sensitive values like API keys and database credentials should be stored in AWS Secrets Manager or Systems Manager Parameter Store and referenced at build or deploy time, rather than hardcoded into pipeline configuration or source code. Manual approval stages also serve a security function — they are a natural checkpoint for confirming a change is expected before it reaches production.

ANTI-PATTERN-01 Avoid
Problem

Giving a pipeline’s IAM role broad administrator-level permissions “so it never fails due to a permissions error.”

Why It’s Harmful

A pipeline with administrator access becomes a single point of catastrophic risk — any bug, misconfiguration, or compromised dependency in the pipeline can affect the entire AWS account, not just the intended deployment target.

Correct Approach

Grant the pipeline’s role only the specific permissions its actions actually require, expanding scope deliberately rather than defaulting to broad access.

9Monitoring, Logging & Metrics

Pipeline visibility is what turns “a deployment happened” into “we know exactly what happened, when, and why it succeeded or failed.”

CodePipeline emits execution state changes to Amazon EventBridge, which teams commonly wire into Slack or email notifications so the right people hear about failures immediately rather than discovering them later. CodeBuild logs stream to Amazon CloudWatch Logs, giving engineers the full build and test output for debugging a failure without needing separate log aggregation tooling.

Metric

Pipeline Execution Success Rate

The percentage of executions that reach the final stage without failing — a core delivery health signal.

Metric

Stage Duration

How long each stage takes, useful for spotting a build or test stage that has quietly grown slow over time.

Metric

Deployment Frequency

How often changes reach production — a widely used indicator of delivery velocity.

Metric

Change Failure Rate

The percentage of deployments that require a rollback or hotfix, reflecting overall pipeline quality.

i
Tip

Alert on stage duration creeping upward over weeks, not just on outright failures — a build stage that slowly grows from five minutes to twenty-five is a productivity problem long before it becomes an outage.

10Deployment & Cloud Integration

CodePipeline is designed to sit at the center of a larger AWS delivery toolchain rather than working alone.

Pipelines are defined through the console, CLI, CloudFormation, CDK, or Terraform, which lets the pipeline definition itself be version-controlled alongside application code — a practice often called “pipeline as code.” Depending on the deployment target, the final stage might invoke CodeDeploy for EC2 or on-premises servers, update an ECS service for containerized applications, deploy a Lambda function directly, or apply a CloudFormation stack update for full infrastructure changes.

flowchart LR
    A[Git Push] --> B[CodePipeline - Source]
    B --> C[CodeBuild - Build & Test]
    C --> D[Manual Approval]
    D --> E{Deployment Target}
    E --> F[Amazon ECS]
    E --> G[AWS Lambda]
    E --> H[CloudFormation Stack]
        
FIG 3 — A pipeline branching to different deployment targets depending on the application type

Third-party integrations extend the pipeline further — connecting to Jira for change tracking, Slack for notifications, or a security scanning tool as a custom action — so the pipeline becomes the coordination point for the entire release process, not just the code deployment step.

11Design Patterns & Anti-patterns

Certain patterns show up again and again in mature CodePipeline setups — and so do certain mistakes.

Pipeline as Code

Defining the pipeline itself in CloudFormation or CDK alongside the application it deploys, so changes to the delivery process go through the same review process as application code.

Promote, Don’t Rebuild

Building an artifact once and promoting that exact same artifact through staging and production, rather than rebuilding from source at each stage and risking subtle differences between what was tested and what ships.

Fan-out Pipeline per Microservice

Giving each independently deployable service its own pipeline, so a slow or broken build in one service never blocks releases for unrelated services.

ANTI-PATTERN-02 Avoid
Problem

Rebuilding the application from source separately for staging and production instead of promoting a single, already-tested artifact.

Why It’s Harmful

Rebuilding introduces the possibility that a dependency version changed between the two builds, meaning what actually runs in production was never truly the same thing that passed staging tests.

Correct Approach

Build the deployable artifact once, store it, and reuse that exact artifact across every downstream environment in the pipeline.

12Best Practices & Common Mistakes

Most pipeline-related incidents trace back to a handful of recurring, avoidable oversights.

Best Practice

Fail fast, fail early

Put quick, cheap checks (linting, unit tests) before slow, expensive ones (full integration suites) so obvious problems surface sooner.

Best Practice

Gate production with real signals

Require actual health metrics, not just “the deployment command returned success,” before completing a rollout.

Mistake

No rollback plan

Building a pipeline that can deploy forward with no automated or well-rehearsed way to reverse a bad release.

Mistake

Skipping the staging environment under pressure

Manually pushing a “quick hotfix” straight to production outside the pipeline, bypassing every safeguard it was built to provide.

!
Common Mistake

Treating a manual approval stage as a formality that gets rubber-stamped without review — an approval gate only adds safety if the approver actually checks something before clicking approve.

13Real-world & Industry Examples

The continuous delivery model behind CodePipeline underpins release practices across very different kinds of organizations.

Financial Services Firms

Banks use CodePipeline with strict manual approval gates and audit logging to satisfy regulatory requirements while still deploying software changes far more often than manual release processes would allow.

Media Streaming Platforms

Streaming companies use per-microservice pipelines with canary deployments, so a bad release affects a small percentage of viewers briefly instead of the entire platform at once.

Enterprise SaaS Vendors

SaaS companies use cross-account pipelines to promote the same tested build through development, staging, and multiple customer-facing production environments in sequence.

What these examples share is not the specific industry, but the shape of the problem: the need to ship software changes frequently and safely, with a repeatable, auditable process that catches mistakes before they reach real users.

14Frequently Asked Questions

A few questions come up in nearly every team’s first serious CodePipeline evaluation.

Q1Does CodePipeline require using other AWS developer tools like CodeBuild and CodeDeploy?

No — while CodePipeline integrates deeply with AWS’s own build and deploy services, it also supports third-party tools and custom actions, so teams can mix AWS-native and external tools within the same pipeline.

Q2What happens if a stage fails partway through a pipeline execution?

The pipeline execution stops at the failed action, later stages do not run, and the pipeline reports a failed status, typically triggering a notification so the team can investigate before retrying or fixing the issue.

Q3Can CodePipeline deploy to multiple AWS accounts or regions?

Yes, cross-account and cross-region deployments are supported through IAM roles configured for cross-account access and replicated artifact buckets in each target region.

Q4How is CodePipeline priced?

Pricing depends on the pipeline type — the V1 type charges a flat monthly fee per active pipeline, while the V2 type charges per pipeline execution, which can be more cost-effective for pipelines that run infrequently.

Q5Is manual approval required in every pipeline?

No, manual approval is optional and typically reserved for sensitive environments like production, while lower environments often deploy fully automatically with no human gate at all.

15Summary and Key Takeaways

AWS CodePipeline takes the repeatable sequence of building, testing, and deploying software and turns it into a managed, automated workflow that runs the same way every single time. The underlying engineering decisions — what tests actually matter, which deployment strategy fits the risk profile, where a human should be in the loop — remain the team’s responsibility, because CodePipeline provides the conveyor belt, not the judgment about what quality bar each station should enforce. Understanding stages, artifacts, deployment strategies, and IAM scoping is what separates a delivery pipeline teams trust from one people quietly route around.

Key Takeaways

  • Stages run in strict order, actions can run in parallel — understanding this shapes how to design for speed without sacrificing safety.
  • Build once, promote everywhere — the artifact that passes staging should be the exact one deployed to production.
  • Deployment strategy is a reliability decision — rolling, blue/green, and canary each trade off risk, cost, and rollout speed differently.
  • IAM scope should match actual need — a pipeline’s permissions should never exceed what its specific actions require.
  • Manual approvals only help if they’re meaningful — a rubber-stamped gate provides no real safety.
  • Monitoring delivery metrics matters as much as monitoring infrastructure — success rate, duration, and change failure rate reveal pipeline health over time.
  • Pipeline as code keeps delivery reviewable — versioning the pipeline definition itself prevents undocumented, ad-hoc changes to how software ships.