Azure Pipelines
A complete, plain-English guide to the service that builds, tests, and delivers your software automatically: how a run really executes, how to keep it fast, secure, and reliable, and how to run it well in production.
Imagine a bakery that sells bread to a whole city. Every morning the bakers could mix the dough by hand, check each loaf with their own eyes, wrap it, and carry it to every shop on foot. It would work for a village. For a city, it would be slow, tiring, and full of mistakes. One tired baker forgets the salt, one loaf is undercooked, one shop gets the wrong delivery. So the bakery builds an assembly line: machines mix the dough in exact amounts, ovens bake on a timer, inspectors check every batch against a checklist, and vans leave for each shop in order, with a record of what went where. The bread is better, the shelves are full on time, and when something goes wrong, the line stops before bad loaves reach customers.
Software teams face exactly the same problem. Writing code is only the first step. The code must be assembled into a working program, tested, packaged, and delivered to the places where people use it, often many times a day. Azure Pipelines is Microsoft’s cloud service for building that assembly line. In this guide you will learn what it is, why it exists, every important part inside it, how one run travels from a code change to a live system, and the mistakes that trip up even experienced engineers. Each new word is explained the first time it appears, so you can begin with no background at all.
1What Azure Pipelines Is
Start with the big picture: what it does, where it lives, and the three letters that define it.
Azure Pipelines is a cloud service that automatically builds, tests, and deploys your code. It is one part of a bigger product family called Azure DevOps Services, which also includes places to store code (Repos), plan work (Boards), manage test cases (Test Plans), and share software packages (Artifacts). Pipelines can be used on its own, and it works with code stored almost anywhere: Azure Repos, GitHub, Bitbucket, or any other Git server. It also works with nearly any language and target: Java, .NET, Python, JavaScript, Go, mobile apps, containers, and more, on Windows, Linux, and macOS, deploying to Azure, other clouds, or your own servers.
Three terms sit at the heart of the idea. Continuous integration, shortened to CI, means that every time someone changes the code, the system automatically builds the program and runs tests, so problems are found within minutes instead of weeks. Continuous delivery means that every change that passes the tests is automatically prepared and made ready to release, so that releasing is a routine button press rather than a nervous event. Continuous deployment, the third idea, goes one step further and releases to users automatically without any button press. Both of the last two are abbreviated CD, which causes endless confusion, so context matters. Together the practice is called CI/CD.
CI is the quality inspector at the end of every shift, checking that today’s work fits with everyone else’s. Continuous delivery is the packing department that has every approved box wrapped and ready on the loading dock. Continuous deployment is the van that leaves for the shops by itself the moment the box is ready. Azure Pipelines is the assembly line and the foreman that runs all three.
A short history
In the early 2000s, teams built software on a single “build server” in a cupboard, often babysat by one person. Microsoft offered build tools in Team Foundation Server, a product installed on a company’s own machines. When the cloud arrived, Microsoft moved these ideas online as Visual Studio Team Services, and in 2018 renamed the family Azure DevOps, with Azure Pipelines as its build and release engine. Around the same time, Microsoft made Pipelines free for open source projects and added support for GitHub, which helped it spread far beyond the Microsoft world. Since then, the biggest shift has been from clicking together pipelines in a visual editor to writing them as text files that live next to the code. Microsoft also owns GitHub Actions, a similar service, and many organizations use one or both. We compare them later.
Prices, free allowances, limits, and agent images change over time. Use the ideas in this guide as a map and confirm exact numbers in Microsoft’s current documentation before you finalize a design.
2The Problem It Solves
Every tool exists because something hurt. Here is the pain that Azure Pipelines removes.
Picture a team of ten developers building a shopping website. Each person changes different parts of the code on their own computer. Without automation, a typical week looks like this. On Monday someone builds the program on their laptop and it works. On Wednesday a colleague builds it on theirs and it fails, because their laptop has a different version of some tool. On Friday the team joins everything together for the first time, and a hundred small conflicts appear at once, a painful event nicknamed integration hell. Releasing to customers happens once a month, late at night, with a long checklist of manual steps, and every release is a risk. When something breaks in production, nobody is quite sure which change caused it.
Automation fixes each of these. A pipeline builds the code the same way every time, on a clean machine, so “it works on my machine” stops being an excuse. It runs tests on every change, so problems are found while the change is still fresh in the author’s mind. It deploys using the same scripted steps to every environment, so releases are boring and repeatable. And it keeps a full record of what was built, tested, approved, and deployed, so every release can be traced back to the exact code that created it.
Works on my machine
Builds on different laptops give different results, so nobody trusts them.
Late, painful integration
Joining everyone’s work only once in a while produces big, tangled conflicts.
Scary manual releases
Long checklists done by hand cause mistakes and keep teams from releasing often.
No traceability
It is unclear what exactly is running in production and who approved it.
Everyday example: the pull request check
A developer proposes a change to the shopping cart. Within minutes, a pipeline checks out the code, builds it on a clean machine, runs two thousand tests, and reports a green tick or a red cross right on the proposal page. A teammate sees a failed test, tells the author, and the bug is fixed before it ever reaches the main code. Nobody had to remember to run anything.
3Core Concepts: The Pipelines Vocabulary
Azure Pipelines has a small set of building blocks. Learn these words and every diagram becomes easy to read.
The central object is the pipeline: the full automated process, from a code change to a result. Each time a pipeline starts, that is a run. A pipeline is organized as a set of boxes inside boxes. The biggest boxes are stages, the major phases such as Build, Test, and Deploy. Inside a stage are jobs, groups of work that run together on one machine. Inside a job are steps, the individual actions, one after another. A step is either a task, a ready-made, packaged action such as “run the .NET build” or “deploy to an Azure web app,” or a script, a few lines of your own commands.
| Term | Plain meaning | Bakery comparison |
|---|---|---|
| Pipeline | The whole automated process | The entire assembly line |
| Run | One execution of a pipeline | One day’s production |
| Stage | A major phase, like Build or Deploy | Mixing hall, baking hall, shipping dock |
| Job | Work done together on one machine | One team in one hall |
| Step | One action inside a job | One instruction on the recipe card |
| Task | A packaged, reusable step | A standard machine bought from a supplier |
| Agent | The machine that runs a job | The worker or station doing the job |
| Agent pool | A group of agents | A shift of workers |
| Trigger | The event that starts a run | The morning bell |
| Artifact | A file produced by a run | A finished, boxed loaf |
| Environment | A named place where you deploy | A particular shop on the delivery route |
| Service connection | A saved, secure link to an outside system | A signed delivery pass for the shop’s back door |
Think of making a school lunch in a big kitchen. The recipe for one lunch is the pipeline. The big phases, prepare, cook, and pack, are stages. Each cook’s area is a job. Every instruction like “slice the bread” is a step. The cooks themselves are the agents, and the bell that wakes the kitchen in the morning is the trigger.
Hold on to the split between definition and execution. The pipeline definition is only text describing what should happen. Nothing runs until a trigger fires, the service queues a run, and an agent picks up the work.
4Architecture and Components
Now let us see how the pieces connect, from your code to the system that runs it.
Azure Pipelines has two big sides. The service side, hosted by Microsoft in the cloud, is the brain. It stores your pipeline definitions, watches for triggers, decides what should run and in what order, keeps the queue of waiting jobs, stores logs and results, and handles permissions and approvals. The agent side is the muscle. An agent is a small program running on a machine, which could be a virtual machine that Microsoft manages for you or a computer of your own. It asks the service, “Do you have any work for me?” When there is a job waiting, it downloads the steps, executes them, and sends back the logs and results.
Two details are worth noting. First, the agent pulls work from the service. The service does not push into your network. For self-hosted agents, this means you only need to allow outbound connections to Azure DevOps, with no inbound holes in your firewall. Second, the service never needs to see your running application. It only coordinates, while the agent does the real work next to your code and your tools.
flowchart LR
DEV["Developer pushes code"] --> REPO["Code repository"]
REPO -->|"trigger event"| SVC["Azure Pipelines service"]
SVC --> Q["Job queue"]
Q --> POOL["Agent pool"]
POOL --> A1["Agent 1"]
POOL --> A2["Agent 2"]
A1 --> ART["Artifacts and packages"]
A2 --> ENV["Deployment environments"]
A1 -.->|"logs and results"| SVC
A2 -.->|"logs and results"| SVC
Read the diagram from left to right. A developer pushes code to the repository. The repository tells the service that something changed. The service checks the pipeline definition, creates a run, and puts the jobs in a queue. Idle agents in the matching pool pick jobs up, execute their steps, and send results back. Along the way, finished files are saved as artifacts, and deployment jobs reach out to the environments where the software will live.
Pipelines service
Stores definitions, watches triggers, queues jobs, and keeps logs and history.
Agents and pools
Programs on machines that pull jobs and run the steps.
Repositories
Azure Repos, GitHub, Bitbucket, or any Git server holding the code.
Artifacts and packages
Built files and software packages saved for later stages.
Environments
Named destinations such as test, staging, and production.
Service connections
Secure links to Azure, container registries, and other systems.
5YAML Pipelines and Classic Pipelines
There are two ways to describe a pipeline. One of them is now strongly preferred.
The modern way is a YAML pipeline. YAML is a simple, human-readable text format that uses indentation to show structure, a little like an outline. You write the pipeline as a text file, usually named azure-pipelines.yml, and store it in the same repository as your code. Because it is just a file in the repository, it enjoys all the benefits that code has. It has a full history, so you can see who changed the pipeline and why. It can be reviewed through a pull request like any other change. It lives on branches, so a new idea can be tried on a branch without disturbing the main line. And it travels with the code, so an old version of the code is built by the old version of the pipeline. This idea is called pipeline as code.
The older way is the classic editor, a visual designer where you click to add tasks and fill in forms. Classic pipelines come in two kinds: build pipelines, which compile and test, and release pipelines, which deploy. They are easy to start, but they live inside the service, not next to your code, so they are harder to review, copy, and reproduce. Microsoft encourages new work to use YAML, which now handles both build and deployment through multi-stage pipelines, and classic release pipelines are considered the legacy approach.
| Feature | YAML pipelines | Classic pipelines |
|---|---|---|
| Where the definition lives | In your repository as a text file | Inside the service as forms |
| Change history and review | Full, through pull requests | Limited history |
| Reuse through templates | Strong | Limited, with task groups |
| Build and deploy in one place | Yes, with multi-stage pipelines | Separate build and release pipelines |
| Learning curve | Needs to learn the format | Easier to start by clicking |
| Recommended for new work | Yes | Mainly for existing pipelines |
A classic pipeline is a recipe someone keeps in their head, and perhaps written on a sticky note inside a drawer in the kitchen. A YAML pipeline is a recipe printed in the shared cookbook, with a page history showing who changed the amount of salt and when. Anyone can read it, suggest a change, and cook the same dish.
The portal also offers a visual assistant that helps you write the YAML by clicking tasks and inserting the matching text. It is a gentle way to learn the format while keeping your pipeline as code.
6Stages, Jobs, and Steps in Depth
The three nested boxes decide what runs in sequence and what runs side by side.
A stage is a logical phase and the main way to organize a pipeline. By default, stages run one after another, but you can describe dependencies so that some stages run in parallel, or one stage waits for two others. Each stage can have conditions, such as “run only on the main branch,” and can contain approvals and checks, which we meet later.
A job is a collection of steps that run on one agent. Jobs in the same stage run in parallel by default, as long as enough agents are free, and you can add dependencies when order matters. A special job type, the deployment job, targets an environment and keeps a history of what was deployed. Every job starts with a clean slate on its agent, so a later job cannot see files from an earlier one unless you pass them through artifacts. This clean-slate rule surprises many beginners, and it is also what keeps builds repeatable.
A step is the smallest unit: a task or a script. Steps inside a job run strictly in order, and by default a failing step stops the job. A step can be marked to run even after a failure, which is useful for clean-up actions such as shutting down a test database. Jobs can also use a matrix strategy, a feature that repeats the same job for several combinations of settings, for example building on three versions of a language and two operating systems, six copies in all, running in parallel.
flowchart LR
subgraph B["Stage - Build"]
B1["Job - compile and unit test"]
B2["Job - code analysis"]
end
subgraph T["Stage - Test"]
T1["Job - integration tests"]
end
subgraph D["Stage - Deploy staging"]
D1["Deployment job"]
end
subgraph P["Stage - Deploy production"]
P1["Deployment job after approval"]
end
B --> T --> D --> P
Stage: Build
Compile the code, run quick tests, and publish the result as an artifact.
Stage: Test
Deploy to a temporary test area and run slower integration tests.
Stage: Staging
Release to a production-like environment for final checks.
Approval
A named person reviews and approves the move to production.
Stage: Production
Release to customers using a careful strategy and verify health.
Every job starts on a fresh agent state. If a later job needs the files built by an earlier job, publish them as an artifact and download them again. Relying on leftover files from a previous job is a common reason for pipelines that work on one run and fail on the next.
7Agents and Agent Pools
The agent is where your code really builds. Choosing the right kind of agent affects cost, speed, and security.
An agent pool is a group of agents. A job asks for a pool, and any free agent in that pool can take it. There are several kinds of agents.
Microsoft-hosted agents
These are virtual machines that Microsoft prepares and runs for you. They come with a wide set of common tools preinstalled: compilers, language runtimes, Docker, cloud tools, and more, available on Ubuntu Linux, Windows, and macOS images. Each job gets a brand-new machine that is thrown away when the job ends. This gives you clean, repeatable builds and removes any maintenance. The trade-offs are that you cannot install permanent software on them, they cannot reach servers sitting inside your private network by default, and they start from a cold state each time, so downloads and restores must be repeated unless you use caching.
Self-hosted agents
These are machines that you own and run, whether physical computers, virtual machines in Azure or elsewhere, or containers. You install the agent program and register it in a pool. Because you control them, they can have special tools, powerful hardware, graphics cards, or access to private networks and on-premises systems. They keep their disk between jobs, which can speed up builds by reusing downloaded files. The cost is that you must patch, secure, and scale them yourself.
Scale set agents and managed pools
For large teams, an agent pool backed by a virtual machine scale set automatically adds and removes self-hosted virtual machines as the queue grows and shrinks, and can reset each machine after use. A newer option, Managed DevOps Pools, lets Microsoft handle much of this scaling and maintenance while the machines run inside your own Azure subscription and network. This gives you private network access with less hands-on work.
| Option | Who runs the machines | Best for | Trade-off |
|---|---|---|---|
| Microsoft-hosted | Microsoft | Most builds, clean and simple | No private network access, cold start each job |
| Self-hosted | You | Special tools, private networks, big hardware | You patch and secure them |
| Scale set agents | You, with automatic scaling | Large, busy teams needing private access | More setup and cost control needed |
| Managed DevOps Pools | Microsoft, inside your subscription | Private network access with less upkeep | Newer option, check current features |
Microsoft-hosted agents are like renting a fully equipped, spotless kitchen by the hour. You walk in, cook, and leave, and someone else cleans it. Self-hosted agents are like your own restaurant kitchen: you can install any special oven you want and keep supplies on the shelves, but you also clean it, repair it, and pay for it even when it is idle.
A self-hosted agent that is reused by many jobs can leak files or secrets from one job to the next, and anyone who can change a pipeline can run commands on that machine. Keep untrusted pipelines, such as those from public forks, off your own agents.
8Triggers: What Starts a Run
A pipeline is only a description until something starts it. Triggers are the starters.
A trigger decides when a pipeline runs. Several kinds exist, and one pipeline can use more than one at the same time.
CI trigger
Starts a run when code is pushed to chosen branches or paths. This is the classic continuous integration trigger.
Pull request trigger
Starts a run when someone proposes a change, so tests run before the change is merged into the main code.
Scheduled trigger
Starts a run at set times, such as a full test suite every night, written with a cron-style schedule.
Pipeline completion trigger
Starts when another pipeline finishes, so a deployment pipeline can follow a build pipeline.
Container or package trigger
Starts when a new container image or package version appears.
Manual run
Someone presses the Run button, often with chosen settings called runtime parameters.
Triggers can be narrowed with filters. A branch filter says which branches count, such as only the main branch and release branches. A path filter says which folders matter, so that a change to the documentation does not start a full build of the application. Skipping a run for a message such as a tag in the commit is also possible, and a batch option groups several quick pushes into one run, which saves capacity when many developers push at once.
The pull request check deserves special attention. By adding a build validation policy to a branch in your repository, you can require that the pipeline passes before a change may be merged. This turns the pipeline from a helpful tool into a gatekeeper that protects the main branch from broken code.
flowchart LR
T1["Push to branch"] --> P["Pipeline run"]
T2["Pull request opened"] --> P
T3["Nightly schedule"] --> P
T4["Another pipeline completes"] --> P
T5["New package or image"] --> P
T6["Manual run"] --> P
A CI trigger is a doorbell that rings whenever a parcel arrives. A pull request trigger is a security guard who checks the parcel before it is allowed through the gate. A scheduled trigger is the alarm clock that wakes the kitchen at the same time each morning, delivery or not.
Use path filters in large repositories that hold several applications. Otherwise every small change anywhere starts every pipeline, and you spend money and waiting time on builds that could not have changed.
9Internal Working: Life of One Run
What actually happens, step by step, between a developer pressing save and a green tick appearing?
Let us follow one push. A developer sends a change to the repository. The repository notifies Azure Pipelines through a connection called a webhook, which is simply a message sent automatically when something happens. The service finds the pipeline definition for that branch and reads it. If the definition uses templates from other places, the service pulls those in and combines everything in a step called expansion. It then checks the triggers and filters. If the change matches, the service creates a run and breaks it into stages and jobs, working out which can start first.
For each job that is ready, the service looks at what pool and what machine type it asks for, and places the job in a queue. It also checks whether you have enough parallel jobs, the number of jobs your organization is allowed to run at the same moment. If all slots are busy, the job waits. When a slot and an agent are free, an agent receives the job. For Microsoft-hosted agents, a fresh virtual machine is prepared. The agent downloads the steps, checks out the source code from the repository, and runs each step in order, streaming the log lines back to the service as they appear, which is why you can watch a run live in your browser.
When the job ends, the agent reports the result: succeeded, failed, canceled, or partially succeeded. The service records it, saves the logs and any published artifacts, and decides what comes next. If the next stage depends on this one, it now becomes ready. If there is an approval waiting, the service pauses and notifies the approvers. The agent’s machine is deleted if it was Microsoft-hosted. At the end of the last stage, the run is marked complete and the final status is sent back to the repository, where it appears as a tick or cross beside the change.
sequenceDiagram
participant Dev as Developer
participant Repo as Repository
participant Svc as Pipelines service
participant Ag as Agent
participant Env as Environment
Dev->>Repo: Push code
Repo->>Svc: Webhook notifies change
Svc->>Svc: Read definition and create run
Svc->>Ag: Offer queued job to free agent
Ag->>Repo: Check out source code
Ag->>Ag: Run steps in order
Ag-->>Svc: Stream logs and send result
Svc->>Env: Start deployment job after checks
Svc-->>Repo: Report status tick or cross
Event arrives
A webhook tells the service that code changed or a schedule fired.
Definition is expanded
Templates and conditions are resolved into a concrete plan of stages and jobs.
Jobs wait in a queue
The service waits for a free parallel slot and a matching agent.
Agent executes
It fetches code, runs steps, and streams logs back live.
Checks and approvals
The service pauses at gates where a human or a rule must agree.
Result is reported
Status returns to the repository and records are kept for history.
It works like a restaurant ordering system. The waiter, the service, takes the order and writes it on a ticket. The ticket goes to the kitchen queue. A free cook, the agent, picks it up, makes the dish step by step, and calls out progress. The waiter checks with the manager before serving anything special, then brings the dish to the table and records the sale.
When all parallel job slots are busy, runs wait in the queue and may look stuck. Before you suspect a bug, check the queue and your parallel job limit. A shortage of slots is among the most common reasons for slow pipelines.
10Variables, Parameters, and Secrets
The same pipeline must work for many settings. Variables and parameters make that possible without copying the pipeline.
A variable is a named value that a pipeline can use in its steps, such as the name of a server, a version number, or a build configuration. Variables can be defined inside the pipeline file, in the portal, or created while a run is going. The service also provides many built-in variables describing the run itself: the name of the branch, the number of the build, the folder where the source code was placed, and the identity of who started it. Using these avoids typing the same facts in many places.
A variable group is a named collection of variables stored in the shared library of a project, and many pipelines can use the same group. It is the right place for values that are shared by several pipelines, such as the address of a test database. A variable group can be linked to Azure Key Vault, a locked cabinet for secrets in Azure. The pipeline then reads secrets from the vault at the moment a run starts, so the secrets are never copied into the pipeline service for long-term storage.
A runtime parameter is different. It is a value that the person starting the run can choose, such as which environment to deploy to or whether to skip a slow test. Parameters have a type, such as text, number, true or false, and can offer a fixed list of allowed choices, which appear as a form in the portal when someone runs the pipeline. Because parameters are decided when the pipeline is expanded, they can even change the shape of the pipeline, for example by deciding whether a whole stage appears.
| Kind | Where it lives | Best for | Care needed |
|---|---|---|---|
| Pipeline variable | Inside one pipeline | Settings used only by that pipeline | Do not store secrets in plain text |
| Variable group | Shared library in the project | Settings shared by several pipelines | Control who can use and edit the group |
| Key Vault linked group | Azure Key Vault | Secrets such as passwords and tokens | The pipeline needs permission to read the vault |
| Runtime parameter | Chosen when starting a run | Choices such as target environment | Limit choices to safe values |
| Built-in variable | Provided by the service | Facts about the run and the branch | Some values can be influenced by users |
How secrets are protected
A variable marked as secret is encrypted in the service and its value is masked in logs, shown as stars. Secret variables are deliberately not passed to scripts automatically. You must map a secret into a step explicitly, which makes it harder to leak by accident. Remember, though, that masking is a courtesy, not a guarantee. A script that you allow to run can still transform a secret, such as printing it with spaces between the letters, and defeat the mask. Never run untrusted code in a pipeline that has access to important secrets.
Variables are labeled jars in the kitchen, such as “sugar” or “oven temperature,” that every recipe can refer to. A variable group is the shared pantry shelf used by every cook. A secret is the key to the locked cupboard of expensive spices, handed to a cook only for the exact dish that needs it, and taken back right afterward.
Never put passwords, keys, or tokens in the pipeline file or in plain variables. They end up in your repository history, where anyone with read access can find them forever. Use secret variables, Key Vault, or better still, identity-based access with no secret at all.
11Templates and Reuse
The difference between a beginner’s pipeline and a professional one is reuse. Templates let a hundred projects share one good design.
A template is a piece of pipeline definition stored in its own file, which other pipelines can include. Instead of copying twenty lines for building a web application into every project, you write them once as a template and each project refers to it, passing in parameters for the few things that differ. When you improve the template, every project that uses it improves too. Templates come in several sizes: a step template holds a few steps, a job template holds a whole job, a stage template holds a stage, and a variable template holds a set of variables. Templates can include other templates, so a stage template might use job templates that use step templates, like nesting dolls.
Templates can live in the same repository, or in a shared repository maintained by a platform team and referenced by many teams. A shared repository lets an organization publish its approved building blocks, such as “how to build and scan a container image” or “how to deploy to our Kubernetes clusters.” Versions of templates can be pinned using branches or tags, so that a change in the template does not break every pipeline at once.
Templates as guardrails
Templates are also a powerful tool for governance, which means keeping everyone within safe rules. In an extends template, the pipeline is not allowed to define its own structure. Instead it must extend a master template written by the security or platform team. The master decides which steps are mandatory, such as vulnerability scanning and compliance checks, and what teams can customize, such as the application’s build commands. You can then add a required template check to protected resources such as a production environment or a service connection, so that only pipelines extending the approved template may use them. A team cannot quietly remove the scan step, because the resource refuses pipelines that do not follow the template.
flowchart TB
PT["Platform team approved template"] --> S1["Mandatory scan step"]
PT --> S2["Mandatory compliance check"]
PT --> SLOT["Team supplied build steps"]
APP1["Team A pipeline extends template"] --> PT
APP2["Team B pipeline extends template"] --> PT
PT --> PROD["Production environment - requires template"]
A step template is a standard recipe card for “make white sauce” that every cook can slot into their dish. An extends template is a franchise operations manual: every branch must follow the same safety and hygiene steps, and head office will refuse to supply stock to a branch that skips them, though each branch may choose its own local specials.
Everyday example: one template, fifty services
A company runs fifty small services. A platform team writes one template that builds, tests, scans, and publishes a container image. Each service has a ten-line pipeline file that calls the template with its own name and settings. When a new security scanner is added to the template, all fifty services get it the next time they run, with no change to any of them.
Keep templates small and focused, name their parameters clearly, and document them with a short example. A template that nobody understands will be copied and edited, and the benefit of sharing is lost.
12Artifacts, Caching, and Packages
Because every job starts clean, you need ways to pass files forward and to avoid repeating slow work.
A pipeline artifact is a file or folder that a run saves so that later jobs and stages can use it, or so that people can download it. For example, the Build stage compiles the application and publishes the result as an artifact named “drop.” Every later stage downloads exactly this same artifact. This is the key to a core rule of good delivery: build once, deploy many. The same tested package that went to the test environment should go on to staging and production, rather than being rebuilt at each step, because a rebuild could differ in tiny ways and make your testing meaningless.
Azure Artifacts is a related service that hosts packages, which are reusable pieces of code that your programs depend on or that you share between teams. It supports common formats such as NuGet, npm, Maven, Python, and a general-purpose universal format. A feed is a private store of packages with its own permissions. Teams publish internal libraries to a feed, and pipelines pull them back when building other projects. A feed can also act as a safe proxy to public registries and keep a copy of the public packages you used, which helps protect you if a public package is removed or tampered with.
Caching saves time. Most builds begin by downloading the same dependencies, such as hundreds of libraries, that did not change since the last run. A cache task stores a folder, such as the downloaded libraries, under a key that is based on the files that describe your dependencies. When the key matches on a later run, the folder is restored in seconds instead of downloaded again. When the dependency list changes, the key changes, and a new cache is saved. This is especially valuable on Microsoft-hosted agents, which start from nothing on every job.
Pipeline artifacts
Publish build output once, download it in later jobs and stages.
Azure Artifacts feeds
Private stores for NuGet, npm, Maven, Python, and universal packages.
Cache task
Restore unchanged dependencies from a cache keyed by your dependency files.
Container registries
Push built container images to a registry such as Azure Container Registry.
Versioning and metadata
Tag every artifact with the build number and the source commit.
Retention policies
Control how long old artifacts and runs are kept to limit storage.
Build once, deploy many is like baking a batch of loaves, sealing them in numbered boxes, and sending the very same boxes to every shop. If you baked fresh bread at each shop, no one could be sure the bread the inspector approved was the bread the customers ate. Caching is like keeping flour in the kitchen’s cupboard instead of going to the mill for every loaf.
Record the commit and the build number inside every package, for example in its version. When something goes wrong in production you can find, within seconds, exactly which code is running.
13Testing in Pipelines
A pipeline that builds but does not test only proves the code compiles. Tests prove it works.
Tests come in layers, often drawn as a pyramid. At the wide base are unit tests, tiny and very fast checks of single functions, which run in seconds and can number in the thousands. In the middle are integration tests, which check that parts work together, such as the application talking to a real database. At the narrow top are end-to-end tests, which drive the whole system like a user would, clicking through a browser. Higher tests are slower and more fragile, so a healthy suite has many unit tests and few end-to-end tests. Other checks add safety: static analysis reads code for mistakes and style problems without running it, security scans look for known vulnerabilities in your code and in the libraries you use, and performance tests check speed under load.
Azure Pipelines understands the results of common test tools. A publish test results step reads the output files, and the run page then shows how many tests passed and failed, which tests are new failures, and how long each took. Code coverage reports show what fraction of the code the tests actually exercised. The results flow back to the pull request page, so reviewers see a failing test beside the change that caused it. Over time, the service tracks test trends and can flag flaky tests, which sometimes pass and sometimes fail on the same code without any change.
Static checks and unit tests
Run first, in seconds to minutes. Fail fast, so the author gets feedback while still thinking about the change.
Security and dependency scans
Look for known weaknesses in code and packages before anything is deployed.
Integration tests
Run against a real database or service in a temporary environment.
End-to-end and performance tests
Run on a staging environment closest to production.
Smoke tests after deployment
A handful of quick checks confirm that the new version is alive and serving.
Testing is like checking a new car. Unit tests are checking each bolt and bulb on the bench. Integration tests are making sure the engine, gearbox, and wheels work together. End-to-end tests are a full drive on the road. A smoke test is turning the key in the showroom to hear the engine start before handing over the keys.
Flaky tests destroy trust. If people get used to re-running a red pipeline until it turns green, real failures will be ignored as well. Fix or quarantine flaky tests quickly rather than hiding them behind automatic retries.
14Deployment: Environments and Strategies
Getting the package to users safely is where pipelines earn their keep.
An environment is a named target for deployment, such as Test, Staging, or Production. It is more than a label. An environment keeps a history of every deployment to it, showing which run, which commit, and who approved it. It can hold resources, such as a Kubernetes namespace or a group of virtual machines, so that the pipeline can see what it is deploying onto. And, importantly, an environment is where you attach approvals and checks, the gates that stop a deployment until a rule is satisfied.
The job that deploys to an environment is a deployment job. It supports deployment strategies, different recipes for rolling out a change with less risk.
| Strategy | How it works | Risk | Good for |
|---|---|---|---|
| Run once | Deploy everywhere in one go | All users affected if it is bad | Simple apps and test environments |
| Rolling | Update a few servers at a time while others keep serving | Medium, since the old and new run together briefly | Groups of virtual machines |
| Canary | Send a small share of traffic, for example five percent, to the new version first, then grow it | Low, a bad change reaches few users | Large user bases and Kubernetes |
| Blue-green | Run the new version beside the old one and switch traffic in one step, with an instant way back | Low, with quick rollback | Web apps with deployment slots |
flowchart LR
NEW["New version"] --> C1["Canary - 5 percent of users"]
C1 --> CHK["Check errors and speed"]
CHK -->|"healthy"| C2["Grow to 50 percent"]
C2 --> C3["Roll out to 100 percent"]
CHK -->|"unhealthy"| RB["Roll back to old version"]
Deployment strategies provide lifecycle hooks, which are named points where you can place your own steps: before deployment, during it, after traffic is routed, and then separate steps for on success and on failure. For example, a failure hook can restore the previous version and send an alert. For web apps in Azure App Service, deployment slots let you deploy to a hidden twin of the live site, warm it up, test it, and then swap the twin into place, which gives zero downtime and a one-click swap back.
A canary rollout is like a restaurant trying a new recipe on one table first and checking the reaction before serving it to the whole room. Blue-green is having a second, fully set-up dining room ready: when it is perfect, you simply open its doors and close the old room, and if guests complain, you open the old room again.
Design database changes so that the old version of the application still works with the new database layout during a rollout. This is often done in small steps, adding before removing, so you can always roll back safely.
15Service Connections and Identity
To deploy into Azure or push to a registry, the pipeline needs permission. Service connections are how that permission is held and controlled.
A service connection is a saved, secure link between your Azure DevOps project and an outside system. Examples include an Azure subscription, a container registry, a Kubernetes cluster, GitHub, or a package server. A pipeline never receives the underlying credential in plain sight. It refers to the connection by name, and the service passes the access to the right task at the right moment. Each connection has its own permissions: which pipelines may use it, and which users may edit it.
For connections to Azure, the older way was a service principal with a secret, which is an identity in Microsoft Entra ID with a password-like value. The weakness is that the secret expires, must be rotated, and could leak. The modern and recommended way is workload identity federation. Here the pipeline proves its identity to Entra ID using a short-lived signed token, through a standard called OpenID Connect, and Entra ID is configured to trust that exact pipeline. Nothing secret is stored, nothing expires silently, and the access the pipeline receives is short-lived. For self-hosted agents running on Azure virtual machines, a managed identity offers a similar secret-free route.
| Method | Secret stored | Lifetime of access | Recommendation |
|---|---|---|---|
| Service principal with secret | Yes | Long, until rotated | Avoid for new work |
| Service principal with certificate | Yes, a certificate | Long, until it expires | Better than secrets, but still stored |
| Workload identity federation | No | Short, per run | Preferred for Azure |
| Managed identity on agent | No | Short, from the machine | Good for self-hosted agents in Azure |
A service principal with a secret is a key copied to the delivery driver, which stays valid until someone remembers to change the lock. Workload identity federation is a day pass that the security desk prints for each delivery after checking the driver’s company ID. It works for that one visit and then turns to paper.
Give each service connection the smallest permission it needs, and one connection per environment. A single connection with owner rights over the whole subscription, usable by every pipeline, turns one compromised pipeline into a compromise of everything.
16Security and Governance
A pipeline can change your production systems. That power makes it a target, and a place where good rules pay off.
Think of security in layers. The first layer is who may change the pipeline. Because pipeline files live in the repository, branch policies and code review protect them: nobody should be able to push a change to the main pipeline file without review. The second layer is protected resources. Environments, service connections, variable groups, agent pools, and secure files can each be marked as protected and carry checks that every run must pass before it may use them.
Approvals
A named person or group must say yes before a stage may use an environment, for example before production.
Branch control
Only runs from allowed branches, such as main, may deploy to production.
Business hours
Allow deployments only in agreed time windows.
Required template
Only pipelines that extend an approved template may use the resource.
Exclusive lock
Only one run at a time may deploy to an environment, so releases do not collide.
Invoke a function or call a service
Run your own automated check, such as a health query or a ticket status test, before allowing the run to continue.
Threats worth knowing
Code from forks. On public projects, strangers can propose changes from their own copies of your repository. By default, secrets are not given to builds of such changes, and you should keep it that way. Injection. If a pipeline builds a command from text that a user controls, such as a branch name or a pull request title, a clever attacker may sneak extra commands into it. Treat these values as untrusted data, never paste them directly into scripts. Supply chain attacks. A tampered library or build tool can slip into your build. Pin versions, use your private feed as a proxy, scan dependencies, and prefer trusted tasks from verified publishers. Over-privileged pipelines. A pipeline that can do everything gives an attacker everything. Use the setting that limits the job authorization scope, so that a run can only touch resources in its own project.
flowchart TB
CODE["Code change by developer"] --> PR["Pull request review and branch policy"]
PR --> RUN["Pipeline run from approved branch"]
RUN --> CHK["Environment checks - approval, branch, template"]
CHK --> CONN["Service connection with least privilege"]
CONN --> PROD["Production deployment"]
RUN -.-> LOG["Audit log and run history"]
Pipeline security is like the safety rules at a bank vault. The guard checks your badge, a second person must sign the form, the vault opens only during working hours, and a log records who went in and when. No single mistake or single stolen key is enough to empty the vault.
Review the audit log regularly. It records changes to permissions, service connections, and policies, and can be sent to a central security system. Many incidents are first spotted as an unexpected permission change.
17Reliability and Failure Handling
Pipelines fail for many reasons, and good ones fail clearly, safely, and recoverably.
Start by understanding the result of a run. A run can succeed, fail, be canceled, or end partially succeeded, which means everything that was required passed, but some step marked as allowed to fail did not. Each step and job can have a timeout, the longest it may run before the service stops it, and it is wise to set a realistic limit. Otherwise a hung job may hold an agent and a parallel slot for hours. Tasks can be given a retry count to try again after a failure, which helps with short problems such as a brief network glitch while downloading a package.
Use conditions to control what runs after trouble. A clean-up step can be set to run always, whether the earlier steps passed or not, such as removing a temporary test environment so it does not keep costing money. A notification step can run only on failure. When a stage fails after a long time, you do not need to start over: you can often rerun failed jobs or rerun from the failed stage, saving the work that already succeeded.
Failure scenarios and recovery
A test fails
The run stops at that stage, the result shows which test failed, and the author fixes the code. No bad build moves forward.
A download fails briefly
A retry on the task cures it without anyone noticing. Persistent failures show up in the logs.
No agent is available
The job waits in the queue. Add more agents, more parallel jobs, or use scale-out pools.
A deployment goes wrong
Health checks fail, and the failure hook rolls back to the previous artifact or swaps the slot back.
A secret expires
Service connections fail. Use identity federation to avoid expiry, and alert before certificates expire.
The service has an outage
Builds cannot start. Keep a documented manual path for an urgent fix, and keep your pipeline definitions in the repository so they can be recreated.
A resilient pipeline is like a well-run airline. If a plane has a fault, ground staff catch it before takeoff, a backup plane is available, and a clear plan says who to tell and what to do. A timeout is the rule that a flight which cannot leave in time gets canceled instead of waiting forever on the runway.
Do not use automatic retries to hide real problems. If a step needs three tries to pass every day, it is broken and the retries are only delaying the repair. Review the retry statistics from time to time and fix the causes.
18Monitoring, Logging, and Metrics
A healthy delivery system is one you can see into. Pipelines give you several windows.
The first window is the run page. For every run you can see the stages and jobs as a live diagram, open the log of each step, download all logs, and read the summary of tests, artifacts, and linked work items. When a step fails, the log shows the exact command and error. For deeper detail, you can run with system diagnostics turned on, which prints extra information about what the agent is doing, though it can reveal more than you want, so use it carefully.
The second window is analytics. Pipeline reports show the pass rate, the average duration, and the time spent in each stage and task over weeks and months. A pipeline that slowly grows from ten minutes to forty is a problem you can spot early. You can also see how often tests fail, which tests are flaky, and how long jobs wait in the queue for an agent. The third window is notifications and service hooks. Notifications send email when runs fail or approvals wait. Service hooks send events to chat tools such as Teams or Slack, ticket systems, or your own web address, so that the right people hear about a failed production deployment within moments.
The fourth window concerns agents and pools. You can watch how many agents are online, how busy they are, and how many jobs are waiting. For self-hosted agents, also monitor the machines themselves: disk space, memory, and operating system updates. A full disk on a build machine is a classic cause of mysterious failures.
| What to watch | Why it matters | Typical action |
|---|---|---|
| Pass rate of the main pipeline | A falling rate shows quality or stability trouble | Investigate the top failing tests and steps |
| Average run duration | Slow pipelines reduce how often people merge and deploy | Use caching, parallelism, and trimming |
| Time waiting in queue | Shows a shortage of agents or parallel jobs | Add capacity or reduce wasteful runs |
| Flaky tests | Erode trust and waste time | Fix or quarantine them |
| Deployment frequency and failures | Core measures of delivery health | Track trends and learn from failures |
| Agent health | Disks, memory, and updates on self-hosted agents | Automate cleaning and patching |
| Audit events | Permission and policy changes | Send to security monitoring |
Measures that tell the real story
Researchers who study software delivery, in a body of work known as DORA metrics, found four numbers that distinguish strong teams: deployment frequency, how often you release; lead time for changes, how long from a code change to production; change failure rate, what share of releases cause trouble; and time to restore service, how quickly you recover. Your pipeline records the data needed for the first two and helps with the others, so it is a natural place to measure them.
Run history is kept according to retention policies, so old runs and artifacts are eventually deleted. If you need long-term records for audits, export the key facts or keep the artifacts you must retain for the period your rules require.
19Cost and Pricing
Azure Pipelines is priced by how much work can run at the same time, not by how many minutes you use in total.
The key unit is the parallel job. One parallel job lets one job run at a time. If you have two parallel jobs, two jobs can run side by side, and a third must wait in the queue. You buy parallel jobs separately for Microsoft-hosted agents and for self-hosted agents. Public projects usually get generous free capacity, and private projects typically get a small free grant that can be limited in minutes per month, so it is wise to check the current terms for your organization on the official pricing page.
Beyond the free grants, you pay a monthly price for each extra parallel job. Self-hosted parallel jobs are cheaper than Microsoft-hosted ones, because you also provide and pay for the machines yourself. Related services, such as Azure Artifacts storage beyond a free amount and user licenses for Azure DevOps, are priced separately.
| Cost driver | What it means | How to control it |
|---|---|---|
| Parallel jobs | How many jobs may run at once | Buy only what queue times justify |
| Minutes on hosted agents | Free grants may limit monthly minutes for private projects | Cache, trim tests, avoid pointless runs |
| Self-hosted machines | You pay Azure or your data center for the virtual machines | Use scale-out pools that shut down when idle |
| Artifact storage | Packages and artifacts beyond the free amount | Set retention and clean old versions |
| Users | Licenses for people beyond the free tier | Review who really needs access |
| Test environments | Cloud resources created during the pipeline | Always tear them down in a final step |
Parallel jobs are like checkout counters in a shop. You do not pay per customer. You pay for how many counters are open. With one counter, customers queue. Too many counters and most sit idle, wasting wages. The right number is the one where queues are short without empty counters.
Use path filters so that a change to the documentation does not trigger a full build and test cycle. Cutting unnecessary runs is the cheapest optimization there is.
20Speed and Performance
Developers wait for pipelines. A faster pipeline means faster feedback, happier teams, and more frequent releases.
There are five main ways to speed things up. First, run in parallel: split independent work into separate jobs, and use a matrix to run test groups at the same time. Second, cache dependencies and build outputs so that unchanged work is not repeated. Third, do less: use branch and path filters, shallow clones that fetch only recent history, and incremental builds that rebuild only what changed. Fourth, order smartly: put fast checks first so that obvious failures stop the run early. Fifth, choose the right agent: a bigger machine, or a self-hosted agent that keeps its tools and caches warm between runs, can cut minutes from every run.
Before changing anything, measure. The run page shows how long each step took. Often a single step, such as downloading packages or running one slow test suite, takes half of the total time. Fix the biggest slice first. Be careful about parallelism, too: more jobs mean more parallel slots consumed, and splitting a small job adds start-up overhead for each piece.
Speeding up a pipeline is like speeding up a kitchen. You chop vegetables at the same time as the water boils (parallel), keep the chopped onions ready in a tub (cache), skip dishes nobody ordered (filters), and taste the sauce early so you do not cook a whole meal on a bad base (fail fast).
21Design Patterns and Anti-Patterns
Teams that have run thousands of pipelines keep finding the same good ideas and the same traps.
Patterns that work
- Pipeline as code. Keep the pipeline file in the repository with the code it builds.
- Build once, deploy many. Promote the same artifact through all environments.
- Shared templates. Publish approved building blocks for all teams.
- Fail fast. Cheap checks first, expensive ones later.
- Trunk-based flow with short branches. Merge small changes often and let the pipeline guard main.
- Progressive delivery. Canary, slots, and checks for production.
- Ephemeral environments. Create a test environment for the run and delete it afterward.
- Identity-based access. Workload identity federation instead of stored secrets.
Anti-patterns to avoid
- Click-built classic pipelines. Nobody can review, copy, or recover them easily.
- Rebuilding per environment. What you tested is not what you ship.
- The mega-pipeline. One enormous file that nobody dares to touch.
- Copy and paste across repositories. Fixes never reach all copies.
- Secrets in the repository. They live forever in history.
- Ignoring red builds. A broken main branch becomes normal.
- Manual steps in the middle. Hand-run scripts that only one person knows.
- One shared, all-powerful service connection. One leak opens every door.
Good patterns are the habits of a tidy workshop: every tool has a place, every job has a checklist, and the same sample is measured at every stage. Anti-patterns are the cluttered workshop where nobody knows which saw is sharp and the only person who understands the machine is on holiday.
22Best Practices and Common Mistakes
A short checklist of what experienced teams do, and the slips that cost beginners the most time.
Define pipelines in YAML, keep them in the repository, and protect the pipeline file with the same review rules as code.
Require the pipeline to pass as a branch policy before a pull request can be merged, so main is always healthy.
Use separate stages and environments with approvals and checks on production, and keep service connections limited per environment.
Pin versions of tools, tasks, templates, and base images so a surprise upgrade does not break your build on a Friday afternoon.
Set timeouts, clean up temporary resources in an always-run step, and keep retention rules sensible.
Mistakes that cost the most
Leaving the default permission too wide
Pipelines that can reach everything are a risk. Restrict scope and authorize resources one by one.
Forgetting that each job starts clean
Files made in one job vanish unless published as artifacts. This confuses almost every beginner.
Trigger loops
A pipeline that pushes a commit can trigger itself forever. Skip CI on automated commits.
Forgetting to tear down test environments
Resources left running after a failed run quietly build up a large bill.
Putting everything in one long job
A failure at the end wastes everything before it, and nothing can run in parallel.
Treating the pipeline as the only documentation
Write a short readme: what the pipeline does, how to run it, who owns it, and how to recover when it breaks.
23Azure Pipelines Compared with Others
There are several good tools for automation. The right choice depends on where your code lives and what you need.
| Tool | Strengths | Watch out for |
|---|---|---|
| Azure Pipelines | Works with code in Azure Repos, GitHub, and others. Strong multi-stage deployment, environments, approvals, and checks. Hosted agents for Linux, Windows, and macOS. Tight Azure integration. | Parallel-job pricing model. Two ways to define pipelines (YAML and classic) can confuse newcomers. |
| GitHub Actions | Built into GitHub, a large marketplace of reusable actions, simple workflow files, and strong community. | Enterprise-style release gates and governance need more design. Check current feature differences. |
| Jenkins | Free and open source, enormous plugin collection, total control over your own servers. | You maintain the servers, upgrades, plugins, and security yourself. |
| GitLab CI/CD | Part of an all-in-one GitLab platform with built-in registry and security scans. | Best when your code is already in GitLab. |
| Cloud-native build services | Deep integration with their own cloud and low setup work. | Tied to a single cloud provider. |
Many organizations use more than one. A common pattern is to keep source code on GitHub and use Azure Pipelines for build and deployment, because it connects well to GitHub while adding environments and approvals. Microsoft invests heavily in GitHub Actions too, so it is sensible to check current guidance on which service best fits new projects, and to avoid moving pipelines just for fashion. Moving has a real cost, and a working, well-designed pipeline is worth keeping.
Choosing a CI/CD tool is like choosing a delivery service. One is built into the shop you already use, another is a flexible do-it-yourself van fleet you must maintain, and another is a large logistics company with strict scheduling and inspection points. None is best in all cases. The best one matches your cargo, your route, and your rules.
24Real-World Examples
Three stories show how the pieces fit together in daily practice.
Example 1: A small web shop with a single pipeline
A small team builds an online shop on Azure App Service. The pipeline triggers on changes to main. It restores packages from a cache, builds the code, runs unit tests, publishes the package as an artifact, deploys to a staging slot, runs a few smoke tests, and then swaps the slot into production after one manager approves. The whole process takes around fifteen minutes, and the team deploys several times a day. If the smoke tests fail, the slot swap never happens, so customers never see the problem.
Example 2: A bank-style platform with strong governance
A large financial company has hundreds of services. A platform team owns an extends template that forces vulnerability scanning, license checks, and signing of container images. Production environments require the approved template, a change ticket check, and approval from two people. Workload identity federation removes stored secrets. Self-hosted agents run inside the company network to reach private systems, and are rebuilt from a clean image weekly. Auditors can see, for any production change, the commit, the run, the approvers, and the test results.
Example 3: A mobile and cloud product with many platforms
A company ships an app for Android and iOS plus a cloud back end. A matrix runs the same test suite on several operating system versions at once. Microsoft-hosted macOS agents build the iPhone app, while Linux agents build the back end. Pull requests run quick checks in minutes, while a nightly scheduled pipeline runs the long performance and end-to-end suites. Release builds go to testers first through a distribution service and only then to the stores.
Notice the shared shape in every story: small changes, automatic checks, one artifact moving forward, and human approval only where risk is high. That shape matters more than any specific tool.
25When to Use Azure Pipelines, and When Not To
A good architect knows both where a tool shines and where it does not.
Advantages
- Works with many source hosts and many languages and platforms.
- Hosted agents for Linux, Windows, and macOS need no maintenance.
- Strong multi-stage releases with environments, approvals, and checks.
- Pipelines as code, templates, and extends for governance at scale.
- Good Azure integration and identity federation.
- Free tiers for getting started and for public projects.
Disadvantages and trade-offs
- Concurrency-based pricing can surprise teams that need a lot of parallel work.
- YAML has a learning curve, and complex templates become hard to read.
- Hosted agents start clean, so caching must be designed carefully.
- Self-hosted agents need patching, monitoring, and security hardening.
- Classic and YAML experiences coexist, which splits documentation and habits.
- A powerful pipeline is a powerful target and needs careful security.
A simple decision guide
Azure-heavy delivery
You deploy mostly to Azure and want approvals, environments, and identity integration.
Mixed platforms
You build for Windows, Linux, and macOS from one place.
Governance needs
You need central rules and audit trails across many teams.
Everything lives in another platform
If your code, issues, and security tooling all sit in a different all-in-one platform, its built-in pipelines may be simpler.
Tiny hobby project
A very small project may be well served by a simpler built-in workflow.
Ultra-custom build farm
If you need deep control over every machine, a self-managed system may suit you better, at the price of upkeep.
26Frequently Asked Questions
No. Azure DevOps is a suite of services, and Azure Pipelines is one of them. The others cover boards for planning work, repositories for code, test plans, and artifact feeds. You can use Pipelines alone, with code stored elsewhere.
Continuous integration builds and tests every change automatically. Continuous delivery keeps every tested change ready to release at the push of a button. Continuous deployment goes one step further and releases automatically with no button. Most teams use deployment automation with a human approval for production.
Use YAML for new work. It lives with your code, can be reviewed, and supports templates. Classic pipelines still work, but YAML is where most improvements have been focused.
An agent is the computer that actually performs the work of a job. It asks the service for work, runs the steps, and reports results. It can be hosted by Microsoft or managed by you.
Usually because all parallel jobs are in use or no agent in the pool matches what the job needs. Check the queue reason on the run page, then add parallel jobs, add agents, or relax over-strict requirements.
Each job normally starts on a fresh, clean agent. Publish files as artifacts at the end of one job and download them in the next.
Prefer no secret at all, using workload identity federation or a managed identity. Where secrets are unavoidable, keep them in Key Vault or secret variables, pass them only to the steps that need them, and never run untrusted code beside them.
Yes. Use deployment slots, blue-green, rolling, or canary strategies, combined with health checks and a quick rollback path. Database changes need extra care so that old and new application versions can both work during the change.
Measure first, then use caching, parallel jobs, filters, shallow clones, and better ordering. Often one slow step is most of the problem.
Yes. Define a stage per environment, each with a deployment job that targets an environment. Add approvals and checks to the environments that need them, and promote the same artifact through all stages.
Later stages that depend on it do not run unless you ask them to. You can read the logs, fix the cause, and rerun failed jobs or the failed stage instead of starting over.
There are free grants, and public projects usually get more. Heavier use needs paid parallel jobs. Terms change, so check the official pricing page for current numbers.
27Summary and Key Takeaways
Azure Pipelines is an automated assembly line for software. A change goes in, and a tested, traceable, deployable package comes out.
We started with the bakery idea: the same careful steps, run by machines, in the same order every time. We then met the hierarchy of pipeline, stage, job, and step; the agents that do the work; the triggers that start it; and the life of a single run from webhook to final status. We looked at variables, templates, artifacts, caching, testing, and deployment strategies. We saw how service connections, identity federation, checks, and approvals keep power under control, and how monitoring, retries, and clean-up keep the system dependable. Finally we compared costs, patterns, and alternatives, so that you can decide when this tool fits.
Key Takeaways
- Azure Pipelines automates build, test, and deployment, and works with code from many sources.
- Use YAML pipelines in your repository, organized as stages, jobs, and steps, with shared templates for reuse and governance.
- Agents do the work, and each job starts clean, so pass files forward as artifacts and speed up with caching.
- Build once and deploy the same artifact through environments, using approvals, checks, and safe rollout strategies.
- Prefer workload identity federation and least-privilege service connections over stored secrets.
- Watch pass rate, duration, queue time, and flaky tests, and fix causes rather than hiding them with retries.
- Pricing is based on parallel jobs, so measure queues before buying more capacity.