Start with the simple definition, the history, and the one-sentence idea you will use to explain it to anyone.

Workflows is a fully managed service from Google Cloud that runs a list of steps, in the order you describe, to complete a job. Each step can call a web address, call a Google Cloud service, make a decision, repeat something, wait for a while, or wait for a person or another system to reply. You describe the whole job once, as a written definition, and Google Cloud runs it for you every time it is needed.

The word “managed” matters. You do not create servers, you do not install an engine, you do not patch anything and you do not decide how many machines to keep running. The word “serverless” means the same thing from another angle: you pay only when your workflow actually runs, and when nothing is running, nothing is charged for compute you are not using.

A short history

Google Cloud introduced Workflows in 2020 as its answer to a gap in the platform. Google already had services that run code (Cloud Functions and Cloud Run), services that move messages (Pub/Sub) and services that store data. What it lacked was a lightweight, serverless way to say “do this, then that, then the other thing, and handle problems along the way.” Teams were gluing functions together by hand, with one function calling the next, and the result was fragile and hard to debug.

Since the first release the service has grown steadily. Connectors were added so that calling Google Cloud services became simpler and safer. Callbacks were added so a workflow could pause and wait for outside events. Parallel execution, better error handling, a visual graph in the console and stronger integrations with event triggers followed. The core idea has stayed the same: describe the process, let Google run it.

To understand why a tool exists, first feel the pain that it removes.

Imagine you run an online store. When a customer pays, you must reserve stock, charge the card, create a shipping label, send a confirmation email and update a sales report. That is five jobs. Now imagine each job is handled by a separate small program. How do you make sure all five happen, in the right order, exactly once?

The naive approach: chains of direct calls

The first idea most teams try is to let each program call the next one. The payment program calls the stock program, which calls the shipping program, and so on. This works on a good day. On a bad day it breaks in confusing ways. If the shipping program is slow, the payment program waits and may time out. If the email program fails, nobody knows whether to retry. If the stock program succeeded but the payment failed, you have reserved stock for a customer who never paid.

The second approach: messages everywhere

The next idea is to publish a message after each step and let the next program react to it. This is called choreography, and it is powerful. But as the number of steps grows, nobody can answer a simple question like “where is order 42 right now?” The logic of the process is scattered across many programs. Changing the order of two steps means editing several teams’ code.

The third approach: a central conductor

Orchestration solves this by putting the process in one place. A single definition says what happens first, what happens next, what to do on failure and how long to wait. Each worker program stays simple and only does its own job. The conductor remembers where the process is, so a failure in step four does not erase the memory of steps one to three.

Eight words carry almost the entire vocabulary of Workflows. Learn them once and the documentation becomes easy to read.

The definition language

Workflows are written in a structured text format, using either YAML or JSON. YAML is the more common choice because it reads almost like an outline. You do not need to learn a full programming language. You learn a small set of step types and how to combine them. This is a deliberate design choice: because the language is small, a new team member can read a workflow and understand it in minutes, and Google can run it reliably at scale.

A bird’s-eye view of what sits around a workflow: who starts it, what it calls, and where its records go.

Workflows sits in the middle of three groups of things. On the left are the triggers, the things that start an execution. In the middle is the execution engine, which reads your definition and runs it. On the right are the targets, the services the workflow calls, and the observation tools that record what happened.

flowchart LR
  T1["Cloud Scheduler - time based trigger"] --> E["Workflows Execution Engine"]
  T2["Eventarc - event based trigger"] --> E
  T3["API call or gcloud or client library"] --> E
  E --> H["HTTP calls to Cloud Run and Cloud Functions"]
  E --> C["Connectors to Google Cloud APIs"]
  E --> S["Subworkflows inside the same workflow"]
  E --> CB["Callback endpoint for waiting"]
  CB --> X["Person or external system replies"]
  E --> L["Cloud Logging and Cloud Monitoring"]
  C --> G["BigQuery and Pub/Sub and Cloud Storage and others"]
    

The triggers

A workflow does nothing until something asks it to run. Cloud Scheduler can start it on a clock, such as every night at two. Eventarc can start it when something happens, such as a file arriving in a storage bucket or a message arriving on a topic. Any program with permission can start it through the service’s programming interface, the command line tool or a client library. Another workflow can also start it.

The execution engine

This is the part Google runs for you. It reads your definition, keeps the variables of each execution, runs the steps one by one, saves progress, applies retries and timeouts, and finally records the outcome. Everything is regional: you choose the region when you deploy, and the executions happen there.

The targets

The workflow reaches out in two ways. It can make a plain web request to any address it is allowed to reach, such as a Cloud Run service or a function. Or it can use a connector to call a Google Cloud service in a more guided way. Connectors exist for a large number of Google Cloud products, from data tools to messaging to machine learning.

The observation tools

Every execution can write logs to Cloud Logging, publish metrics to Cloud Monitoring and keep a step-by-step history that you can inspect in the console. Together they answer the question every operator asks at three in the morning: what exactly happened?

Every workflow, however large, is assembled from a handful of step types. Knowing them is like knowing the pieces of a chess set.

Notice how small this list is. There is no way to write an endless loop that eats the machine, because the engine itself limits how long and how many steps an execution may take. That small, bounded list is exactly what keeps every execution predictable.

The standard library

The language ships with helper functions for common chores. There are helpers for making web requests with the right sign-in, for pausing, for writing logs, for working with text, lists and maps, for simple math, for reading the current time, and for creating and waiting on callbacks. You do not install them. They are always available.

You do not need to see the engine to use it, but knowing the broad idea explains almost every behavior you will notice.

Saving progress after every step

The most important idea is durability. After the engine finishes a step, it saves the state of the execution: which step comes next and what every variable currently holds. This is called checkpointing. If the machine running your execution disappears, another machine can pick up from the last saved point. Your workflow does not start over and does not forget what it already did.

Why long waits are cheap

Because progress is saved, an execution that is only waiting does not occupy a machine. When your workflow sleeps for an hour, or waits two days for a human to click approve, the engine stores the paused state and does not keep a program running. This is why Workflows can hold executions open for a very long time without you paying for idle servers. Compare that with a normal program that would need to stay alive, using memory, for the whole wait.

Being careful with repeated work

There is one consequence of checkpointing that every user must understand. The engine saves state between steps, but a call to an outside service might succeed on the other side while the answer is lost on the way back. When the engine retries, the outside service may do the work a second time. This is the classic “at least once” problem. The workflow engine can guarantee the order of its own steps, but it cannot guarantee that a service you call did the job exactly once unless that service is built to handle repeats. We return to this in the chapter on error handling.

Isolation between executions

Each execution has its own private variables. Two executions of the same workflow running side by side cannot see each other’s data. This is why you can safely run thousands of orders through one workflow at the same time.

Follow one execution from the moment it is requested to the moment it ends.

Input in, result out

An execution starts with an input, a small package of data supplied by whoever triggered it. For a storage event the input describes the file. For a scheduler call the input might be empty. The workflow reads that input into variables, passes data from step to step through variables, and finally returns a result. When a workflow is started by an event, the result is usually not used by anyone, so the more useful output is often a side effect, such as a row written to a database.

Two ways to start: wait or not wait

A caller can start an execution and move on immediately, asking for the result later. This is the common style, because workflows can run for a long time and holding a connection open would be wasteful. A caller that needs a quick answer can use a shorter, direct style where the call waits for the result, but this is best kept for short workflows.

The life of an execution

flowchart TD
  A["Execution requested with input"] --> B["ACTIVE - steps are running"]
  B --> W["Paused - sleeping or waiting for callback"]
  W --> B
  B -->|"all steps finish"| S["SUCCEEDED - result stored"]
  B -->|"unhandled error"| F["FAILED - error stored"]
  B -->|"someone cancels"| C["CANCELLED"]
  S --> H["History kept for review"]
  F --> H
  C --> H
    

An execution is active while steps run. It may pause while sleeping or waiting for a callback, then resume. It ends in one of three ways: success, failure or cancellation. In every case a record of the execution is kept for a limited time so that you can look back at it. Because the history is limited, anything you need to keep for audits or reports should be written by the workflow to your own storage as it runs.

There are two ways for a workflow to talk to the world. Choosing between them is one of the most common design decisions.

Plain web requests

The workflow can make a standard web request to any address it can reach. This works with your own services on Cloud Run, with functions, with partner systems and with Google Cloud programming interfaces. You decide the method, the address, the headers and the body. You also decide how to sign in, usually by asking the engine to attach a token for the workflow’s own identity. This route is flexible, but you carry more responsibility. You must read the response, check the status, and decide what to do if the remote service is slow or has not finished its work.

Connectors

A connector is a prebuilt wrapper around a Google Cloud service’s interface. The big benefit is that many Google Cloud operations are long-running: you ask for something, such as starting a data processing job, and the service answers “accepted, I will work on it” with a ticket number. With a plain request, your workflow must keep asking whether the job is finished. A connector does this for you: it polls, it waits, it retries sensibly, and it returns only when the work is truly done.

Real processes are rarely a straight line. This chapter shows how a workflow branches, repeats and does several things at once.

Making decisions

A switch step lets the workflow choose a path. A payment workflow might check whether the order total is above a limit. If yes, it sends the order to a manual review path. If no, it goes straight to charging the card. Decisions can look at the input, at the response from an earlier call or at any variable. Keeping decisions explicit in the workflow, instead of buried inside a service, makes the business rules visible to anyone who reads the definition.

Repeating work

A loop walks through a list. If a customer ordered five items, a loop can reserve stock for each item. Loops are natural, but remember that each pass counts toward your step total and your cost. A loop over fifty thousand records inside a single workflow is usually the wrong tool. Hand large batches to a data service and let that service do the heavy lifting.

Doing things at the same time

Sometimes steps do not depend on each other. Sending the confirmation email and updating the loyalty points can happen together. A parallel step runs several branches at once and waits until all are done. A parallel loop does the same thing across the items of a list, with a limit on how many run at a time so that you do not flood the services you call. Parallel work can cut total time sharply, but it also means that failures can arrive from several branches, and you must decide whether one failed branch should stop the others.

Subworkflows: write once, use many times

A subworkflow is a named group of steps in the same workflow that you can call from several places, passing it inputs and receiving a result. Use them to avoid copying the same five steps in three spots, and to give a meaningful name to a group of steps, such as “charge the customer” or “notify the warehouse.” A subworkflow cannot be started on its own and cannot be called from outside. If you want a reusable process that other teams can start independently, make it a separate workflow and call it using the execution connector.

Distributed systems fail in small, boring ways all day long. A good workflow expects this and plans for it.

Kinds of errors

Errors in Workflows fall into broad families. Some come from the remote service, such as a response saying the request was invalid or the server had a problem. Some come from the connection, such as a timeout or a network break. Some come from your own definition, such as using a variable that does not exist. Some you raise yourself on purpose. The engine gives each family a recognizable label so that you can handle them differently.

Catch and recover

A try block wraps a group of steps. If an error happens inside, control jumps to an except block where you can look at the error, write a log, set a fallback value, or raise a clearer error. Without a try block, an error ends the execution as failed. With it, you choose what failure means for your business. For example, a failure to send a thank-you email should probably not cancel an order that is already paid for.

Retry with patience

Many errors are temporary. A service was restarting, the network blinked, a database was briefly busy. A retry policy tells the engine to try again after a delay. A good policy waits a little at first, then longer each time, up to a limit. This is called exponential backoff, and it prevents a struggling service from being hammered by thousands of impatient callers at once. The platform offers ready-made default policies, one for calls that are safe to repeat and one for calls that are not, and you can write your own policy with the number of tries, the starting delay, the maximum delay and the growth factor.

The danger of repeating non-safe calls

Retrying “read the price” is harmless. Retrying “charge the card” can charge the customer twice. This is why the idea of idempotency matters. A call is idempotent when doing it twice has the same effect as doing it once. Good payment services accept a unique request key: if the same key arrives again, they return the first result instead of charging again. When you design workflows, pass such keys on any call that changes the world.

Undoing work: the saga pattern

A workflow cannot wrap five different services in one giant database transaction. Instead, when a later step fails, it runs compensating steps that undo the earlier ones. If stock was reserved and the card was charged but the shipping label failed, the workflow refunds the card and releases the stock. This pattern is called a saga.

flowchart TD
  A["Reserve stock"] --> B["Charge card"]
  B --> C["Create shipping label"]
  C -->|"success"| D["Send confirmation"]
  C -->|"failure"| E["Refund card"]
  E --> F["Release stock"]
  F --> G["Notify customer of cancellation"]
  B -->|"failure"| F
    

Some steps cannot finish by themselves. They need a human decision or an answer from a slow outside system.

A callback lets a workflow pause on purpose. The workflow asks the engine to create a unique callback address, sends that address to whoever must reply, for example inside an approval email or a ticket, and then waits. When the person clicks approve or the outside system sends its answer to the address, the workflow wakes up and continues with the data it received. You can set a time limit so that the workflow does not wait forever, and you can handle the timeout like any other error.

Why callbacks beat polling

Without callbacks you would have to check every few minutes whether the answer has arrived. That wastes steps, adds cost and still delays the result by up to one polling interval. A callback wakes the workflow exactly when the answer comes in.

Writing a workflow is half the job. Putting it into production, safely and repeatably, is the other half.

Deploying a workflow

When you deploy, you give the definition a name, choose a region and choose the identity, called a service account, that the workflow will use when it calls other services. Every deployment creates a new revision. Executions use the revision that was current when they started, so deploying a change does not disturb runs already in progress. This is helpful in practice: an execution that began yesterday and is still waiting for approval finishes on yesterday’s logic.

Treat definitions as code

The definition file belongs in version control, next to the services it calls. Deploy it from an automated pipeline, using an infrastructure tool such as Terraform or a simple build script, and review changes the same way you review application code. Clicking changes together in the console is fine for learning, but it is a risky habit for production.

The common ways to start an execution

Choosing a region

Pick the region closest to the services the workflow calls and to your data. Latency between steps adds up, and some organizations must keep data in a specific country. Workflows replicates its state across zones inside a region, so a single zone failure does not stop your executions.

Environments

Use separate projects for development, testing and production, with the same definition deployed through the pipeline to each. Pass environment-specific values, such as service addresses, through inputs or runtime settings rather than editing the definition for each environment.

A workflow holds the keys to many systems at once, so it deserves the same care you give to any powerful automation.

Identity: who is the workflow?

Each workflow runs as a service account. Every call it makes to Google Cloud is made as that identity, and Google’s access control decides whether the call is allowed. This leads to the central rule of workflow security: give the service account only the permissions the workflow actually needs. A workflow that only reads from one storage bucket and publishes to one topic should hold exactly those two permissions, not broad editor rights over the whole project.

Who may start or change a workflow?

Separate the people and programs that may run a workflow from those who may edit it. A trigger service needs permission to start executions. A developer may need permission to deploy new revisions. These are different roles, and mixing them is a common mistake. Anyone who can change the definition and also use its service account can, in effect, borrow its power.

Signing in to other services

When the workflow calls your own Cloud Run services, it attaches a token proving who it is, and the receiving service checks the token. For Google Cloud programming interfaces it uses a different kind of token suited to those interfaces. In both cases the engine fetches and refreshes the tokens, so you never paste a password into a definition.

Protecting data

Data is encrypted in transit and at rest by default. Organizations with stricter rules can use their own encryption keys. Perimeter controls such as VPC Service Controls can limit which services may talk to each other and stop data from leaving a defined boundary. Be careful with what you log: if you write whole responses to logs, you may copy personal or sensitive information into a place with wider access.

Audit trail

Administrative actions, such as deploying or deleting a workflow, are recorded in the cloud audit logs. Combined with execution history, you can answer both “who changed the process” and “what did the process do.”

A workflow you cannot observe is a workflow you cannot trust. Here is how to see inside.

Execution history

The console shows each execution with its status, start and end time, input, output and the path it took through the steps. A visual graph highlights the steps that ran and the one that failed. This is usually the first place to look when someone says “order 42 is stuck.”

Logging

Workflows can send logs to Cloud Logging at several levels of detail: only errors, all calls, or nothing. Higher detail helps while you are learning and debugging, but it increases log volume and may record sensitive data. You can also write your own log lines from inside the workflow at meaningful moments, such as “payment accepted for order 42.” Meaningful, searchable messages with an order or request identifier turn a mystery into a five-minute search.

Metrics and alerts

Cloud Monitoring collects numbers such as how many executions started, how many finished, how many failed and how long they took. Build alerts on failure rate and on unusual duration. A sudden rise in failures means a downstream service is in trouble. A slow rise in duration often means a service is getting slower before it breaks.

Correlation

Pass a single identifier, such as an order number, through every call the workflow makes, and have each service include it in its own logs. Then one search shows the whole story across all services, which is the real value of tracing a business process.

Every managed service has boundaries. Knowing them early avoids unpleasant surprises in production.

Performance

Workflows is built for reliability and coordination, not for raw speed. Each step has a small amount of overhead, and a workflow made of thousands of tiny steps will feel slower than one function doing the same work in a single call. Use parallel branches for independent work, use connectors that wait efficiently instead of polling loops, and push heavy calculation into services designed for it.

Scale

The service handles many executions at the same time, up to a per-project, per-region concurrency quota. Quotas exist for the number of concurrent executions, the number of steps in one execution, the size of variables and responses, the maximum duration of an execution and the rate of starting executions. Exact numbers change from time to time, so check the current quota page of Google Cloud before designing something near an edge. If your design needs numbers close to a quota, either request an increase early or redesign.

Cost

Pricing is based on the number of steps executed, with internal steps (things that happen inside the engine) priced lower than external steps (calls to outside addresses). There is a monthly free allowance. Waiting costs nothing, so the shape of the bill is driven by how many steps run, not by how long executions live. The most common cost trap is a polling loop that checks something every few seconds for hours. Replace it with a connector or a callback.

No tool is perfect. An honest list of strengths and weaknesses is the best guide to deciding when to use it.

How it compares to neighbors

The central trade-off is control versus convenience. With your own code in a long-running service you can do anything, and you must also keep it alive, scale it and recover it. With Workflows you give up some freedom and gain a service that carries the burden of durability for you.

Experienced teams keep returning to the same shapes. Learn the good ones and recognize the bad ones early.

Patterns that work

Anti-patterns to avoid

A short checklist from teams who have run workflows in production for a long time.

Best practices

• Name steps clearly. Step names appear in logs and graphs. “charge_customer” is far better than “step7.”
• Give each workflow its own service account with the smallest set of permissions it needs.
• Make every changing call safe to repeat by sending a stable request key.
• Set timeouts everywhere. A call without a time limit can hold an execution far longer than you expect.
• Prefer connectors and callbacks over polling loops.
• Keep inputs and results small. Pass identifiers, not documents.
• Store definitions in version control and deploy through a pipeline.
• Log one meaningful line per important business moment, always with the business identifier.
• Alert on failures and on long durations, not only on total outages.
• Plan the undo path before you write the forward path.

Common mistakes

These scenarios show the patterns above in familiar business settings. They illustrate how large companies face these problems, and are not statements about any specific company’s internal tools.

Short answers to the questions people ask most often.

No. A function runs your code. Workflows runs a list of steps that calls services, possibly including functions. Use functions for work, Workflows for coordination.

Basic comfort with structured text and simple logic is enough. The language is small by design. You will still need to understand the services you call.

Yes. Executions can last a very long time, up to a documented maximum duration. Waiting does not consume paid compute, which is why approvals that take days are practical.

If you did not handle the error, the execution ends as failed and the error is recorded. If you wrapped the step in error handling, the workflow follows your retry or fallback path.

No. The engine may retry a call whose answer was lost, so the other service may run twice. Use idempotent services or unique request keys.

Keep them in a secret manager and let the workflow’s service account read them at run time. Never place them in plain text in the definition or input.

Yes, through the execution connector. The parent can wait for the child or move on. Use this for reusable, separately owned processes.

Open the execution in the console to see its input, output and step path, and search Cloud Logging for the business identifier. History is kept only for a limited time.

Choose it when you have large scheduled data pipelines with many interdependent tasks and want a mature scheduling tool for that. Choose Workflows for lightweight, serverless service coordination.

Avoid loops with many tiny steps, replace polling with connectors and callbacks, and push bulk work into services designed for it.

No. Limits and prices change from time to time. Always confirm the current numbers in the official quota and pricing pages before a big design decision.