Before learning any product, understand the problem it solves. Once the problem is clear, the features feel natural instead of random.

One server is never enough

A tiny website can live on a single server. That server receives every request, runs the application, and sends back answers. This works until one of three things happens. The number of visitors grows beyond what one machine can handle. The server crashes or needs maintenance. Or the application grows into several pieces, such as a shopping cart service, an image service, and an account service, each running on different servers.

The usual fix is to run many servers and place a load balancer in front of them. A load balancer is a device or service that receives incoming traffic and spreads it across a group of servers, so no single server is overwhelmed and failures can be hidden from visitors.

Why a plain load balancer is not always enough

Some load balancers only look at the envelope, not the letter. They see which address a message came from and which address it is going to, and they spread connections evenly. They have no idea what the visitor is asking for. A request for a product page and a request for a heavy video download look the same to them.

Modern web applications need more. They want to send requests for the pictures folder to one group of servers and requests for the checkout page to another. They want to handle the secure padlock (HTTPS encryption) in one central place instead of on every server. They want to block attacks such as someone trying to inject malicious commands through a web form. They want to redirect visitors who type the plain web address to the secure version. These jobs require a device that can read the contents of web requests. That is exactly what a Layer 7 load balancer does, and Application Gateway is Azure’s regional service of that kind.

The jobs of a smart front door

An illustrative example

Imagine a hypothetical ticket-selling website. It has a home page service, a search service, a seat-selection service, and a payment service, each running on its own set of virtual machines. On the night tickets for a popular concert go on sale, traffic grows twenty times larger within minutes. The company wants requests for search to hit the search servers, payment requests to hit the payment servers, attackers to be blocked before they reach any server, and capacity to grow by itself. One Application Gateway placed in front can do all of that, and the rest of this guide shows how.

A few simple ideas about how the web works will make everything else in this guide easy to follow. None of them are difficult, and each comes with a comparison.

Layers: the post office model

Network engineers describe communication in layers, a stack of jobs where each layer relies on the one beneath it. For our purposes two layers matter. Layer 4, the transport layer, deals with connections: which computer is talking to which computer, on which numbered door, called a port. Layer 7, the application layer, deals with the actual conversation: which web page was requested, which form data was sent, which language the browser prefers.

Anatomy of a web request

When you type a web address into a browser, a few things happen. First, your computer asks the Domain Name System (DNS), the internet’s phone book, to turn a name such as shop.example.com into a number called an IP address. Next, it opens a connection to that address. If the address starts with https, the two sides perform a TLS handshake, a quick exchange that sets up encryption. Finally, the browser sends an HTTP request.

An HTTP request has a few key parts. The method says what the browser wants to do, such as GET to fetch something or POST to send something. The path says which resource, such as /images/logo.png. The host header says which website name the visitor typed, which allows one server address to host many websites. Headers carry extra details such as cookies and language. Sometimes there is a body, such as the contents of a form. The server replies with a status code, a number that summarises the result: 200 means success, 404 means not found, 502 means a gateway received a bad answer from a server behind it, and so on.

HTTPS and TLS in plain words

TLS (Transport Layer Security) is the technology behind the padlock in the browser. It does two things. It encrypts the conversation so eavesdroppers see only scrambled data, and it proves the server’s identity using a certificate, a digital document issued by a trusted authority that says “this website really is who it claims to be.” Certificates expire, so they must be renewed, which is a classic source of outages.

Reverse proxy: the representative

A reverse proxy is a server that sits in front of other servers and speaks to the outside world on their behalf. Visitors talk to the proxy, and the proxy talks to the real servers, then passes the answers back. Application Gateway is a reverse proxy. This has a useful side effect: visitors never connect directly to your application servers, which can live on private addresses that the internet cannot reach.

Load balancing methods

How should a load balancer choose among servers? The simplest method is round robin, which takes servers in turn: first, second, third, then back to the first. It is fair when all servers and requests are similar. Other methods consider how busy each server is, or keep a visitor on the same server using a cookie so their shopping cart stays put. This last idea is called session affinity or sticky sessions, and we return to it later.

Latency, throughput, and connections

Three words recur in performance talk. Latency is the delay for one request, like the time for a single parcel to arrive. Throughput is the total amount of data moved per second, like how many parcels per hour a depot handles. A connection is a conversation channel kept open between two computers. A gateway must handle many connections, a high throughput, and low latency at the same time, which is why capacity planning matters.

With the basics in place, we can describe the service itself: what it is, what it is not, and how its generations differ.

The short definition

Azure Application Gateway is a managed, regional, Layer 7 load balancer for web traffic. “Managed” means Microsoft runs and maintains the underlying machines, applies updates, and keeps them healthy, so you configure behaviour rather than babysit servers. “Regional” means a gateway lives in one Azure region, such as West Europe, and serves applications in that region. “Layer 7” means it understands HTTP, HTTPS, HTTP/2, and WebSocket traffic, not just raw connections.

Two generations: v1 and v2

The service has existed in two main generations, called v1 and v2. The original v1 SKU (a SKU is simply a product tier or edition) had fixed instance sizes and manual scaling. The v2 generation, which is the one to use for any new design, brought autoscaling, support for spreading across availability zones, a static public address, faster deployment and updates, header and address rewriting, better performance, and integration with Azure Key Vault for certificates. Microsoft announced the retirement of v1 with a date in 2026, so existing v1 gateways should be migrated, and new projects should start with v2. Check the current Microsoft documentation for the exact status and migration help.

Standard and WAF flavours

Within each generation you choose between two flavours. The Standard flavour provides the load balancing and routing features. The WAF flavour adds a web application firewall, a security layer that inspects requests for attacks. In v2 these are called Standard_v2 and WAF_v2. If your application faces the public internet, the WAF flavour is strongly recommended, and many organisations require it by policy.

What it is not

Application Gateway is often confused with its neighbours, so let us be clear. It is not a global service: it does not route users to the nearest region of the world. It is not a general-purpose network load balancer: it speaks web protocols and is meant for web traffic. It is not an API management product with subscriptions, quotas, and developer portals. It is not a content delivery network that caches files near users. Each of those jobs belongs to a different Azure service, and chapter 13 compares them in detail.

Where it can send traffic

The servers behind a gateway are called backends. A backend can be Azure virtual machines, a virtual machine scale set (a group of identical virtual machines that can grow and shrink), servers identified by an IP address or a domain name, even servers outside Azure reachable over a private network connection, and multi-tenant services such as Azure App Service web apps. This flexibility means one gateway can front a mixture of old and new application parts.

An Application Gateway is assembled from a handful of building blocks. Learn the names and the relationships, and every configuration screen becomes readable.

flowchart LR
  USER["Visitor browser"] --> FIP["Frontend IP configuration"]
  FIP --> LST["Listener"]
  LST --> WAF["WAF policy"]
  WAF --> RULE["Request routing rule"]
  RULE --> BS["Backend settings"]
  RULE --> POOL["Backend pool"]
  BS --> POOL
  PROBE["Health probe"] --> POOL
  POOL --> VM1["Server 1"]
  POOL --> VM2["Server 2"]
  POOL --> APP["App Service"]

Frontend IP configuration

This is the address visitors use to reach the gateway. It can be a public IP address, reachable from the internet, a private IP address inside your virtual network, or both at once. Internal-only applications, such as a company intranet, use a private frontend. In v2, the public address is static and must use the Standard SKU of public IP. Newer capabilities also allow gateways that are private-only, with no public address at all; check the current documentation for how to enable that.

Listener

A listener is the part that waits for incoming connections on a specific combination of frontend address, port, and protocol. It answers the question “which traffic should I accept?” A basic listener accepts all traffic on a port. A multi-site listener also checks the host name, which is how one gateway can serve shop.example.com and blog.example.com separately. If the listener uses HTTPS, it holds the certificate used to establish encryption with the visitor.

Request routing rule

A rule connects a listener to a destination. It says “when a request arrives at this listener, send it to that backend pool using those backend settings,” or “redirect it to somewhere else.” A basic rule sends everything from the listener to one pool. A path-based rule looks at the path of the web address and sends different paths to different pools.

Backend pool

A backend pool is a named group of servers that can answer the same kind of request. The gateway spreads requests across the healthy members of the pool. A gateway can have many pools, such as one for the website, one for the API, and one for static images.

Backend settings

Backend settings (earlier called HTTP settings) describe how the gateway should talk to the servers in a pool. They include the protocol and port to use, whether to encrypt the connection to the backend, how long to wait for an answer, whether to use cookie-based session affinity, how to handle connection draining when a server is removed, and whether to override the host name sent to the backend.

Health probe

A health probe is a small, regular check the gateway makes against each backend server, such as asking for a particular page and expecting a good status code. Servers that fail the check are taken out of rotation until they recover.

Other pieces

• Rewrite rule sets: instructions to add, change, or remove request and response headers or parts of the web address.
• Redirect configurations: instructions to send visitors elsewhere, such as from HTTP to HTTPS.
• WAF policy: the set of security rules, which can be attached to the whole gateway, to a single listener, or even to specific paths.
• SSL and TLS settings: the certificates and the allowed protocol versions and cipher choices.
• Dedicated subnet: a gateway lives in a subnet of your virtual network reserved for it, so that its instances have private addresses to use.
• Private Link configuration: lets other networks reach the gateway privately, without crossing the public internet.

Follow a single request from the visitor’s browser to your server and back. Understanding each hop explains behaviours that otherwise seem mysterious.

sequenceDiagram
  participant B as Browser
  participant D as DNS
  participant G as Application Gateway
  participant S as Backend server
  B->>D: Where is shop.example.com
  D-->>B: Gateway public IP address
  B->>G: TLS handshake and HTTP request
  G->>G: Listener match, WAF check, rule match
  G->>S: New or reused connection with request
  S-->>G: Response with status code
  G->>G: Optional header rewrite
  G-->>B: Response to visitor

Step by step

Two separate connections

A vital idea is that there are two separate connections: one from the visitor to the gateway and another from the gateway to the server. The gateway is not a wire that passes bytes through; it is a go-between that ends one conversation and starts another. This explains why the server sees the gateway’s address as the sender, instead of the visitor’s address.

To keep the visitor’s real address available, the gateway adds a header called X-Forwarded-For listing the original sender. It can also pass the original host name and protocol. Applications that log visitor addresses or make decisions by country must read these headers rather than the connection’s source address.

Connection reuse

Opening a new connection and encryption session is expensive. The gateway keeps connections to backends open and reuses them for many requests, which reduces delay and server load. This is one reason a gateway can improve performance even though it adds an extra hop.

Choosing a server

By default the gateway distributes requests across healthy backend servers in a round robin fashion. If cookie-based affinity is on, the first response includes a cookie naming the chosen server, and later requests with the cookie go to that same server so long as it is healthy. This helps older applications that keep visitor state in a server’s memory, but it can make load uneven, so modern designs avoid needing it by keeping state in a shared database or cache.

Routing is the heart of an application gateway. This chapter explains the choices you have and when each one fits.

flowchart TD
  IN["Incoming request"] --> L1{"Which host name"}
  L1 -->|"shop.example.com"| R1{"Which path"}
  L1 -->|"blog.example.com"| P3["Pool: blog servers"]
  R1 -->|"/images/*"| P1["Pool: image servers"]
  R1 -->|"/api/*"| P2["Pool: API servers"]
  R1 -->|"everything else"| P4["Pool: web servers"]

Basic and multi-site listeners

A basic listener accepts every request that arrives at a certain address and port. It suits a gateway that serves a single website. A multi-site listener adds a check of the host name, so the same gateway, on the same public address and port, can serve many different website names, each with its own certificate and rules. This is called host-based routing, and it saves money because you do not need a separate gateway and public address for every site. You can even use wildcard host names, such as any name ending in example.com, when you want one listener to cover a family of subdomains.

Path-based routing

With path-based routing the gateway looks at the part of the web address after the host name. A rule can say that anything starting with /images/ goes to the pool of image servers, anything starting with /api/ goes to the API servers, and everything else goes to the default web servers. The rules are checked in order, and a default path catches all requests that match no other pattern.

This pattern lets a team split a large application into independent services behind one public web address, which is a typical step when moving from a single large application to smaller services. Each service can be scaled, deployed, and fixed on its own, while visitors still see a single website.

Redirects

Sometimes the right answer is not to forward a request but to send the visitor somewhere else. A redirect tells the browser, “ask this other address instead.” The most common use is redirecting plain HTTP visitors to HTTPS, so that nobody accidentally uses an unencrypted connection. Redirects can also move visitors from an old site name to a new one, or from one path to another. A permanent redirect tells search engines the move is lasting, and a temporary one tells them to keep the old address.

Rewrites

A rewrite changes part of a request or response as it passes through, without the visitor seeing any change in the browser’s address bar. Common rewrites include adding a security header to every response, removing a header that reveals server details, changing the path before sending it to the backend so the backend does not need to know about the public layout, and changing the host name header. Rewrites can depend on conditions, such as only acting when a certain header is present, and can use values from the request itself, called server variables.

Rule priority and order

When several rules could apply, the gateway needs a way to decide. In v2 each routing rule has a numeric priority, and the lowest number wins. Plan your priorities with gaps between them, such as 100, 200, and 300, so you can slot in a new rule later without renumbering everything. A very common source of confusion is a broad rule with a low number silently catching traffic meant for a more specific rule with a higher number.

A hypothetical example

A university runs one gateway with two multi-site listeners: one for the student portal and one for the library catalogue. The student portal listener has a path rule that sends /video/* to a pool of streaming servers and everything else to the general portal servers. A separate listener on port 80 redirects everyone to HTTPS. A rewrite rule adds a strict transport security header to every response. All of this lives in one gateway, with clear names and priorities.

Encryption is one of the biggest reasons people deploy an application gateway. It also causes some of the most common configuration problems, so it deserves careful attention.

Option one: TLS termination

TLS termination (sometimes called SSL offload) means the gateway ends the encrypted conversation with the visitor, reads the request, and then sends it to the backend over a plain, unencrypted connection. The expensive encryption work is done once, centrally, and the backend servers are freed from it. Only one place needs the certificate and private key. The trade-off is that traffic between the gateway and the servers is not encrypted, which is acceptable only when that stretch of network is private and trusted, and when your compliance rules allow it.

Option two: end-to-end TLS

With end-to-end TLS, the gateway terminates the visitor’s encrypted connection so it can read and route the request, and then opens a second encrypted connection to the backend. Data is encrypted on both stretches of the journey, though it is briefly visible in plain form inside the gateway. Regulated industries such as finance and healthcare often require this approach. The backend servers need their own certificates, and the gateway must be told how to trust them, by providing the trusted root certificate that issued the backend’s certificate.

Option three: TLS passthrough is not the same thing

Some people want the gateway to simply pass the encrypted bytes through without reading them. A classic Layer 7 gateway cannot do that, because it must read the request to route it. If you truly need a pure pass-through, a Layer 4 service such as Azure Load Balancer is the right tool. Newer versions of Application Gateway have added Layer 4 proxy features, so check the current documentation for their status and limits before relying on them.

Certificates: where they live

The certificate for a listener can be uploaded directly to the gateway as a file with a password, or, better, referenced from Azure Key Vault. Key Vault is Azure’s secure store for secrets and certificates. When the gateway references a Key Vault certificate, it uses a managed identity (an identity Azure creates for the gateway so it needs no stored password) to fetch the certificate. When you renew the certificate in Key Vault, the gateway picks up the new version automatically within a short time, which removes the dreaded “certificate expired at midnight” outage.

TLS policy: choosing protocol versions

TLS has versions, and older versions have known weaknesses. A TLS policy on the gateway sets the minimum protocol version and the allowed cipher suites, which are the specific encryption recipes. Modern guidance is to require TLS 1.2 or higher. Predefined policies exist that bundle sensible choices, and you can build a custom policy when compliance demands it. Setting the policy once at the gateway means every site behind it benefits, which is far easier than configuring each server.

Mutual TLS

Normally only the server proves its identity. With mutual TLS (mTLS), the visitor’s client must also present a certificate that the gateway verifies. This is common for business-to-business interfaces and device fleets, where only known clients should connect. The gateway can pass details of the client certificate to the backend through headers, so the application can make decisions based on who connected.

Backend certificate validation

For end-to-end TLS in v2, the gateway checks that the backend’s certificate comes from a trusted authority and that the name on it matches the host name the gateway uses to contact the backend. A mismatch produces a failed health check and a “502 Bad Gateway” for visitors. The usual fixes are to make the host name used by the gateway match the certificate, or to override the host name in the backend settings, and to supply the right root certificate if the backend uses a private authority.

Certificate habits that prevent outages

• Store certificates in Key Vault and reference them rather than uploading files by hand.
• Set calendar alerts or monitoring on certificate expiry at least thirty days ahead.
• Use certificates that cover the exact host names you serve, or well-chosen wildcards.
• Test a new certificate on a staging listener before switching production.
• Document which certificate authority issued each certificate and who renews it.

A gateway is only as good as its knowledge of the servers behind it. This chapter explains how it learns which servers are healthy and how it treats visitors who move between them.

Health probes: the pulse check

Every few seconds, the gateway sends a small request to each backend server and watches the answer. If the server answers with an acceptable status code in time, it is marked healthy. If it fails several times in a row, it is marked unhealthy, and the gateway stops sending visitors to it. When it passes again, it returns to rotation. This happens automatically, which is how failures stay invisible to visitors.

A default probe is created for you, which contacts the backend on the same port and protocol as the backend settings and requests the root path. The documented defaults are a check every 30 seconds, a 30-second timeout, and three failures before marking a server unhealthy. A custom probe lets you choose the path, the host name, the interval, the timeout, the failure threshold, and what counts as a good answer, such as status codes between 200 and 399.

Writing a good health endpoint

The default probe asks for the home page, which may be slow, may redirect to a login page, or may return success even when the database is broken. It is far better to build a dedicated health endpoint in your application, a small page such as /health, that quickly checks the things the application truly needs and answers with a clear success or failure. Keep it light, because the gateway calls it constantly from every instance. Avoid making it so deep that a slow optional dependency causes all servers to be marked unhealthy at once, which would take the whole site offline.

Backend settings that matter

Session affinity and why to be wary

With cookie-based session affinity, the gateway sets a cookie on the first response, naming the server that answered. Future requests with that cookie go to the same server. It is a bridge for older applications that keep each visitor’s state, such as a shopping basket, inside one server’s memory. The costs are uneven load, since popular sessions pile onto certain servers, and lost sessions when a server fails or is replaced. The healthier design is a stateless application that stores visitor state in a shared place such as a database or a distributed cache, so any server can serve any visitor.

Connection draining

When you remove a server from a pool, for example during an update, you do not want to cut off visitors halfway through a download. Connection draining lets the gateway stop sending new requests to that server while allowing existing requests to finish, up to a time limit you set. Used together with rolling updates, it gives updates with no visible interruption.

Talking to App Service and other multi-tenant backends

Azure App Service hosts many customers’ websites behind shared addresses and chooses which site to serve by looking at the host name header. If the gateway forwards the original visitor-facing host name, App Service may not recognise it. The usual solution is to have the gateway send the App Service’s own default host name to the backend, while the visitor continues to see the custom name. If your application builds absolute links or sets cookies using the host name it receives, this change can break things, so test login flows carefully, and consider keeping the original host with the right custom domain configured on the App Service instead.

The internet is full of automated scanners probing every public website for weaknesses. A web application firewall is your first line of defence against the most common attacks.

What a WAF does

A web application firewall (WAF) inspects the contents of web requests and compares them to rules describing known attack patterns. Unlike a traditional network firewall, which mainly looks at addresses and ports, the WAF reads the web request itself: the path, the query string, the headers, the cookies, and the body. Because the gateway already terminates and reads each request, adding the WAF is a natural fit.

The attacks it helps stop

• SQL injection: tricking a database by sneaking commands into a form field or web address.
• Cross-site scripting: planting malicious scripts that run in other visitors’ browsers.
• Command injection and file inclusion: tricking a server into running instructions or loading files it should not.
• Protocol violations: malformed requests designed to confuse servers.
• Scanners and bad bots: automated tools hunting for weaknesses, or scraping content aggressively.

Managed rule sets

Microsoft provides managed rule sets based on the OWASP Core Rule Set, a respected open collection of rules for the most common web attack types, and newer managed rule sets that Microsoft updates over time. You choose which set and version to apply. A separate bot protection rule set classifies automated visitors as good bots, such as search engines, bad bots, or unknown, so you can allow, block, or watch them. Rule set names and versions change over time, so check the current documentation when you plan a deployment.

Custom rules

Besides the managed rules, you can write custom rules: your own conditions and actions. Examples include blocking requests from specific countries or address ranges, allowing only your office addresses to reach an admin path, blocking requests with a certain header value, and applying rate limiting, which restricts how many requests a single client may make in a time window to blunt brute-force and scraping attempts. Custom rules are evaluated before managed rules and have priorities, so order matters.

flowchart TD
  REQ["Incoming request"] --> CR{"Custom rules match"}
  CR -->|"Allow rule"| OK["Forward to backend"]
  CR -->|"Block rule"| BLK["Return 403 and log"]
  CR -->|"No match"| MR{"Managed rules score"}
  MR -->|"Below threshold"| OK
  MR -->|"Above threshold"| MODE{"Mode"}
  MODE -->|"Prevention"| BLK
  MODE -->|"Detection"| LOGONLY["Log and forward"]

Detection mode and prevention mode

The WAF can run in two modes. In detection mode it only records what it would have blocked and lets everything through. In prevention mode it actively blocks matching requests, normally returning a 403 Forbidden response. The wise rollout is to start in detection mode, study the logs for a couple of weeks, tune out false alarms, and then switch to prevention. Turning on prevention for a live application without testing is a reliable way to block legitimate customers.

False positives and exclusions

A false positive is a harmless request that looks like an attack. A text field where a customer pastes a code sample, or a rich-text editor that sends HTML, may trigger rules built to catch script injection. The remedy is not to switch the WAF off. Instead, you add an exclusion, which tells a specific rule to ignore a specific field, or you disable one rule for one path. Keep a record of every exclusion and the reason for it, because each one is a deliberate small hole.

Policies and where they apply

Modern WAF settings live in a separate resource called a WAF policy, which can be attached to the whole gateway, to individual listeners, or to specific paths. This lets you apply strict rules to a login page and a lighter set to a public image folder, or give different teams their own policies on a shared gateway. Using policies also makes it easy to reuse the same security configuration across many gateways.

A front door that collapses during the busiest hour is worse than no door. This chapter explains how Application Gateway grows, how to estimate its size, and how to keep it available.

Autoscaling in v2

A v2 gateway can autoscale: you give it a minimum and a maximum number of instances, and Azure adds or removes instances as traffic changes. Scaling out takes a few minutes, so it is not instantaneous. A sudden spike larger than your current capacity can briefly cause slowness or errors before new instances arrive. For known events, such as a planned sale, set a higher minimum instance count beforehand, and for unpredictable spikes keep a comfortable cushion above your normal level.

Capacity units

In v2, you are billed and measured in capacity units. A capacity unit is a blend of three kinds of load, and the gateway counts whichever dimension is highest at a given moment. The documented approximations are that one capacity unit can handle about 2,500 persistent connections, or about 2.22 megabits per second of throughput, or one compute unit of processing, where a compute unit is roughly ten new TLS connections per second using common 2048-bit certificates. These figures are guides, and real results depend on your pages, certificate type, rules, and WAF settings, so always load test.

Each gateway instance supports roughly ten capacity units, so a load needing about 90 capacity units would settle at around nine instances. The WAF, large response bodies, and complex rewrite rules all increase the processing share. Plan your maximum to be comfortably above your biggest expected peak, because hitting the maximum means no more growth.

Availability zones and redundancy

An availability zone is a physically separate data centre location within an Azure region, with independent power, cooling, and networking. A v2 gateway can be zone redundant, meaning its instances spread across zones, so the loss of an entire data centre does not take the gateway down. This is the default choice for production. Your backend servers should also be spread across zones, otherwise the gateway survives but its servers do not.

Beyond one region

Because Application Gateway is regional, a regional outage is outside what a single gateway can survive. For the highest level of availability, teams deploy a gateway and application stack in two regions and place a global service, such as Azure Front Door or Traffic Manager, in front to steer visitors to the healthy region. Chapter 13 explains how these fit together.

Performance habits

• Enable HTTP/2 for visitor-facing listeners to reduce delays for pages with many resources.
• Keep the gateway close to the backends in the same region, ideally the same zones, to cut round-trip time.
• Keep rewrite rules and custom WAF rules lean, because every rule costs processing time.
• Tune timeouts to match real application behaviour, so slow operations do not fail while stuck requests do not hold connections forever.
• Load test before launch, including TLS handshake rates, because handshakes are far more expensive than reused connections.
• Serve static files elsewhere when possible, such as from a content delivery network, so the gateway handles mainly dynamic requests.

A gateway does not float in empty space. It lives inside your virtual network, and its placement decides who can reach it and what it can reach.

flowchart LR
  NET["Internet visitors"] --> PIP["Public IP"]
  PIP --> AGW["Application Gateway in its own subnet"]
  AGW --> WEB["Web subnet servers"]
  AGW --> API["API subnet servers"]
  AGW --> PAAS["App Service through private endpoint"]
  CORP["Corporate network via VPN or ExpressRoute"] --> AGW
  AGW --> LOG["Log Analytics workspace"]

The dedicated subnet

A virtual network is your private network in Azure, divided into subnets, which are smaller address ranges. Application Gateway needs a subnet used only for gateways, because it creates its instances there and hands each one a private address. You cannot mix v1 and v2 gateways in the same subnet, and you should not place virtual machines or other services in it. The common guidance for v2 is a subnet of size /24, which provides 256 addresses, so there is room for the maximum number of instances plus upgrades and growth. A subnet that is too small is a classic reason for being unable to scale.

Network security groups

A network security group (NSG) is a set of allow and deny rules for network traffic. You can attach one to the gateway’s subnet, but v2 has mandatory requirements. You must allow inbound traffic from the GatewayManager service tag on ports 65200 through 65535, which Azure uses to manage the gateway, and you must allow traffic from the Azure load balancer probes. You must also allow the visitor ports you serve, typically 80 and 443. If any of these is blocked, the gateway may show errors or fail to update. Always check the latest Microsoft guidance, since requirements evolve.

On the backend side, remember that backend servers must allow traffic from the gateway’s subnet on the port the gateway uses. A server NSG that only allows traffic from the internet will refuse the gateway, and the health probe will fail. This is one of the most common reasons for a brand-new gateway showing all servers unhealthy.

User-defined routes

A route table with user-defined routes (UDRs) overrides Azure’s default traffic paths, often to send traffic through a firewall appliance. On the gateway subnet, be careful. Management traffic must reach Azure’s control plane directly, so a default route that sends everything to a firewall can break the gateway. Specific supported patterns exist for sending backend-bound traffic through a firewall, and the allowed combinations have changed over time, so follow the current documentation closely and test in a non-production environment first.

Public, private, or both

A gateway can have a public frontend for internet visitors, a private frontend for internal visitors reachable only inside your network or through a VPN or private link to your office, or both with different listeners. Internal applications such as HR portals typically use only a private frontend. Many organisations also want a gateway with no public address at all, and recent capabilities make private-only deployments possible with added network controls. Confirm the current feature status and requirements before you design around it.

Private endpoints and Private Link

A private endpoint gives a platform service, such as a web app or storage account, a private address inside your network. A gateway can route to a backend through such an address, so the backend can disable all public access. In the other direction, Private Link on the gateway lets other networks, including other subscriptions and organisations, connect to the gateway privately without crossing the public internet. These two tools help you reduce the public surface of your whole application.

The hub and spoke layout

Larger organisations commonly organise their network as a hub and spoke. A central hub network holds shared services such as firewalls and VPN connections, and each application lives in a separate spoke network connected to the hub. An application gateway typically lives in the spoke of the application it serves, close to its backends. Some designs put a shared gateway in the hub to serve many applications, which saves money but concentrates risk and makes teams share rules, so decide with your security and platform teams.

Different kinds of backends call for different patterns. Here is how a gateway works with the most common ones.

Virtual machines and scale sets

The simplest backend is a group of virtual machines, or a virtual machine scale set, a group of identical machines that can grow and shrink. You can add the network interfaces of the machines to a backend pool, and when the scale set adds a new machine, it joins the pool automatically. This is the classic pattern for web servers and application servers, and it works well with rolling updates, since you can take one machine out at a time with connection draining.

App Service and other platform services

For Azure App Service and similar managed platforms, you add the service’s address as a backend target. The key concern, covered in chapter 8, is the host name. A second concern is that, by default, the App Service is also reachable directly on the internet, so visitors might bypass the gateway and its WAF. To close that gap, restrict the App Service to accept traffic only from the gateway, using access restrictions based on the gateway’s address or a private endpoint.

Kubernetes: AGIC and Application Gateway for Containers

Kubernetes is a system that runs and manages containers, which are small, packaged units of software. A Kubernetes cluster needs an ingress: a component that brings outside web traffic to the right service inside the cluster. Azure offers two ways to use gateway technology for this.

In both approaches, the person managing the application describes routing in Kubernetes files, and the controller translates it into gateway settings. This keeps routing rules next to the application’s own configuration. The trade-off is that the cluster controller now owns the gateway’s configuration, so manual changes made in the portal may be overwritten. Treat the controller as the single source of truth.

Multiple backends behind one gateway

A real company often has a mix: an older application on virtual machines, a new customer portal on App Service, and microservices on Kubernetes. One gateway can front all three, using multi-site listeners and path rules to separate them. This can be the glue that lets teams modernise one piece at a time, since visitors do not notice whether /orders lives on an old server or a new container.

The strangler pattern

This is a well-known approach for gradually replacing an old system. You place a gateway in front of the old application. Then you build a replacement for one feature, such as search, and add a path rule that sends /search to the new service while everything else continues to go to the old application. Feature by feature, the old system shrinks until nothing is left. The name comes from a vine that slowly grows around a tree and eventually replaces it.

Azure has several services that distribute traffic, and their names sound alike. Knowing the differences prevents buying the wrong tool.

flowchart TD
  A["Need to distribute traffic"] --> B{"Is it web HTTP or HTTPS traffic"}
  B -->|"No, any TCP or UDP"| C["Azure Load Balancer"]
  B -->|"Yes"| D{"Single region or global"}
  D -->|"Single region"| E["Application Gateway"]
  D -->|"Global, many regions"| F{"Need edge caching and acceleration"}
  F -->|"Yes"| G["Azure Front Door"]
  F -->|"DNS level steering is enough"| H["Traffic Manager"]

The four main services

Combining them

These services are not rivals. They are layers. A well-known pattern places Azure Front Door at the global edge for acceleration and early filtering, with one Application Gateway in each region for detailed routing and in-network protection, and Load Balancer or direct pools for servers behind. If you do this, lock the gateway so it accepts web traffic only from Front Door, using the Front Door service address tag and by verifying the Front Door identifier header, so attackers cannot skip the edge and hit the gateway directly.

Which to choose when

• One region, a web application, you need a WAF and routing inside your network: Application Gateway.
• Users around the world, you need fast delivery and automatic regional failover: Front Door, often combined with gateways.
• Non-web traffic such as a database or a game server: Load Balancer.
• Only need simple DNS-based failover, possibly for non-web endpoints: Traffic Manager.
• You want API subscriptions, quotas, and a developer portal: look at API Management, and place a gateway in front of it if you need a WAF.

Security is never one switch. It is a set of layers, so that the failure of one does not expose everything. Here is how the layers around an application gateway fit together.

Layer by layer

Hardening the response

The gateway can add security headers to every response using rewrite rules. Examples include a strict transport security header that tells browsers to always use HTTPS, a header that prevents browsers from guessing file types, and a header that limits where framing and scripts may load from. It can also strip headers that reveal server software and versions, which would help an attacker choose an exploit. This is a cheap, centralised improvement for every application behind the gateway.

Controlling who changes the gateway

A misconfigured rule on a gateway can expose or break an entire application, so changes should be controlled. Use Azure role-based access control to limit who can modify gateway resources, apply resource locks to prevent accidental deletion, require changes through infrastructure code reviewed by a second person, and record every change in the activity log. Treat the gateway like production code, because it is.

Protecting the direct path

A frequent weakness is the bypass: a path to the backend that avoids the gateway. If a web server still has a public address with open ports, attackers can find it by scanning and simply skip the WAF. Close every other door. Remove public addresses from backends where possible, allow inbound traffic only from the gateway subnet, and verify with an external scan that the backends are not reachable directly.

Compliance and data protection

The gateway sees decrypted request data, including form fields and sometimes sensitive personal information, so treat its logs with care. Avoid logging full request bodies. Limit who can read the logs, set a retention period that matches your policies, and mask sensitive values where possible. When regulations such as data residency apply, remember that a gateway is regional, so logs and traffic stay in the region you chose, which helps compliance but requires careful planning when you use global services in front.

Every gateway fails to behave at some point. The skill is to find out which part is at fault in minutes instead of hours.

Three sources of truth

Send diagnostics to a Log Analytics workspace, a central store you can search with a query language. Combine it with alerts so the right person is told when something crosses a threshold, for example when unhealthy hosts exceed zero for five minutes, when the failed request rate rises, or when capacity units approach the maximum you set.

Reading the common error codes

A step-by-step method

Typical root causes of unhealthy backends

• The probe asks for a path that returns an error or a redirect the probe does not accept.
• The probe host name does not match the backend’s certificate or web site binding.
• A backend NSG or server firewall blocks the gateway subnet.
• The backend’s certificate has expired, or its authority is not trusted by the gateway.
• The application is up, but its health endpoint depends on something broken.
• The backend is only listening on a different port from the one in backend settings.

Network Watcher and other helpers

Azure Network Watcher offers tools to test connectivity between the gateway subnet and the backends, view effective network rules, and capture packets when you need to see exactly what crossed the wire. Use them when logs show the gateway sent a request but no answer came back, since the problem is then in the network path rather than in the gateway’s settings.

A gateway is not set up once and forgotten. It has a lifecycle of design, build, test, release, change, and retirement. Handling that lifecycle with discipline is what keeps a front door reliable.

flowchart LR
  A["Design"] --> B["Build as code"]
  B --> C["Test in staging"]
  C --> D["Release to production"]
  D --> E["Monitor and tune"]
  E --> F["Change safely"]
  F --> C
  E --> G["Retire or migrate"]

Design: write the decisions down

Before creating anything, record the basics. Which applications will sit behind the gateway? Which host names and paths? Public, private, or both? Which certificates and who owns them? Which WAF mode and rule set? What peak traffic do you expect, and what minimum and maximum instance counts follow from it? Which regions and zones? A short design document written first saves long meetings later, and it becomes the reference for the next engineer.

Build as code

Infrastructure as code means describing your cloud resources in text files that a tool reads and turns into real resources. Azure offers Bicep, ARM templates, and Terraform support for this. For a gateway, which has many interlocking pieces, code is far better than clicking in a portal. It is repeatable, so staging and production match. It is reviewable, so a colleague can spot a risky change before it ships. It is recoverable, so you can rebuild a gateway after a disaster. And it leaves a history of who changed what and why.

Test in staging

Run a staging gateway that mirrors production in structure, even if smaller. Test the full chain: DNS, certificate, listener, WAF, rule, backend, probe, and application behaviour such as login and file uploads. Run a load test to learn the capacity units your traffic really uses. Test failure: stop a backend and watch traffic move away, expire a test certificate on purpose, and break the health endpoint to see what visitors experience.

Release and change safely

Changes to a gateway can affect all visitors at once, so use patterns that limit risk. One approach is blue-green deployment: you keep the current backend pool (blue) and prepare a new one (green) with the updated application, test green privately, and then switch a routing rule to point to it, with the option to switch back within seconds. Another approach is canary release: send a small share of traffic to the new version first, using a separate path or a separate listener for testers, and expand if all looks well. Always keep changes small and reversible.

Operate: the regular chores

• Certificates: track expiry dates and renewals, and test renewed certificates early.
• WAF tuning: review blocked and flagged requests monthly and adjust exclusions.
• Capacity review: compare peak capacity units against your maximum every quarter.
• Access review: check who can change the gateway and remove people who no longer need access.
• Rule review: remove old listeners, rules, and pools that nobody uses, since clutter hides mistakes.
• Documentation: keep the design note and run book in step with reality.

Migrating from v1 to v2

If you still run a v1 gateway, plan its migration. Microsoft provides guidance and tooling to copy the configuration of a v1 gateway into a new v2 gateway. Key points to plan for include the new gateway getting a new public address, so DNS records must be updated at cutover. Certificates and backend trust settings must be re-checked, because v2 validates backend certificates more strictly. Health probes and rewrite behaviour may differ slightly, so test every listener. A good approach is to build the v2 gateway alongside the old one, test it using a temporary host name, lower the DNS cache time in advance, and then switch DNS and keep the old gateway running for a short fallback period.

A design pattern is a proven answer to a recurring problem. An anti-pattern is a tempting approach that keeps causing trouble. Here are the most useful of each for application gateways.

Patterns worth adopting

Anti-patterns to avoid

More anti-patterns in brief

• One giant rule set nobody understands. Hundreds of unnamed rules with overlapping priorities. Split by application and name everything.
• Turning off the WAF to fix an error. Instead, find the exact rule and add a narrow exclusion.
• Tiny subnet, maximum instances set high. The gateway cannot scale because there are no free addresses.
• Using affinity to hide an unhealthy design. It masks state problems that will bite later.
• Making manual portal changes to a code-managed gateway. The next deployment erases them, usually at the worst moment.

This chapter gathers the habits that separate calm operations from late-night panics, and explains where the money goes so the bill holds no surprises.

Where the cost comes from

Application Gateway v2 is billed mainly in two parts: a fixed hourly charge for having the gateway running, and a variable charge based on the capacity units it consumes each hour. The WAF flavour costs more than the Standard flavour. You also pay for the public IP address, for data leaving Azure, and for the logs you store. Prices differ by region and change over time, so always check the current price list or use Azure’s pricing calculator when you plan.

Habits that pay off

Common mistakes, in plain words

• Forgetting the management ports. A network security group that blocks the gateway management range leaves the gateway unable to update or report health.
• Mismatched host names. The gateway, the backend, and the certificate disagree on the name, causing 502 errors or redirect loops.
• Setting timeouts too short. Report generation or large uploads fail with 504 errors even though the backend would have finished.
• Counting only on autoscale. Scale-out takes minutes, so large planned peaks need a raised minimum.
• Ignoring the 413 limit. File uploads fail because of body size limits in the WAF or backend.
• Not testing renewals. A renewed certificate may have a different chain or name, and the first real test happens in production.
• Treating WAF logs as noise. They are the best early warning of real attacks and of broken legitimate flows.
• Mixing environments. One shared gateway for test and production lets a test mistake affect real customers.

The scenarios below are illustrative composites that show how the ideas combine. The organisations are invented and the numbers are chosen to teach, not to describe any real company.

What these scenarios share

In each case, the gateway is used as a place to apply consistent rules once instead of repeating them everywhere: encryption, routing, security, and visibility. Successful teams plan capacity before peaks, close the back doors, treat certificates and health checks as first-class concerns, and keep their gateway configuration in reviewed code.

Short answers to the questions beginners ask most often.

You began with the problem of too many visitors and too many servers, and you now have a complete map of one of Azure’s most important web services: what it is, how it works inside, how to configure it, secure it, scale it, watch it, and fix it.

Think of Application Gateway as the reception desk of a large hotel, run by a professional service. It reads each request, decides where it belongs, checks it for danger, speaks the right language of encryption, watches the health of every department, and brings in extra staff when the lobby fills. You supply the rules, and Azure supplies the staff. The better your rules and your preparation, the calmer your busiest day will be.