Before settings and diagrams, we need a clear picture of the problem Azure Monitor solves and of the pieces it is made from.

The short definition

Azure Monitor is the Azure service that collects, stores, analyses, and acts on telemetry from your cloud and on-premises environments. Telemetry is the word for the stream of measurements and messages that systems send about themselves: how busy a server is, how long a web request took, which error a program raised, or who changed a setting. Azure Monitor does not run your application. It watches your application and everything around it.

The service is not one tool but a family of connected capabilities that share one data platform. It collects data from many sources, stores it in purpose-built stores for numbers, text records, and traces, gives you tools to explore the data, and provides alerts and automatic actions so that people and systems can respond. Many Azure services also use it quietly: when you open the monitoring page of almost any Azure resource, you are looking at Azure Monitor.

Monitoring versus observability

Traditional monitoring asks known questions: is the server up, and is the disk full? Observability goes further. It lets you ask new questions about problems you did not predict, by exploring rich data about what the system did. A system is observable when its outputs give enough clues to explain its inner state. Azure Monitor supports both: simple health checks and threshold alerts for known risks, and deep search and tracing for surprises.

What it is not

Azure Monitor is not a security information and event management product, although its logs often feed one such as Microsoft Sentinel, which builds on Log Analytics workspaces. It is not a replacement for a service desk or for a deep profiler used by developers on a single machine. It is also not free: collecting and keeping data costs money, so a large part of using it well is deciding what is worth collecting, a theme we will return to often.

Why it matters to the business

Outages cost money and trust, and slow pages quietly push customers towards competitors. Monitoring shortens the time between a problem starting and a person understanding it. That gap, often measured as time to detect and time to resolve, is where most damage happens. Teams with good telemetry find faults in minutes, fix the right thing the first time, and spend less effort arguing about whose component failed. The evidence in charts and traces replaces opinion with fact, which also improves relationships between development, operations, and security teams.

A handful of terms appear in every conversation about Azure Monitor. Learn them once and the rest of the guide becomes easy to read.

Two ways to read the same data

Metrics answer “how much and how often” quickly. Logs answer “what exactly happened”. A healthy design uses both: a metric alert tells you that error counts jumped, and a log search tells you which error and why. Traces then connect the failing request across services. These three views of one system are sometimes called the pillars of observability.

Platform data versus data you add

Some data flows automatically. Every Azure resource produces platform metrics, and the control-plane activity log records management operations. Other data must be switched on: resource logs through diagnostic settings, guest operating system data through the agent and rules, and application telemetry through instrumentation. Knowing which is which explains why a new resource may show charts in the portal on day one but have empty log tables until you configure collection.

Scope: where monitoring settings live

Azure organises resources in a hierarchy of management groups, subscriptions, resource groups, and resources. Monitoring settings follow this hierarchy. Alert rules and diagnostic settings attach to resources or scopes, workspaces belong to a subscription and region, and policies assigned high in the tree reach everything beneath. Thinking in scopes helps you place each setting at the lowest level that still covers what you need, and it explains why a policy at the top can quietly create settings on resources that teams never touched.

Azure Monitor can be understood as a pipeline with four stages: sources, collection, storage, and consumption.

flowchart LR
    subgraph SRC[Sources]
        A1[Applications]
        A2[Virtual machines]
        A3[Containers and Kubernetes]
        A4[Azure resources]
        A5[On-premises through Azure Arc]
    end
    subgraph COL[Collection]
        C1[Instrumentation and OpenTelemetry]
        C2[Azure Monitor Agent with rules]
        C3[Diagnostic settings]
        C4[Platform metrics and activity log]
    end
    subgraph STO[Data platform]
        S1[Metrics database]
        S2[Log Analytics workspace]
        S3[Azure Monitor workspace]
    end
    subgraph USE[Consumption]
        U1[Metrics explorer]
        U2[Log queries]
        U3[Workbooks and dashboards]
        U4[Alerts and action groups]
        U5[Autoscale]
    end
    A1 --> C1
    A2 --> C2
    A3 --> C2
    A4 --> C3
    A4 --> C4
    A5 --> C2
    C1 --> S2
    C2 --> S2
    C2 --> S3
    C3 --> S2
    C4 --> S1
    S1 --> U1
    S2 --> U2
    S3 --> U1
    S2 --> U3
    S1 --> U4
    S2 --> U4
    S1 --> U5
        

Stage one: sources

Sources include your own applications, the guest operating systems of virtual machines, Kubernetes clusters and the containers inside them, every Azure resource, and machines outside Azure that are connected through Azure Arc. Tags, resource groups, and subscriptions describe where each source belongs, which later helps you filter and control access.

Stage two: collection

Different sources use different collection paths. Azure resources send platform metrics automatically and send resource logs when a diagnostic setting asks them to. Machines run the Azure Monitor Agent, which follows data collection rules. Applications use instrumentation libraries, increasingly based on OpenTelemetry, an open standard for producing telemetry. Container platforms use managed collectors for logs and for Prometheus metrics. Whichever path is used, rules can filter or reshape data before it is stored, which is the cheapest moment to remove waste.

Stage three: storage

Azure Monitor keeps data in stores built for its shape. Platform metrics go to a time-series database that is fast for charts and alerts. Logs and traces go to Log Analytics workspaces, where each kind of record lives in a table. Prometheus and OpenTelemetry style metrics go to Azure Monitor workspaces. These workspace types are separate resources with separate query languages, a point that surprises many newcomers.

Stage four: consumption

People use portal charts, query screens, workbooks, shared dashboards, and managed Grafana. Machines use alert rules, action groups, autoscale settings, data export to other systems, and application programming interfaces. The same stored data can serve all of them, which is why the storage design matters so much.

Latency and ordering across the pipeline

Each stage adds a little delay, and the delays differ by data type. Platform metrics travel the shortest path and usually appear within a minute or two. Logs pass through collection, optional transformation, and ingestion before they can be queried, so they appear later. Knowing this order explains many puzzles, such as a chart that shows a spike while the matching log search is still empty. When you design alerts or investigate an incident, always ask which stage the missing data might still be sitting in.

Each data type has its own strengths, costs, and best uses. Understanding them prevents the two classic errors: logging everything and measuring nothing.

Metrics: light and fast

A metric is a number with a time and a few labels, called dimensions. Examples are processor percentage, request count by status code, or queue length by queue name. Metrics are small, so they can be collected every minute or faster and kept in a fast store. They are the best fuel for dashboards and for alerts that must react quickly. Their weakness is detail: a metric tells you that errors rose but not which user hit them.

Logs: rich and searchable

A log record is a row with many columns. Resource logs record what a service did, such as each request to a storage account or each firewall decision. Application logs record what code decided to say. Operating system logs record events from the guest machine. Logs are stored in tables inside a Log Analytics workspace, and each table has a schema. You query them with KQL, which is designed for filtering, grouping, joining, and shaping large amounts of data. Logs cost more than metrics because they are bigger, so volume must be managed.

Traces: the story of one request

When a customer clicks a button, the request may pass through a gateway, a web service, a queue, a database, and a payment service. A trace follows this path with timing at each hop, and a shared identifier ties the pieces together. Traces answer the hardest question in distributed systems: where did the time go, and where did it break? Application Insights collects them, and the data lands in a Log Analytics workspace.

Changes and events

Many incidents start with a change. The activity log records control-plane operations such as creating a resource or editing a rule. Service Health and Resource Health events tell you about problems on the Azure side. Change analysis helps connect a recent change to a new problem. Putting a change timeline next to a metric chart often reveals the cause in seconds.

flowchart TD
    Q[A problem is reported] --> M{Start with metrics}
    M -->|Spike or drop found| L[Search logs for the time window]
    L --> T[Follow a failing request with traces]
    T --> C[Check recent changes and health events]
    C --> R[Root cause and fix]
    M -->|Nothing unusual| L
        

Where each type is stored

Dimensions, cardinality, and the cost of detail

Labels make data useful, but too many distinct label values cause trouble. A label such as user identifier may take millions of values, and storing a separate series for each makes charts slow and bills large. This is called high cardinality. Use labels for groups you actually compare, such as region, status code, or service name, and keep unique identifiers in logs or traces where they belong. This habit matters most for Prometheus style metrics, where every unique label combination becomes its own stored series.

Collection is where most cost and most mistakes are born. This chapter explains each path and how to control it.

Azure Monitor Agent and data collection rules

The Azure Monitor Agent runs on Windows and Linux machines in Azure, on-premises, and in other clouds. It does nothing until it is told what to collect, and that instruction comes from data collection rules. A rule names the data sources, such as performance counters, event logs, or text files; optionally defines a transformation that filters or reshapes records; and names the destination, such as a particular workspace. A rule is associated with machines, and one machine can have several associations. Because rules are separate resources, one standard rule can be applied to a thousand machines, and changing the rule changes them all.

Transformations: cleaning data before it is stored

A transformation is a small query that runs inside the pipeline before storage. It can drop noisy rows, remove sensitive columns, or add helpful fields. Since ingestion is usually billed by volume, removing data here saves money and risk at the same time. Transformations also let you send different slices of one stream to different destinations, for example sending security-relevant events to one workspace and performance data to another.

Diagnostic settings for Azure resources

Most Azure services can emit resource logs and metrics through a diagnostic setting. Each setting picks categories of logs and chooses one or more destinations: a Log Analytics workspace for analysis, a storage account for cheap long-term keeping, an event hub to stream data to other systems, or a partner solution. Because each resource needs its own setting, doing this by hand for hundreds of resources is slow, so teams use Azure Policy to deploy settings automatically.

Application instrumentation

Applications are watched through Application Insights, a feature of Azure Monitor for application performance monitoring. It is built on OpenTelemetry, which means one set of libraries can produce traces, metrics, and logs in a vendor-neutral way. Some platforms, such as certain web hosting services, can attach monitoring without code changes, while others need a small library added by developers. The data includes request rates, response times, failure rates, dependency calls, exceptions, and page views from browsers.

Containers and Prometheus

For Kubernetes, managed collection gathers container logs and cluster data. A managed service for Prometheus stores metrics in an Azure Monitor workspace, and managed Grafana can draw them. Teams that already use Prometheus find that their existing metrics, rules, and dashboards carry over, with Azure handling the storage and scaling.

Checking that collection really works

After configuring any path, prove it. Look for a recent heartbeat from each machine, confirm that the expected tables contain rows from the last few minutes, compare record counts with what the source should produce, and check that sensitive fields are really masked. Add a standing alert for sources that stop sending, because silent failure is the most common monitoring defect. A short validation step in every deployment pipeline catches broken rules before they reach production.

The workspace is the heart of log storage. A few early design choices decide your cost, your security boundaries, and how easily teams can work together.

What a workspace holds

A Log Analytics workspace is an Azure resource that stores log data in tables, offers a query engine, and defines settings for retention, access, and billing. Each table has a plan, a retention period, and a schema. Tools such as Application Insights, Microsoft Sentinel, and Microsoft Defender for Cloud can all write to workspaces, which is why a workspace often becomes a shared home for several kinds of data.

One workspace or many

Microsoft’s general advice is to start with as few workspaces as your needs allow, because fewer workspaces mean simpler queries, fewer alert rules, and often a lower cost through pooled commitment pricing. Add workspaces only when a real reason demands it.

Table plans

Tables in a workspace use one of three plans. The Analytics plan is the default and supports full queries, alerts, and all features, so it suits continuous monitoring and time-sensitive analysis. The Basic plan costs less to ingest and suits troubleshooting and incident response, with a smaller set of query features. The Auxiliary plan, also described as the lake plan, is meant for high-volume, verbose data that you keep cheaply for a long time and use for occasional searches or aggregated trends. Support for the cheaper plans varies by table for Azure-provided tables, while custom tables created through data collection rules can use any plan. You can change a table’s plan, but only about once a week per table, and moving to the Auxiliary plan stops alerts from working on that table.

Retention

New tables keep data in interactive retention for a default period, commonly thirty days for the main plan, and you can extend it. Beyond interactive retention, data can stay in a cheaper long-term state, which is not queried directly but can be searched or restored when needed. Total retention can run for years. Choose retention by table, because security logs may need years while verbose debug logs need days. Always verify the current limits and defaults on the official page, since they are updated regularly.

Access control

Workspaces support two models. In workspace-context access, a person who has rights to the workspace sees tables in it. In resource-context access, a person who has rights to an Azure resource can see only the logs of that resource, even without access to the workspace. The second model lets application owners see their own logs inside a shared central workspace. Table-level role assignments can further limit who sees sensitive tables.

Naming and organising workspaces

Names should reveal purpose, environment, and region at a glance, so that nobody has to open a workspace to learn what it holds. Place workspaces in resource groups that match their owners, tag them with cost centre and data classification, and document the retention and access decisions in a short design record. Good naming also makes policy and automation easier, since scripts can find every production workspace in a region with a simple filter instead of a hand-maintained list.

Collected data is useful only when someone can ask it questions. Azure Monitor offers several tools, each suited to a different habit of working.

Metrics explorer

Metrics explorer is the quickest way to chart numbers. You pick a resource, a metric, a time range, and a way to combine values, such as average, maximum, or sum. You can split a chart by dimension, such as status code, and overlay several metrics to see relationships. Charts can be pinned to dashboards or turned into alert rules directly. It is the right tool when the question is “what is the trend?”.

Log queries

Log Analytics provides a query screen where you write KQL. KQL reads like a pipeline: start with a table, filter the rows, group them, count, sort, and draw. A beginner can ask “how many failed sign-ins per hour yesterday” in a few lines. Queries can join tables, parse text, compute percentiles, and detect unusual patterns. You can save queries, share them in packs, and use them as the basis for alert rules and workbooks. Learning basic KQL is one of the best investments an operations team can make.

Workbooks

Workbooks are interactive reports that combine text, parameters, metric charts, log query results, and links. A parameter such as a subscription or time range lets a reader change the view without editing a query. Workbooks suit shared investigation guides and weekly reviews, and many insights ship prebuilt ones. Because they are Azure resources, they can be stored, versioned, and deployed like other infrastructure.

Dashboards and Grafana

Azure dashboards pin tiles from many services in one place and suit a wall display or a landing page for an operations team. Managed Grafana, an Azure service, offers the popular open-source dashboard tool with built-in connections to Azure Monitor and to Prometheus data, which suits teams already trained on Grafana. Power BI can also read Log Analytics data for business reporting.

flowchart LR
    DATA[Stored telemetry] --> ME[Metrics explorer]
    DATA --> LQ[Log queries]
    DATA --> WB[Workbooks]
    DATA --> DB[Dashboards]
    DATA --> GR[Managed Grafana]
    DATA --> PB[Power BI]
    ME --> Q1[Trends and comparisons]
    LQ --> Q2[Deep investigation]
    WB --> Q3[Guided reports]
    DB --> Q4[At-a-glance status]
    GR --> Q5[Cloud-native views]
    PB --> Q6[Business reporting]
        

Choosing the right tool

Summarising for speed and cost

Querying months of verbose data for a chart is slow and expensive. Summary rules run a query on a schedule and save the aggregated result in a smaller table, so dashboards and long-range reports read the small table. This pattern also lets you keep verbose raw data on a cheaper plan while giving people a fast summarised view.

Habits of effective investigators

Skilled operators follow a rhythm. They fix the time range first, because a wrong range wastes the whole investigation. They start with counts and groupings to see the shape of the problem before reading single records. They compare a bad period with a good period, often the same hour on the previous day. They write down each finding and the query behind it, so a colleague can repeat the steps. Saving good queries into shared packs turns one person’s skill into a team habit and speeds up every later incident.

Insights are curated experiences built on top of Azure Monitor data. They save weeks of design by giving you tested views, collection rules, and alerts for common workloads.

How insights work

An insight usually does four things. It enables the right collection, often by deploying a data collection rule. It stores data in your workspace so you own and can query it. It provides workbooks and views that answer common questions. It may offer recommended alert rules. Because they use standard building blocks, you can extend an insight by adding your own queries or alerts.

Monitoring Azure Monitor itself

A forgotten truth is that the monitoring platform needs monitoring. Workspace insights and usage views show which tables are growing, who sends the most data, how long queries take, and whether ingestion is delayed. Alerts on ingestion volume, on data collection errors, and on agent heartbeats protect against silent failure, where charts look calm only because data has stopped arriving.

Extending an insight without breaking it

Insights are maintained by Microsoft, so changes to the product can update them. When you want something different, copy the workbook and edit the copy rather than changing the original, and keep your own collection additions in separate rules. This way product updates still flow in while your custom views stay intact. Record which insights each team depends on, so a change announcement can be matched quickly to the people who use the affected views.

Infrastructure health is only half the story. Users care whether the application works, and Application Insights is the part of Azure Monitor that answers that.

What Application Insights gives you

Application Insights collects request rates, response times, failure rates, calls to dependencies such as databases and external services, exceptions, custom events, and browser data such as page load time. It draws an application map that shows how components connect and highlights the ones with failures or slow responses. It offers live metrics with very short delay for watching a deployment. It runs availability tests that probe your site from several places on the internet. Smart detection watches for unusual changes in failure rates or response times and warns without a handwritten rule.

Workspace-based by design

Application Insights resources store their data in a Log Analytics workspace. This means application telemetry can be queried together with infrastructure logs in a single query, shares the retention and access settings of the workspace, and can be billed in the same place. Older classic resources that stored data separately have been migrated to workspace-based resources, so any remaining classic design should be treated as legacy.

OpenTelemetry

OpenTelemetry is an open standard for creating and sending traces, metrics, and logs. Azure Monitor supports it through distributions of the libraries for common languages and through collectors. The benefit is portability: if your code produces standard telemetry, you are less tied to one product and can send data to more than one place. The cost is that you must understand sampling, naming, and attribute choices, since those decide what you can see and what you pay.

sequenceDiagram
    participant U as User browser
    participant W as Web app
    participant A as API service
    participant D as Database
    participant P as Payment service
    U->>W: Request with trace identifier
    W->>A: Call with same identifier
    A->>D: Query
    D-->>A: Result
    A->>P: Charge
    P-->>A: Slow response
    A-->>W: Result
    W-->>U: Page
    Note over U,P: All hops share one trace so the slow payment call is easy to see
        

Sampling: seeing enough without paying for everything

Recording every request of a busy site is costly. Sampling keeps a representative share of telemetry while preserving statistical accuracy of counts. Good sampling keeps all the parts of one trace together and may keep all failures while thinning out routine successes. Set sampling deliberately, because too little hides rare problems and too much floods the bill.

Synthetic tests and real-user views

Availability tests act like a patient visitor who checks your site every few minutes from different locations, which finds outages before customers do. Browser telemetry shows what real users experience, including slow pages for particular countries or devices. Use both, since a synthetic test shows that a path works and real-user data shows how it feels.

Connecting application health to business meaning

Technical metrics gain power when linked to business events. Custom events such as orders placed, payments failed, or documents uploaded show whether the application is doing its job, not merely running. A sudden drop in orders with no rise in errors may reveal a broken button rather than a broken server. Agree with product owners on a few such business signals, chart them next to technical ones, and consider alerts on them, since they describe harm to customers more directly than processor graphs.

Alerts turn watching into acting. A good alert is rare, meaningful, and followed by a clear action. A bad alert is noise that trains people to ignore the pager.

The parts of an alert

An alert rule combines the resources to watch, the signal to evaluate, and the condition that counts as a problem. When the condition is met, an alert fires, its action group runs, and its state is updated. Each monitored resource is evaluated separately, so one rule can fire separate alerts for several resources.

Alert types

Alert states

An alert has an alert condition, which is fired when triggered and resolved when the underlying problem clears, and a user response, which is new, acknowledged, or closed, set by a person. These are separate: a condition can resolve while the alert is still unacknowledged. Fired alerts are stored for a limited period, currently thirty days, so use exports or incident tools if you need longer history.

Stateless and stateful alerts

A stateless alert fires every time the condition is evaluated as true, which may repeat notifications while the problem persists. All activity log alerts are stateless. A stateful alert fires once, stays quiet while the problem continues, and sends a resolved notification when the problem ends. Metric alerts resolve after the condition is not met for several consecutive checks, and log search alerts resolve after a window related to their frequency. Stateful behaviour reduces noise and is the sensible default for many rules.

Severity

Each alert rule has a severity from critical to verbose, and you should define what each level means for your organisation. A common agreement is that the most severe level pages a person immediately at any hour, the next level creates a ticket for working hours, and lower levels appear only on dashboards. Without such an agreement, everything becomes urgent and nothing is.

Dynamic thresholds

Choosing a static number such as eighty percent processor use is easy but crude, because normal load differs by hour and day. Dynamic thresholds learn the pattern of a metric and alert when it departs from its own normal. They help for metrics with daily or weekly cycles, and they reduce the effort of tuning many rules. They need a little history before they become accurate.

stateDiagram-v2
    [*] --> Watching
    Watching --> Fired: Condition met
    Fired --> Fired: Stateless repeats or stateful stays quiet
    Fired --> Resolved: Condition clears
    Resolved --> Watching
    Fired --> Acknowledged: Person takes ownership
    Acknowledged --> Closed: Fix confirmed
        

Designing alerts people trust

Testing and tuning alerts

An alert that has never fired is an untested assumption. Test new rules by creating a safe, temporary condition and checking that the alert fires, that the right people receive the message, and that the resolved message arrives. After launch, review each alert’s history: how often it fired, how often someone acted, and how long it took to resolve. Alerts that fire often without action should be tuned or removed, and alerts that fired too late should be moved to a faster signal or a shorter window.

Detecting a problem is half the work. The other half is getting the right message to the right place, quietly handling known events, and acting automatically when a person is not needed.

Action groups

An action group is a reusable collection of notifications and automated responses. Notifications include email, text message, voice call, and mobile push. Automated responses include Automation runbooks, Azure Functions, Logic Apps, webhooks and secure webhooks, connections to IT service management tools, and event hubs for streaming alerts into other systems. Because many alert rules can share one group, you can change who is notified in one place. Create groups by team and severity rather than by alert, for example one for the platform team’s urgent pages and another for the application team’s tickets.

Alert processing rules

Alert processing rules change what happens after an alert fires, without editing the alert rules themselves. They can suppress notifications during planned maintenance, add an action group to every alert from a certain subscription, or limit rules to certain times, such as sending only to the on-call group outside office hours. They apply to fired alerts that match filters on scope, severity, or other fields, and they can run on a one-time or recurring schedule. This separation keeps alert logic clean and operational decisions flexible.

flowchart LR
    SIG[Metric or log signal] --> RULE[Alert rule]
    RULE -->|Condition met| FIRED[Alert fires]
    FIRED --> APR{Processing rule matches}
    APR -->|Maintenance window| SUP[Notifications suppressed]
    APR -->|Needs extra recipients| ADD[Extra action group added]
    APR -->|No match| AG[Action group]
    ADD --> AG
    AG --> N1[Email and text]
    AG --> N2[Ticket in service desk]
    AG --> N3[Runbook or function]
    AG --> N4[Event hub for other tools]
        

Autoscale

Autoscale is a related feature that acts on metrics without involving a person. It adds or removes instances of supported services, such as virtual machine scale sets and app service plans, according to rules based on metrics, schedules, or both. A rule might add two instances when average processor use stays high for ten minutes and remove one when it stays low for twenty. Use a wider gap between the scale-out and scale-in thresholds and a cooling-off period so the system does not flap between sizes. Autoscale saves money at quiet times and protects users at busy times, but it does not repair slow code, so monitor the application as well.

Alerting at scale

In a large estate you cannot hand-build alert rules for every resource. Several approaches help. One metric alert rule can monitor many resources of the same type in the same region. Log search rules can split results by a dimension such as computer name so that each resource has its own alert. Azure Policy can deploy alert rules to every new resource of a given type. The Azure Monitor baseline alerts collection offers a tested starting set for common services. Recommended alert rules are available out of the box for virtual machines, Kubernetes resources, and Log Analytics workspaces. The trade-off is maintenance: a large rule set needs owners, naming, and review like any other code.

Writing messages people can act on

The text of a notification decides how fast someone responds. Put the affected service, the symptom, the severity, and the time at the top. Add a short sentence on likely impact and a link to the runbook and to the first query or chart to open. Avoid vague titles such as “alert fired” and avoid pasting long raw data. A person woken at night should understand within ten seconds whether to get out of bed, what is wrong, and where to look first.

Monitoring data is a map of your whole system and sometimes holds personal or secret information. It deserves the same care as production data.

Who may do what

Azure role-based access control governs the platform. Built-in roles include Monitoring Reader, who can view data and settings, Monitoring Contributor, who can also change settings and create alerts, and Log Analytics Reader and Contributor for workspaces. Creating an alert needs read permission on the target resource, write permission on the group where the rule lives, and read permission on the action group. Give application teams access to their own resource logs through resource-context access rather than broad workspace roles, and limit sensitive tables with table-level permissions.

Protecting the network path

By default, data reaches Azure Monitor over public endpoints with encryption in transit. For stricter environments, Azure Monitor Private Link Scope lets you connect to monitoring services over private endpoints in your virtual network, and settings allow you to block ingestion and queries from the public internet. Private connectivity needs careful design of name resolution and of which workspaces and rules belong to the scope, so test with a pilot before enforcing it broadly.

Encryption and keys

Data is encrypted at rest by default. Some organisations use customer-managed keys for workspaces to control the key lifecycle, which requires a dedicated cluster arrangement and adds responsibility for key availability. Decide this with your security team, since losing access to a key can make data unreadable.

Keeping secrets and personal data out

Developers sometimes write passwords, tokens, or personal details into log messages by accident. Once stored, such data can be read by anyone with access to the table. Reduce the risk by agreeing logging rules with developers, by using transformations to remove or mask sensitive fields before storage, by limiting who may query sensitive tables, and by setting short retention for tables that may contain personal data. If data must be removed, purge features exist but are slow and meant for exceptional cases, so prevention is better.

Governing the platform with policy

Audit and change control

Treat dashboards, alert rules, data collection rules, and workspace settings as code. Store them as templates in version control, review changes, and deploy through pipelines. The activity log records who changed monitoring settings, and an alert on changes to action groups or on deletions of diagnostic settings protects against both mistakes and tampering. An attacker who wants to hide will try to turn off logging first.

Separating duties

Different people should hold different powers. Platform engineers may manage workspaces and policies, application teams may write queries and build their own alerts for their own resources, and security staff may read audit tables but not change collection. Separate the right to see data from the right to change settings, and review assignments regularly. Use groups rather than named individuals, and give temporary elevated access through just-in-time approval instead of permanent broad roles.

Handling regulated and sensitive environments

Organisations in finance, health, and government often need extra care. Keep monitoring data in approved regions, apply retention that matches legal duties, restrict the tables that may contain personal data, and record who queried them. Document the data flow from source to workspace for auditors, including transformations that mask data. Test that deletion and export requests can be honoured. Compliance is easier when these controls are built into the standard platform than when each team must remember them separately.

Monitoring bills surprise many teams, because they grow silently with every new log source. Understanding the cost drivers lets you spend on what matters.

The main cost drivers

Pricing models

Workspaces can be billed pay-as-you-go by the gigabyte ingested, or through commitment tiers, where you commit to a daily volume at a lower unit price. If your daily volume is steady and large, a commitment tier usually costs less, and sharing a workspace across teams helps reach the tier. Prices change and differ by region, so use the official pricing page and calculator for real numbers.

A cost-control routine

Showing teams their own spend

Cost drops when owners can see it. Use tags, resource-level usage queries, and shared reports so each team can see how much data its applications send. A short monthly review with the biggest senders often finds a forgotten debug setting or a chatty library within minutes.

Trade-offs between cost and visibility

Cost control is not about collecting as little as possible. It is about collecting what protects the business at the lowest sensible price. Security audit data, availability signals, and application errors deserve full-featured storage. Chatty debug output, repetitive success records, and duplicated data do not. Ask for each source what decision it supports and how often that decision is made. When the answer is “none” or “never”, stop paying for it. When the answer is “rarely, but urgently”, use a cheaper plan with search ability.

If your monitoring fails during an outage, you fly blind exactly when you need to see. Plan the resilience and speed of Azure Monitor like any other critical system.

Availability and redundancy

Azure Monitor services are built for high availability inside a region, and many components use availability zones in regions that support them. Workspaces live in one region, so a regional disaster affects data in that region. Options exist to improve this, including workspace replication to a second region and exporting data continuously to storage or event hubs, but capabilities, supported regions, and extra costs change over time, so check the current documentation and choose according to how critical the data is.

Designing for the day your region fails

Think about what you need during a regional outage. For alerting, consider rules that watch from outside the affected region, such as availability tests from several locations, so that a total failure of one region is still noticed. For investigation, keep a copy of critical logs elsewhere or send security data to a second destination. For notification, make sure the action groups do not depend on systems hosted in the region that is failing.

flowchart TD
    subgraph R1[Primary region]
        APP1[Application]
        WS1[Primary workspace]
    end
    subgraph R2[Second region]
        WS2[Replica or exported copy]
    end
    EXT[External availability tests] --> APP1
    EXT --> AL[Alert rules outside the failing region]
    APP1 --> WS1
    WS1 -->|Replication or export| WS2
    AL --> AG[Action groups not tied to one region]
        

Ingestion delay

Data does not appear instantly. Metrics are usually available within a minute or two, while logs may take several minutes, depending on the source, the pipeline, and load. Alert designs must allow for this latency. A log alert that looks at the last five minutes may miss data that arrives late, so use windows that include a margin, and understand the delay of each table you rely on.

Query performance

Queries are fast when they filter early by time and by a few key columns, select only needed columns, and avoid scanning very wide time ranges. Slow queries are usually a sign of too much data scanned. Summary tables, shorter ranges, and well-chosen table plans fix most problems. Very expensive or very frequent queries from dashboards or tools can also trigger throttling, so share results through summary tables instead of running the same heavy query for every viewer.

Limits and quotas

Azure Monitor has documented limits on items such as the number of alert rules per subscription, action group sizes, the rate of notifications per phone number or email address, ingestion rates per workspace, the size of query results, and API request rates. These numbers are updated from time to time, so read the current service limits page before designing a very large environment, and request increases early if you need them. Rate limits on notifications explain why a storm of alerts may arrive late or in summary form.

Testing your monitoring like a feature

Run game days where you deliberately break something in a test environment and watch whether monitoring notices. Measure how long detection took, whether the first alert pointed at the real cause, and whether the on-call person had the dashboards and queries they needed. Treat gaps as defects and fix them with the same care as product bugs. Teams that practise find that real incidents feel calmer, because the tools and the habits have already been proven.

Capacity thinking for ingestion spikes

Traffic and logging are not steady. A release, an attack, or an error loop can multiply log volume in minutes. Plan for spikes by sampling application telemetry, by alerting on unusual ingestion growth, and by keeping verbose levels switchable without redeploying code. Where possible, decouple producers from the platform with buffers such as event hubs, so a burst is smoothed instead of rejected. Review limits before big events so that nobody learns about a rate limit during the event itself.

Sooner or later you will ask why data is missing, why an alert did not fire, or why a bill jumped. A few systematic checks solve most cases.

Why is there no data?

Work from the source towards the screen. Is the resource or machine running? Is the agent installed and healthy, with a recent heartbeat? Is a data collection rule associated with the machine, and does the rule list the data you expect? Is the destination workspace the one you are querying? Is a transformation filtering the data away? Is the network path open, or is a private link scope blocking the route? Did a daily cap stop ingestion? Is the time range of the query correct? Checking in this order finds the break quickly.

flowchart TD
    A[No data in the workspace] --> B{Source running}
    B -->|No| B1[Fix the resource]
    B -->|Yes| C{Agent or setting present}
    C -->|No| C1[Install agent or add diagnostic setting]
    C -->|Yes| D{Rule associated and correct}
    D -->|No| D1[Associate or fix the rule]
    D -->|Yes| E{Network and private link allow traffic}
    E -->|No| E1[Fix endpoints and name resolution]
    E -->|Yes| F{Filter, cap, or wrong workspace}
    F -->|Yes| F1[Adjust transformation, cap, or query scope]
    F -->|No| G[Open a support case with evidence]
        

Why did my alert not fire?

Why did the bill jump?

Start with ingestion by table and by source for the days before and after the jump. Common causes are a new deployment with verbose logging turned on, an application stuck in an error loop that writes the same message thousands of times, a new diagnostic setting that sends high-volume categories, a monitoring solution enabled on all machines, or a change of table plan. Fix the cause, and consider an alert on unusual ingestion growth.

Why is a query slow or failing?

Narrow the time range, filter early, select fewer columns, and avoid heavy operations over huge data. If the workspace is under heavy load, spread queries or use summary tables. If results are cut off, you have probably reached a result size limit and should aggregate more.

Why do agents misbehave?

Check the agent’s version, the operating system support list, the identity used to authenticate, connectivity to the required endpoints, and local resources such as disk space. Many problems come from outdated agents, missing managed identities, or firewalls that block the data collection endpoints. Azure also offers troubleshooting tools and health views for agents, which show the last heartbeat and error messages.

Building a troubleshooting runbook

Write down the checks in this chapter as a short runbook for your own environment, with the exact names of your workspaces, rules, and endpoints filled in. Include who owns each part, where to find agent health, which queries show ingestion by table, and how to contact support. A runbook turns a stressful guess into a calm checklist, and it keeps knowledge in the team when people move on. Review it after every incident and remove steps that nobody used.

Azure Monitor has changed a great deal in recent years. Old components have been retired, and designs that mention them need updating.

From the Log Analytics agent to the Azure Monitor Agent

The older Log Analytics agent, also known as the Microsoft Monitoring Agent or OMS agent, was retired on 31 August 2024. Microsoft announced that its cloud ingestion would be shut down, that a test pause of uploads would take place on 26 January 2026, and that after 2 March 2026 data upload could stop at any time without further notice. Any machine still using it should be treated as urgent. The replacement is the Azure Monitor Agent, which uses data collection rules for flexible, reusable configuration and supports filtering and multiple destinations. Migration tools include a helper workbook that finds machines still using legacy agents, and a configuration generator that turns workspace-based settings into rules. One exception: an agent connected only to an on-premises System Center Operations Manager installation is outside this retirement.

From classic to workspace-based Application Insights

Classic Application Insights resources that kept their own storage have been replaced by resources that write to Log Analytics workspaces. If any old design remains, move it so that application and infrastructure data can be queried together under common retention and access controls. Related interfaces have also changed: API keys for reading Application Insights data have been scheduled for retirement in favour of identity-based access with Microsoft Entra, so move scripts and tools to token-based access.

Older alerts and APIs

Classic alert rules have been replaced by the current alert types, and the preview or beta versions of some Log Analytics interfaces have been retired, such as a beta query interface scheduled for retirement on 31 March 2026. Teams should check Azure Service Health and Advisor for retirement notices on their subscriptions and keep an inventory of the interfaces and tools they use, since an old script that calls a retired interface fails without warning.

Keeping up with change

Azure Monitor evolves quickly, so assign a person or a small group to follow service announcements, the documentation change log, and notices in Service Health and Advisor. Hold a short review each quarter to ask which features became generally available, which previews you rely on, and which retirements are approaching. Treat preview features carefully in production: they may change, may lack full support, and may be priced differently once released. A little steady attention prevents rushed migrations later.

Patterns are proven answers to repeated problems. This chapter gathers the ones that appear in almost every healthy monitoring estate, with the traps to avoid and a checklist for reviews.

Helpful patterns

flowchart LR
    SUB1[Production subscriptions] --> POL[Policy deploys settings and rules]
    SUB2[Test subscriptions] --> POL
    POL --> WSP[Production workspace]
    POL --> WST[Test workspace]
    WSP --> RC[Resource-context access for teams]
    WSP --> AL[Shared alert rules and processing rules]
    WSP --> SUMM[Summary tables for dashboards]
    AL --> AGS[Action groups by team and severity]
        

More anti-patterns

Common mistakes and their fixes

A pre-launch checklist

Choosing patterns by team size

A small team with a handful of applications can run happily with one workspace, a few standard rules, Application Insights for each service, and ten well-tuned alerts. A large organisation needs a platform team, policy-driven collection, shared alert libraries, cost reports by owner, and a clear process for exceptions. Do not copy the structure of a huge company if you are small, because complexity is also a cost. Start simple, watch where the pain appears, and add structure only when the pain justifies it.

Every platform has strengths and costs. Here is an honest look at what Azure Monitor gives you and what it asks in return.

When Azure Monitor is the right fit

It is the natural choice when most of your workloads run on Azure, when you want monitoring to follow resources automatically, and when you value integration with Azure policy and identity. It is also a good fit for hybrid estates through Arc. Teams with an existing third-party platform may keep it for some data and still use Azure Monitor for platform metrics, activity logs, and autoscale, since these exist anyway.

When to think twice

If your applications run mostly outside Azure and your team is deeply invested in another observability tool, a full move may bring little. If you must retain very large volumes of raw logs for many years at the lowest cost, consider cheap plans, export to storage, or a dedicated data lake, and use Azure Monitor for the recent, queryable window.

Comparing with other approaches

Some organisations assemble open-source tools for metrics, logs, and traces and run them on their own servers. This gives control and avoids per-gigabyte fees, but it adds work: capacity planning, upgrades, security, and availability all become your job. Others buy a third-party observability service that spans clouds, trading higher fees for convenience. Azure Monitor sits between them: managed, tightly integrated with Azure, and open enough to accept standard formats. The right choice depends on skills, scale, and how much of your estate lives on Azure.

These short stories show how the concepts fit together in situations you may recognise.

flowchart TD
    N[Need arises] --> T{Type of need}
    T -->|Know when users are hurt| A[Symptom alerts and availability tests]
    T -->|Explain a failure| B[Logs, traces, and workbooks]
    T -->|Control cost| C[Filtering, table plans, summary tables]
    T -->|Standardise at scale| D[Policy, templates, shared workspaces]
    T -->|Survive an outage| E[External tests, replicated data, independent notifications]
    A --> G[Review and improve regularly]
    B --> G
    C --> G
    D --> G
    E --> G
        

What these stories have in common

In every scenario, success came from decisions made early and habits kept up over time. Teams that defined what mattered caught problems sooner. Teams that standardised through policy and templates gained coverage without extra effort. Teams that watched cost and reviewed alerts kept the platform affordable and trusted. Teams that rehearsed failures found gaps while they were still cheap to fix. None of this required exotic features, only clear ownership, sensible defaults, and regular review.

Short, direct answers to questions that beginners and busy teams often ask.

No. It is built into Azure, and platform metrics and the activity log are collected automatically. You configure additional collection, such as resource logs, agents, and application instrumentation.

Azure Monitor is the whole platform. Log Analytics is the part that stores and queries log and trace data in workspaces, and it is also the name of the tool used to run queries.

A Log Analytics workspace stores logs and traces and is queried with KQL. An Azure Monitor workspace stores Prometheus and OpenTelemetry style metrics and is queried with PromQL. They are separate resources.

Platform metrics are automatic, but resource logs need a diagnostic setting that sends them to a destination, and machine data needs the agent and a data collection rule.

The Azure Monitor Agent, configured with data collection rules. The older agent was retired and its data upload service is being shut down, so migrate any remaining machines.

They are reusable recipes that say what data to collect, how to transform it, and where to send it. One rule can apply to many machines.

As few as your needs allow. Split only for data residency, strict separation, billing needs, or regional egress cost, and use resource-context access for team separation inside a shared workspace.

They are settings that trade features for cost. Analytics supports full features, Basic is cheaper for troubleshooting data, and Auxiliary or lake is the cheapest for high-volume data kept long term. Alerts do not work on Auxiliary tables.

A stateless alert fires every time the condition is met. A stateful alert fires once, stays quiet while the problem continues, and sends a resolved notification when the problem clears.

Yes. Use an alert processing rule with a schedule to suppress notifications for chosen resources during the window, while alerts still record in history.

Fired alerts are stored for a limited time, currently thirty days. Export them or use an incident tool if you need a longer history.

Find the largest tables, remove data nobody uses, filter and transform at collection, move verbose data to cheaper plans, shorten retention, apply sampling to application telemetry, and consider commitment tiers for steady large volumes.

Rarely for important data. A cap stops ingestion when reached, which creates blind spots. Prefer filtering, plan choice, and alerts on unusual growth.

Yes. Use Azure Monitor Private Link Scope with private endpoints, and block public ingestion and query access if your rules require it. Plan name resolution carefully and pilot first.

The most common reason is ingestion delay: the data arrived after the alert’s window was evaluated. Use a window with a margin, and check other causes such as suppression rules or an unresolved stateful alert.

Yes. Machines in other clouds or on-premises can be connected through Azure Arc and monitored with the Azure Monitor Agent, and applications anywhere can send telemetry using OpenTelemetry-based libraries.

Azure Monitor is the cockpit for your cloud. It collects metrics, logs, traces, and events from applications, machines, containers, and Azure services, stores them in purpose-built stores, and gives you tools to explore the data and alerts to act on it. A good design starts with questions and actions, not with switches. It collects deliberately through agents, data collection rules, diagnostic settings, and instrumentation, filters at the door, uses table plans to match cost to purpose, and builds alerts that people trust because they are rare, meaningful, and routed well. It treats telemetry as sensitive data, governs the platform with policy and code, plans for its own failure, and keeps up with Microsoft’s retirements such as the move to the Azure Monitor Agent. Regular review of volume, alerts, and ownership keeps it useful and affordable.