Log Analytics in Azure: The Central Diary That Explains What Your Systems Are Doing
A beginner-friendly guide to Log Analytics in Azure Monitor. Learn how logs are collected into workspaces, how the Kusto Query Language finds answers, how alerts and workbooks use the data, and how to control access, retention, and cost.
Imagine a huge hotel with hundreds of rooms, a kitchen, a parking garage, a swimming pool, and a front desk. Every part keeps its own little notebook: the kitchen notes what was cooked, the garage notes which cars came in, the front desk notes who checked in. When a guest complains about a missing wallet, you would have to run from notebook to notebook. Now imagine every notebook copying its entries, as they happen, into one giant searchable diary, and a smart assistant who can answer questions like “Who entered the pool area between ten and eleven last night?” in seconds. That giant diary and its smart assistant are what Log Analytics is for your Azure systems.
1What Is Log Analytics?
Start with a plain definition, then add the details one layer at a time.
Log Analytics is the part of Azure Monitor that collects, stores, and lets you search log data. A log is a record that a system writes about something that happened: a user signed in, a request failed, a disk filled up, a setting was changed. Almost every Azure service, virtual machine, and application produces logs, but on their own they are scattered and hard to use.
Log Analytics gathers these records into a central place called a workspace. Once the data is there, you can search it with a powerful query language, draw charts from it, create alerts when something goes wrong, and share dashboards with your team.
Two names often cause confusion. Azure Monitor is the overall family of monitoring tools in Azure. Log Analytics is the tool, and the workspace, used to store and query log data inside that family. When people say “Azure Monitor Logs”, they usually mean the same logging feature that Log Analytics provides.
If every Azure service is a department in a big company, each writing its own notes, then a Log Analytics workspace is the company’s central archive with an excellent search engine. You no longer visit each department. You ask the archive.
2Why Do We Need Centralized Logs?
A tool makes sense when you know the pain it removes.
Modern systems are made of many parts: web apps, databases, networks, containers, and security tools. When something breaks, the cause may be in any of them, and often in the way they interact. Looking at each part separately is slow and misses the big picture.
Scattered evidence
Logs sit in many places and formats, so finding the cause of a problem means checking many tools one by one.
Logs disappear
Many services keep logs only for a short time or only on the machine itself, which may be deleted or fail.
Slow detection
Without alerts, teams learn about problems from angry customers instead of from their own monitoring.
Hard to connect events
A failed login in one system and a strange data download in another may be related, but you only see the link when both logs are together.
Compliance and audits
Many regulations require you to keep and review records of who did what and when.
Log Analytics solves these problems by collecting data into one place, keeping it for as long as you choose, and giving you fast ways to search, alert, and report on it.
3Logs, Metrics, and Traces: Know the Difference
Monitoring uses several kinds of data. Each answers a different kind of question.
| Type | What it is | Good for | Example |
|---|---|---|---|
| Metrics | Numbers measured at regular intervals | Fast health checks and trend charts | CPU percentage every minute |
| Logs | Detailed records of individual events, with text and many fields | Investigating what happened and why | A record saying a request to the payment page failed with a certain error |
| Traces | A record of the path one request took through many services | Finding which service made a request slow | A checkout request moving through web, API, and database |
Metrics are the dashboard gauges of a car: speed and fuel level at a glance. Logs are the detailed trip diary: where you stopped, what warning lights flashed, what time. Traces are the route map of one specific journey from start to finish.
Azure Monitor keeps metrics and logs in different stores, but they work together. Log Analytics is where the detailed, searchable log data lives. Metrics are stored separately for fast, near real-time charts, although some metrics can also be sent to a workspace.
4The Core Building Blocks
A handful of words explain almost everything in Log Analytics.
Log Analytics workspace
The Azure resource that stores your log data. It has its own region, access rules, retention settings, and cost settings.
Table
A named collection of similar log records inside a workspace, with columns, like a spreadsheet tab. Different data types go to different tables.
Record or row
One single log entry, such as one sign-in event.
Column
One property of a record, such as the time, the user name, or the result code.
Kusto Query Language (KQL)
The query language used to search and analyze log data. It reads as a pipeline of steps joined by a vertical bar.
Data source
Anything that sends logs: an Azure service, a virtual machine, an application, or a custom program.
Data collection rule (DCR)
A rule that defines what data to collect, how to filter or change it on the way, and where to send it.
Retention
How long data is kept in the workspace before it is removed or moved to cheaper storage.
flowchart TD
SRC["Data Sources: Azure services, VMs, apps"] --> DCR["Data Collection Rules and Diagnostic Settings"]
DCR --> WS["Log Analytics Workspace"]
WS --> T1["Table: sign-in logs"]
WS --> T2["Table: VM performance"]
WS --> T3["Table: app requests"]
WS --> T4["Table: activity logs"]
T1 --> KQL["Queries in KQL"]
T2 --> KQL
T3 --> KQL
T4 --> KQL
KQL --> OUT["Alerts, Workbooks, Dashboards"]
The workspace is a large library. Tables are the different bookshelves: one for sign-in records, one for performance records, one for application records. Each row is one page, each column is a field on the page, and KQL is the librarian who can find and combine pages from any shelf.
5How Data Gets Into a Workspace
Several routes carry data from sources to your workspace.
Diagnostic settings
For Azure resources such as storage accounts, databases, and web apps. You switch on categories of logs and metrics and choose a workspace as a destination.
Azure Monitor Agent
Software installed on virtual machines and servers, in Azure or elsewhere, that collects operating system logs, performance counters, and custom files.
Activity log
The record of management actions on Azure resources, such as who created or deleted something. It can be sent to a workspace for longer storage and queries.
Application Insights
Application monitoring that stores requests, exceptions, dependencies, and traces. Modern setups store this data in a Log Analytics workspace.
Container and Kubernetes insights
Collect logs and performance data from clusters and containers.
Logs ingestion API
Lets your own programs and scripts send custom data into chosen tables through data collection rules.
Other Microsoft services
Services such as Microsoft Defender for Cloud and Microsoft Sentinel read from and write to workspaces.
flowchart LR
A["Azure resources"] -->|Diagnostic settings| W["Log Analytics Workspace"]
B["Virtual machines and servers"] -->|Azure Monitor Agent and DCR| W
C["Applications"] -->|Application Insights| W
D["Containers and Kubernetes"] -->|Insights| W
E["Custom programs"] -->|Logs ingestion API| W
F["Azure Activity Log"] --> W
Assuming that logs flow automatically. For most Azure resources, nothing is sent until you create a diagnostic setting. Many teams discover during an incident that the logs they need were never turned on.
Think of pipes bringing water into a reservoir. Each source has its own pipe, such as diagnostic settings or an agent. But a pipe that was never installed brings no water, no matter how much you wish for it.
6Agents and Data Collection Rules
Virtual machines need a small helper to send their logs.
Azure resources like web apps can send logs by themselves through diagnostic settings, but a virtual machine’s operating system does not automatically send its internal logs. A small program called the Azure Monitor Agent runs on the machine and sends the data you choose. It replaces older agents that were used in the past, so older guides may mention different names.
The agent does not decide on its own what to collect. A data collection rule tells it. A rule defines the data sources, such as certain event logs, system logs, or performance counters. It can filter out unwanted records and transform data before it is stored. It also names the destination workspace and table. The same rule can be attached to many machines, which makes large environments easy to manage.
Create a workspace
Choose a region and a name, ideally in the same region as most of the data sources.
Create a data collection rule
Select what to collect, such as security events and performance counters, and select the workspace as the destination.
Associate the rule with machines
Attach the rule to chosen virtual machines or groups of machines. The agent is installed automatically in many cases.
Data starts to flow
The agent collects the data on schedule and sends it securely to the workspace, where it appears in the matching tables.
Review and refine
Check what arrives, remove noisy data that you never use, and add anything that is missing.
sequenceDiagram
participant VM as Virtual Machine with Agent
participant DCR as Data Collection Rule
participant WS as Log Analytics Workspace
VM->>DCR: Which data should I collect and where do I send it?
DCR-->>VM: Collect these logs and counters, send to this workspace
VM->>WS: Send collected records securely
WS-->>VM: Accepted
WS->>WS: Store records in the matching tables
The agent is a courier. The data collection rule is the delivery instruction sheet: which parcels to pick up, whether to remove anything unwanted first, and which warehouse to deliver to. Change the sheet and every courier following it changes behavior.
7Tables and Table Plans
Not all logs deserve the same price or the same features.
Each type of data lands in its own table. Some tables are built in, such as those for virtual machine performance, security events, or Azure activity. Others are created by services you enable, and you can create custom tables for your own data. Tables have a plan that controls cost and features, and the names and details of plans evolve, so check the current documentation.
| Plan | Best for | Query abilities | Cost idea |
|---|---|---|---|
| Analytics | Data you query often, alert on, and use for monitoring and security | Full query power and alerts | Higher ingestion price, includes interactive querying |
| Basic | High-volume, lower-value data that you only occasionally search | Limited, simpler queries, with a charge for queries | Lower ingestion price |
| Auxiliary | Very high-volume data kept for compliance or rare investigation | Limited, slower queries | Very low ingestion price, with query charges |
Match value to cost
Send the logs that drive alerts and daily troubleshooting to the full-featured plan, and cheaper noisy data to lower-cost plans.
Plans can differ per table
One workspace can hold tables with different plans, so you tune cost table by table.
Check feature limits
Cheaper plans trade away some features, such as alerting on the table, so check what you need before moving data.
Plans are like storage choices in a company. Documents you use daily stay on your desk, where they are instantly reachable but take valuable space. Rarely needed boxes go to a cheaper basement, where fetching takes longer and may cost a small fee.
8Kusto Query Language: Asking Questions of Your Logs
KQL is the heart of Log Analytics, and it is easier than it looks.
KQL is a read-only language designed for searching large amounts of log data quickly. A query starts with a table name and then passes the rows through a series of steps, each separated by a vertical bar. Each step filters, reshapes, or summarizes the rows it receives from the step before it. Reading a query from top to bottom tells you the story of how the answer is built.
The most common steps
where
Keeps only the rows that match a condition, such as errors from the last hour.
project
Chooses which columns to keep, or creates and renames columns.
summarize
Groups rows and calculates counts, averages, totals, and other statistics.
extend
Adds a new calculated column based on existing ones.
sort and top
Orders rows, or takes the first few after sorting, such as the ten slowest requests.
join and union
Combines rows from different tables, either by matching values or by stacking them together.
render
Draws the result as a chart, such as a line or bar chart.
parse and extract
Pull useful pieces out of long text fields, turning free text into structured columns.
Pick the table
Begin with the table that holds the kind of events you care about, such as sign-ins or web requests.
Narrow by time first
Restrict to the period you care about. This is the single best way to make queries fast and cheap.
Filter the rows
Keep only matching records, such as failures, a certain user, or a certain resource.
Summarize
Count or average the rows, grouped by something meaningful such as time bucket, server, or error code.
Present the result
Sort the result and show it as a table or chart, so people can understand it at a glance.
flowchart LR
T["Start with a table: all web requests"] --> W["where: last 24 hours and failed"]
W --> S["summarize: count by hour and page"]
S --> O["sort: highest count first"]
O --> R["render: line chart or table"]
A KQL query is like an assembly line of sieves and scales. Start with a huge bucket of sand, pass it through a coarse sieve to remove stones, then a fine sieve to keep only gold-colored grains, then weigh each pile, then line the piles up from heaviest to lightest.
Log Analytics has built-in example queries and a query editor with suggestions. Start from an example close to your question and change it step by step. Copilot features may also help write queries, depending on availability.
9Writing Fast and Cheap Queries
A few habits make queries return in seconds instead of minutes.
Always set a time range
Use the time picker or a time filter at the start. Searching all history when you need one hour is wasteful.
Filter early
Put the most selective filters near the top so later steps handle fewer rows.
Keep only needed columns
Use project to drop wide columns you do not need, which reduces the amount of data moved around.
Search a specific table
Avoid searching across every table at once unless you truly do not know where the data is.
Prefer exact matches over wide text searches
Comparing a column to a value is faster than scanning large text fields for a phrase.
Limit results while exploring
Use a small sample first, and then widen the query when it looks right.
Problem
Running queries over very long time ranges and across all tables, with filters only at the end.
Why It’s Harmful
Queries become slow, may time out, put heavy load on the workspace, and on some plans increase query charges.
Correct Approach
Narrow the time range first, choose the right table, filter early, and widen only when needed.
Searching with no time range is like asking a librarian to look through every newspaper ever printed to find yesterday’s weather. Telling them the exact date saves everyone time.
10Using the Logs: Investigations in Practice
See how queries answer everyday questions without needing any special tools.
Why did the website slow down at three in the afternoon?
Query the request table for that hour, group by page and response time, and compare with the dependency table to see if a database call became slow at the same moment.
Who deleted that resource group?
Search the activity log table for delete operations on that resource group, and read the caller identity, the time, and the origin address.
Which virtual machines are almost out of disk space?
Query performance data for free disk space, take the latest value per machine, and list those below a threshold, sorted from most critical.
Is someone trying to guess passwords?
Count failed sign-ins per account and per origin address over the last hour, and flag any with an unusually high count.
Which errors are new since the last release?
Compare error counts per type before and after the deployment time, and list types that did not exist before.
Start with the symptom
Begin with when it happened and what the user saw, then narrow down.
Follow one request
Use an operation or correlation identifier to follow one request across services and tables.
Compare with a normal period
Always compare a bad hour with a good hour, to see what really changed.
Save useful queries
Store good queries in query packs or share them, so teammates do not rewrite them in a crisis.
Investigating with logs is detective work. You start with the time of the crime, gather witness statements from each system, line them up on one timeline, and look for the moment when things changed from normal.
11Alerts: Letting the Logs Tap You on the Shoulder
Nobody can stare at a query all day. Alerts watch for you.
A log search alert runs a query on a schedule and takes action when the result meets a condition. For example, it can run every five minutes, count server errors in that period, and fire if the count is above a threshold. Alerts do not work alone. They use action groups, which define who to notify and what to trigger.
Alert rule
Defines the query, how often it runs, the condition, the severity, and the scope.
Action group
A reusable list of actions, such as email, text message, push notification, webhook, or running an automation.
Alert processing rules
Rules that suppress alerts during planned maintenance or route them differently without editing every alert.
Alert state
Alerts can be fired, acknowledged, and resolved, so teams can track who is handling what.
flowchart LR
WS["Workspace data"] --> Q["Scheduled query runs every few minutes"]
Q --> C{"Condition met?"}
C -->|No| N["Do nothing"]
C -->|Yes| A["Alert fires"]
A --> AG["Action group"]
AG --> E["Email or text"]
AG --> W["Webhook or ticket"]
AG --> R["Automation runbook"]
| Alert type | Based on | Good for |
|---|---|---|
| Metric alert | Numbers measured regularly | Fast, simple conditions such as high CPU |
| Log search alert | Results of a query over log data | Complex conditions and patterns across events |
| Activity log alert | Management events in Azure | Knowing when a resource is created, changed, or deleted |
Creating too many noisy alerts. When people receive dozens of unimportant messages, they stop reading them, and the one critical alert gets lost. Alert only on things that need human action, and tune thresholds regularly.
A log alert is a night watchman who checks the same list every few minutes and rings the right bell when something is wrong. The action group decides which bell rings and who gets woken up.
12Workbooks, Dashboards, and Sharing Insights
Good answers should reach the people who need them.
Azure Workbooks
Interactive reports that combine text, queries, charts, and parameters. Users can pick a time range or a resource and the whole report updates.
Azure dashboards
Pinned tiles from queries and metrics on a shared page in the Azure portal, useful for a quick overview.
Managed Grafana
A hosted version of the popular Grafana dashboards, which can read from Azure Monitor and workspaces for teams that prefer Grafana.
Power BI
Query results can be exported or connected to Power BI for polished business reports.
Insights experiences
Ready-made views for virtual machines, containers, storage, and applications built on workspace data.
| Need | Good choice |
|---|---|
| An interactive troubleshooting guide with filters | Workbook |
| A wall display of key numbers | Dashboard or Grafana |
| A monthly report for managers | Power BI or an exported workbook |
| A quick ready-made view of VM health | VM insights |
The workspace is the raw ingredients in the kitchen. A workbook is a recipe card that people can follow and adjust. A dashboard is the menu board showing the daily specials at a glance.
13Designing Your Workspaces
One workspace or many? This design decision matters for cost, security, and convenience.
Many organizations start with one workspace per environment or per team, and later discover problems. The right answer depends on your needs. A few forces pull in different directions.
Fewer workspaces are simpler
Everything is in one place, so queries across systems are easy and there is less to manage.
Security and ownership
Some teams or customers must never see each other’s data. Separate workspaces make this simple, though table-level and resource-level access can also help.
Data location and regulation
A workspace lives in one region. Rules about where data may be stored can force separate workspaces in different regions.
Retention and cost settings
Settings such as retention and commitment pricing apply at the workspace level, so very different needs may justify separate workspaces.
Cost of moving data
Sending data between regions can add networking charges, so keep workspaces near their sources where practical.
| Design | Strength | Weakness | Often used when |
|---|---|---|---|
| One central workspace | Simple, easy cross-system queries, one place for security tools | Access control and charge-back need careful design | One organization, one region, shared operations and security teams |
| One per environment | Clear separation of production from testing | Cross-environment views need cross-workspace queries | Strict production isolation is required |
| One per team or business unit | Clear ownership and cost allocation | Many workspaces to manage, harder to see the whole picture | Teams are independent and run their own operations |
| One per region | Meets data residency, reduces cross-region traffic | More workspaces, split views | Regional compliance rules apply |
Start with as few workspaces as your security, regulatory, and ownership needs allow, and use access control and cross-workspace queries to bridge gaps. Too many small workspaces create blind spots and extra work.
Workspaces are like filing rooms. One big room is easy to search but needs good locks inside. Many small rooms are naturally separate but make it harder to see a connection between papers in different rooms.
14Security and Access Control
Logs often contain sensitive information, so who can read them matters.
Azure role-based access control
Roles such as Log Analytics Reader and Log Analytics Contributor decide who can query or manage a workspace.
Access modes
Workspace-context access lets a user see everything they have been granted in the workspace. Resource-context access lets a user see only the logs for resources they already have access to.
Table-level access
Roles can be granted for specific tables, so someone can read performance logs but not security logs.
Network isolation
Azure Monitor Private Link Scope lets data travel and be queried over private connections, and public access can be restricted.
Encryption and customer-managed keys
Data is encrypted at rest. Dedicated clusters can allow you to bring your own key for extra control.
Auditing the auditors
Query activity and workspace changes can themselves be logged, so you know who looked at what.
Data minimization
Filter out sensitive fields before storage using transformations, so secrets and personal data are not kept unnecessarily.
Applications sometimes write passwords, tokens, or personal details into log messages by mistake. Anything stored in a workspace can be read by people with access, so avoid logging sensitive values in the first place, and use transformations to remove them when needed.
Problem
Giving everyone Contributor access to the central workspace so they can “look at logs whenever they need”.
Why It’s Harmful
Contributors can change settings, delete data, and read sensitive tables. A mistake or a stolen account can damage your monitoring and expose private information.
Correct Approach
Give read-only roles, scope access to the resources or tables people actually need, and reserve management roles for a small trusted group.
Logs are the security camera footage of your systems. Not everyone in the company should watch every camera. Staff see the cameras for their own area, and only a few trusted people can delete or edit recordings.
15Retention, Archiving, and Search Jobs
How long should logs be kept, and where?
Data in a workspace is first kept in interactive retention, where it is available for fast queries and alerts. A default period applies, and you can change it per workspace and per table within allowed limits. After that, data can be kept in long-term retention, sometimes called archive, which is much cheaper but cannot be queried directly in the normal way. The total retention can extend for many years to support audits and legal requirements. Exact defaults, maximums, and pricing change over time, so confirm them in the current documentation.
Interactive retention
Recent data stays fully queryable and can drive alerts. Keep this period as short as your daily needs allow.
Long-term retention
Older data moves to cheaper storage inside the same table, kept for compliance or rare investigations.
Search job
When you need old data, a search job runs a query across the long-term data and writes matching rows into a new results table that you can query normally.
Restore
For a larger look at a specific time period, a restore operation brings a chunk of archived data back into a temporary table for fast queries.
Deletion
After the total retention period, the data is removed permanently.
flowchart LR
ING["New data arrives"] --> INT["Interactive retention: fast queries and alerts"]
INT --> LTR["Long-term retention: low cost storage"]
LTR --> SJ["Search job: pull matching rows into a results table"]
LTR --> RS["Restore: temporarily bring back a time range"]
LTR --> DEL["Deleted after total retention ends"]
Data export
Continuously copy selected tables to a storage account or event hub for external archiving, processing, or integration with other tools.
Per-table retention
Set different retention for different tables, keeping security logs long and noisy debug logs short.
Immutability needs
Some regulations require logs that cannot be changed. Investigate export options and storage with immutability when required.
Retention is like keeping paperwork. Recent files sit in the drawer on your desk. Older files go into the cheap warehouse across town. When you need an old file, you send a request and the warehouse brings back only the pages you asked for.
16Cost Management: Paying Only for What You Need
Log Analytics is billed mainly by how much data you send in, so volume is the key lever.
The biggest cost is usually data ingestion, counted in gigabytes sent into the workspace, and its price depends on the table plan. Other costs include long-term retention beyond the free period, queries on cheaper plans, search jobs, restores, and data export. Some data types have free allowances or special rules. Prices change, so always check the pricing page and use the Azure pricing calculator.
Pay-as-you-go
You pay per gigabyte ingested with no commitment. Best for small or unpredictable volumes.
Commitment tiers
You commit to a daily volume at a discounted rate. Best when your daily ingestion is large and steady.
Dedicated clusters
Reserved capacity for very large volumes, with extra features such as customer-managed keys.
Daily cap
A limit on how much data is ingested per day. Useful as a safety net, but when reached, new data is not collected for the rest of the day, so you may lose visibility.
Practical ways to reduce cost
Find the biggest tables
Use the workspace usage views to see which tables and sources produce the most data.
Collect only what you use
Turn off diagnostic categories you never look at, and trim the event lists in data collection rules.
Filter and shrink on the way in
Use transformations to drop noisy rows and unneeded columns before they are stored.
Use cheaper table plans for low-value data
Move high-volume, rarely queried data to lower-cost plans.
Set sensible retention
Keep interactive retention short and use long-term retention for what you must keep.
Consider a commitment tier
If daily volume is large and stable, a commitment tier can lower the price per gigabyte.
Set cost alerts
Create alerts on unusual ingestion spikes and on budgets, to catch a runaway logging bug before the bill arrives.
Problem
Turning on every diagnostic category for every resource “just in case”, and sending it all to the highest-cost plan.
Why It’s Harmful
Most of the data is never read, yet it is billed by volume. Real problems become harder to find among the noise, and the monitoring bill can exceed the cost of the system being monitored.
Correct Approach
Decide what questions you need to answer, collect the logs that answer them, filter noise, and use cheaper plans and shorter retention for low-value data.
Paying per gigabyte is like paying for the weight of luggage on a flight. Packing everything you own is expensive, and you will never use most of it. Pack what you will really need and the trip costs far less.
17Transformations: Cleaning Data on the Way In
You do not have to store data exactly as it arrives.
A transformation is a small query attached to a data collection rule that runs on incoming data before it is stored. It can remove rows you do not need, drop columns, mask sensitive values, add calculated columns, or split one stream into several tables. Because it runs before storage, it can reduce ingestion volume and lower cost, while improving privacy.
Filter noise
Drop successful health-check requests or debug messages that nobody reads.
Remove columns
Leave out large, unused fields to shrink every record.
Mask sensitive data
Hide parts of identifiers or remove personal details before they are stored.
Shape custom logs
Turn free-text lines from files into structured columns that are easier to query.
Route to different tables or plans
Send valuable records to a full-featured table and bulk records to a cheaper one.
flowchart LR
SRC["Incoming records"] --> TR["Transformation in the DCR"]
TR -->|Keep and reshape| T1["Analytics table: valuable data"]
TR -->|Bulk data| T2["Basic or Auxiliary table: low cost"]
TR -->|Dropped| X["Noise removed before storage"]
Transformations are powerful but permanent for the records they drop, because dropped data is never stored. Test them carefully, start with small changes, and keep a record of what you filter and why.
A transformation is a security-and-sorting desk at the entrance of the archive. Junk mail is thrown away at the door, private details are blacked out, and valuable papers go to the main shelves while bulk papers go to the cheap storage room.
18Log Analytics with Other Azure Services
The workspace is a hub that many Azure services connect to.
| Service | How it uses a workspace |
|---|---|
| Application Insights | Stores application telemetry in a workspace so app and infrastructure logs can be queried together |
| Microsoft Sentinel | A security information and event management solution that is enabled on a workspace and uses its data for threat detection and investigation |
| Microsoft Defender for Cloud | Sends security data and recommendations that can be stored and queried in a workspace |
| VM insights and Container insights | Ready-made monitoring views built on workspace data |
| Azure Automation and Logic Apps | Can be triggered by alerts, and can write their own run logs to a workspace |
| Azure Policy | Can enforce or deploy diagnostic settings at scale so resources always send logs |
| Event Hubs and Storage accounts | Destinations for exported data, for integration with other tools or long archives |
Policy-driven diagnostics
Use Azure Policy to automatically create diagnostic settings for new resources, so logging never depends on someone remembering.
Cross-workspace queries
Query several workspaces in one query when your design has more than one, for example across production and testing.
Alert-driven automation
Let an alert trigger a runbook or workflow that fixes common problems automatically, such as restarting a stuck service.
The workspace is the central train station of your monitoring. Lines from many cities arrive there, and passengers, meaning insights and alerts, can change trains to reach any destination.
19Real-World Use Cases
See how different organizations use Log Analytics every day.
Operations team keeping a web store healthy
Request, exception, and dependency logs flow into one workspace. Alerts fire when error rates rise, and a workbook shows the slowest pages and the database calls behind them.
Security team hunting for threats
Sign-in, network, and activity logs feed Microsoft Sentinel on a workspace. Queries and rules look for unusual locations, repeated failures, and unexpected changes to permissions.
Platform team managing hundreds of virtual machines
One data collection rule is applied to all machines through Azure Policy. VM insights show which machines are low on disk or memory, and patch compliance is tracked in the same place.
Compliance team keeping audit trails
Activity and access logs are kept in long-term retention for years, with search jobs available when auditors ask about a specific event.
FinOps team controlling monitoring cost
The team reviews ingestion per table, trims unused categories, applies transformations, and moves bulk logs to cheaper plans, cutting the bill without losing important visibility.
Developers troubleshooting a release
After a deployment, developers compare error counts before and after and trace a failing request across services using a shared operation identifier.
20Log Analytics Compared with Related Options
Knowing the neighbors helps you choose the right tool.
| Option | What it is | Choose it when |
|---|---|---|
| Log Analytics workspace | Central store and query engine for logs in Azure Monitor | You want searchable, alertable logs across Azure and beyond |
| Azure Monitor metrics | Fast numeric time series for resources | You need quick health charts and simple, fast alerts |
| Application Insights | Application performance monitoring | You want deep insight into your own application’s requests and failures |
| Azure Data Explorer | A large-scale analytics database using the same query language | You need massive custom analytics, very long history, or full control of cluster sizing |
| Storage account archive | Cheap file storage for logs | You only need to keep logs, not query them regularly |
| Microsoft Sentinel | Security analytics built on a workspace | Security detection and response is the main goal |
| Third-party logging tools | Splunk, Elastic, Datadog, and others | Your organization has standardized on them. Azure Monitor can export data to many of them |
Use metrics for fast, simple health signals, and logs for investigation and deeper analysis. Use a workspace as your central log home, and export to cheaper or external stores only when you have a clear reason.
21Advantages and Trade-offs
An honest look helps you decide with open eyes.
Advantages
- One central place to collect and search logs from many sources
- Powerful and fast query language for large volumes of data
- Deep integration with Azure services, alerts, workbooks, and security tools
- Flexible retention, archive, and table plans to balance cost and access
- Fine-grained access control and private networking options
- Managed service, so no servers or storage to maintain
Disadvantages / Trade-offs
- Costs grow with data volume and can surprise teams that collect everything
- Learning KQL takes time, although the basics are quick to pick up
- Workspace design mistakes are tedious to correct later
- Cheaper table plans limit query and alert features
- Services do not send logs automatically, so gaps appear if diagnostics are forgotten
- A workspace is regional, which affects data residency and cross-region cost
22Best Practices and Mistakes to Avoid
Habits that keep your logging useful, secure, and affordable.
Start from the questions
Decide what you need to know, such as failures, security events, and performance, and collect the logs that answer those questions.
Automate diagnostic settings
Use Azure Policy and templates so every new resource sends logs to the right workspace from day one.
Design workspaces deliberately
Use as few workspaces as your security, regulatory, and ownership needs allow, in suitable regions.
Control access tightly
Grant read-only roles, scope access by resource or table, and protect the workspace with private networking when required.
Manage cost continuously
Review ingestion by table, filter noise with transformations, choose table plans wisely, and watch for spikes.
Make alerts meaningful
Alert only on conditions that need action, route them through action groups, and tune them regularly.
Share knowledge
Save and document useful queries and workbooks so the whole team benefits, especially during incidents.
Set retention to match need
Keep fast-query data short, archive what you must keep, and confirm compliance requirements with your legal team.
Problem
Setting a daily cap very low to avoid bills, and then forgetting about it.
Why It’s Harmful
Once the cap is reached, new data is no longer collected for the rest of the day. An outage or an attack late in the day may leave no logs at all.
Correct Approach
Reduce cost by filtering and choosing the right plans, set cost and spike alerts, and use a cap only as a deliberate, monitored safety net.
Problem
Creating dozens of workspaces, one for each small project, with no overall plan.
Why It’s Harmful
Logs are scattered again, cross-system investigations are hard, policies and alerts are duplicated, and nobody has the full picture.
Correct Approach
Share workspaces where security and ownership allow, use resource-context access and table-level access for separation, and use cross-workspace queries when needed.
Problem
Setting up logging only after a serious incident has already happened.
Why It’s Harmful
Logs from before the incident do not exist, so the cause may never be known, and the same problem can happen again.
Correct Approach
Turn on essential logs, alerts, and retention as part of every new system’s launch checklist, before real users arrive.
23Frequently Asked Questions
No. Azure Monitor is the overall monitoring platform in Azure, including metrics, alerts, and insights. Log Analytics is the tool and workspace within Azure Monitor for collecting, storing, and querying log data. People often say Azure Monitor Logs for the same logging feature.
Usually not. Many resources keep only limited platform data unless you create diagnostic settings that route chosen log categories to a workspace. Virtual machines also need an agent and a data collection rule. Policy can automate this at scale.
You can use ready-made insights, workbooks, and saved queries without writing much. But a basic knowledge of KQL greatly improves your ability to investigate problems, because most questions are answered by filtering, summarizing, and sorting data.
A default retention period applies to each workspace and table, and you can extend it, including long-term retention for many years at lower cost. The exact defaults and limits change, so check the current documentation and set retention to match your needs and any regulations.
Almost always because of high ingestion volume, often from one or two noisy tables or sources. Review ingestion by table, switch off unneeded categories, filter with transformations, use cheaper table plans where appropriate, and consider a commitment tier for large steady volumes.
Yes. Cross-workspace queries let you combine data from several workspaces in one query, as long as you have access to each one. This is helpful when your design has separate workspaces for different environments or teams.
Use resource-context access so people see logs only for the resources they can already access, or use table-level role assignments, or give the team its own workspace if separation must be strict.
Yes. You can use Application Insights, the Azure Monitor Agent to collect custom log files, or the logs ingestion API with a data collection rule to send custom data into custom tables.
24Summary and Key Takeaways
Log Analytics is Azure’s central place for collecting, storing, and querying log data. Data flows into a workspace from Azure resources through diagnostic settings, from machines through the Azure Monitor Agent and data collection rules, from applications through Application Insights, and from custom programs through ingestion APIs. Inside the workspace, data lives in tables with plans that balance cost and features, and you ask questions with the Kusto Query Language. Alerts, workbooks, dashboards, and other services turn that data into action and understanding. Good workspace design, tight access control, sensible retention, transformations that clean data on the way in, and ongoing cost management keep the system secure and affordable. Collect the right data, ask good questions, and your logs become the clearest explanation of what your systems are doing.
Key Takeaways
- One central diary — a Log Analytics workspace collects logs from many sources into tables that you can search together.
- Turn the pipes on — use diagnostic settings, the Azure Monitor Agent, and data collection rules, ideally automated with Azure Policy, so logs actually arrive.
- Learn the query pipeline — KQL filters, summarizes, and sorts rows step by step, and fast queries start with a time range and early filters.
- Alerts need care — log search alerts and action groups notify the right people, but only meaningful alerts keep people paying attention.
- Design workspaces deliberately — use as few as security, regulation, and ownership allow, and control access with roles, resource-context, and table-level permissions.
- Match plans and retention to value — keep valuable data in full-featured tables, move bulk data to cheaper plans, and archive long-term data with search jobs and restore when needed.
- Control cost at the source — most cost is ingestion volume, so collect what you use, filter with transformations, watch for spikes, and consider commitment tiers for steady volume.