Azure Bot Service: Building, Connecting and Running Conversational Bots
A complete, plain-English guide to Azure Bot Service: how bots work, the activity protocol, channels, state, dialogs, language understanding, generative AI, identity, hosting, security, monitoring, cost, and how to plan around a fast-changing Microsoft bot ecosystem.
Imagine a busy hotel front desk that never sleeps. Guests can walk up, phone in, send a text message, write on a chat app, or even speak through a screen in their room. Whichever way they reach out, the desk answers in the same helpful way: it checks the booking, answers questions about breakfast, orders extra towels, and hands difficult cases to a human manager. The hotel does not build a separate desk for each method of contact. It has one trained receptionist, and a switchboard that connects every method to that receptionist. A bot is the trained receptionist, and Azure Bot Service is the switchboard and the staff manager that connects the receptionist to the world.
This guide assumes you know nothing about bots. Each new term is explained when it first appears, with an everyday comparison. We cover how a message travels from a person’s phone to your code and back, how the service keeps conversations secure, how to remember what a user said, how to add language understanding and generative AI, and how to run, watch and pay for a bot in production. One important note up front: Microsoft’s bot technologies have been changing quickly. Some tools have been retired, some renamed, and some replaced by newer offerings. We teach the lasting ideas and flag the places where you must check the current official documentation before you commit to a design.
1What a Bot Is and Why Organizations Build Them
Before the technology, we need a clear picture of the job.
A program you talk to
A bot, short for robot, is a software program that holds a conversation with people through text, voice or buttons. Instead of clicking through menus and forms, a person types or says what they want, such as “reset my password” or “where is my order”, and the bot figures out the request, performs the action or looks up the answer, and replies. Bots live inside the places where people already spend their time: workplace chat tools, websites, mobile apps, phone lines and messaging applications.
Organizations build bots for several reasons. Customer service teams want to answer common questions at any hour without adding staff. Human resources and information technology teams want employees to get quick answers about leave, benefits and device problems. Sales teams want to qualify visitors on a website. Operations teams want to trigger workflows from a chat window. In every case the bot saves time for people and reduces the load on busy human teams.
A bot is like a vending machine that understands plain language. A traditional website is a shop where you walk the aisles yourself. A bot is a helpful shop assistant who asks what you need and fetches it, and who can still call a human colleague when the question is too tricky.
Three kinds of bots
Rule-based bots
Follow scripted paths with menus and keyword matching. Predictable and cheap, but they fail when users phrase things in unexpected ways.
Intent-based bots
Use language understanding to detect what the user wants, called an intent, and pull out details, called entities. More flexible, and easier to extend with new skills.
Generative AI bots
Use large language models to understand and write natural responses, often grounded in the organization’s own documents so that answers stay accurate.
Most real bots are hybrids. They use scripted flows for sensitive tasks such as payments, language understanding to route requests, and generative AI to answer open questions from a knowledge base. Mixing the three gives control where you need it and flexibility where you want it.
2What Azure Bot Service Is and the Changing Landscape
Understand what the service does, and what it does not do.
Azure Bot Service is the Azure resource that registers a bot, gives it an identity, and connects it to channels such as Microsoft Teams, a website chat window and other messaging platforms. It acts as the switchboard between channels and your bot’s code. It does not, by itself, contain the bot’s brain. Your bot’s logic runs in code that you host somewhere, such as an Azure App Service, and the Azure Bot resource points to that code through a web address called the messaging endpoint.
flowchart LR
A["User on a channel"] --> B["Channel such as Teams or web chat"]
B --> C["Azure Bot Service connector"]
C -->|HTTPS to messaging endpoint| D["Your bot code hosted on Azure"]
D --> E["Data, language and AI services"]
D --> C
C --> B
B --> A
A landscape in motion
The bot ecosystem around the service has seen many changes. A software development kit, or SDK, is a set of libraries that makes building a bot easier. For years the Bot Framework SDK was the main option, available for several programming languages, together with companion tools such as an emulator for local testing and a visual design tool. Microsoft has announced that this SDK and several of its companions are moving out of active development and support, and is directing developers toward newer tools, including a Microsoft 365 oriented agents SDK and low-code agent builders. Several language-understanding and question-answering services that bots used to rely on have also been retired and replaced by newer language services. The older Power Virtual Agents product has been folded into a newer agent-building product.
Support dates, retirement schedules, recommended SDKs, supported authentication types and pricing for bot technologies have been changing. Before starting any new bot or committing to a long-lived design, read the current Microsoft guidance on which SDKs and services are recommended for new projects. The concepts in this guide, such as channels, activities, state, identity and hosting, carry over to the newer tools, but the code libraries and names may differ.
| Layer | What it is | Stability of the idea |
|---|---|---|
| Azure Bot resource | Registration, identity and channel connections | Stable concept, details evolve |
| Activity protocol | The message format used between channels and bots | Stable and widely used |
| Bot code SDK | Libraries that help your code handle activities | Changing, check current recommendation |
| Language and knowledge services | Intent detection, question answering, generative AI | Changing, older services retired |
| Hosting on Azure | App Service, containers, functions | Stable |
The switchboard and the language of calls, who is calling and what they said, stay the same even when the company replaces its receptionist training program. Learn the switchboard well, and you can adapt to new receptionist tools.
3Architecture: The Pieces and How They Fit
A bot solution has more parts than a single program.
Channel
The place where the user talks, such as Teams, a web page, a messaging app or email. Each channel has its own look and abilities.
Bot Connector service
The Microsoft-run service that receives messages from channels, translates them into a common format, and forwards them to your bot. It also carries your replies back.
Azure Bot resource
The configuration object in your Azure subscription: name, identity, messaging endpoint, channels and sign-in connections.
Bot application
Your own code, hosted on a web server, that receives messages and decides how to respond. This is the actual brain.
State storage
A database or storage account that remembers conversation details between messages, because the web server itself forgets between requests.
Supporting services
Language understanding, knowledge search, generative AI, business systems and monitoring that your bot calls to do its job.
flowchart TD
subgraph Channels["Channels"]
T["Teams"]
W["Web chat"]
M["Messaging apps"]
end
Channels --> BC["Bot Connector service"]
BC --> BR["Azure Bot resource: identity and endpoint"]
BR --> APP["Bot application on App Service or containers"]
APP --> ST["State storage"]
APP --> AI["Language and AI services"]
APP --> BIZ["Business systems and APIs"]
APP --> MON["Application Insights"]
Channels are the doors into the hotel, the connector is the lobby that checks and routes visitors, the bot resource is the registration desk that knows which receptionist is on duty, and the bot application is the receptionist. State storage is the guest notebook, and supporting services are the departments the receptionist calls for help.
Draw this diagram for your own project before writing any code, with the real names of each part. It exposes missing pieces, such as where state lives or how the bot authenticates to a business system, while changes are still cheap.
4The Activity Protocol and the Life of a Turn
Everything a bot receives or sends is an activity.
Channels differ greatly. One shows rich cards, another only plain text, another sends voice. To avoid writing separate code for each, Bot Service uses a common message format called the activity protocol. An activity is a small structured message that says what happened. The most common type is a message activity, which carries text the user typed. Others tell the bot that a user joined the conversation, that someone is typing, that a button was pressed, or that a special event occurred.
Message
A person sends text, an attachment or a button selection, or the bot sends a reply.
Conversation update
Members were added or removed. Often used to greet a new user when they first open the chat.
Typing
Shows that the other side is composing a message, and can be sent by the bot to signal that work is in progress.
Event and invoke
Special signals from a channel or an application, such as a card action or a sign-in result, some of which expect an immediate structured reply.
End of conversation
Indicates the conversation has finished, so the bot can clean up.
What happens in one turn
A turn is one round trip: the user sends something, the bot processes it, and the bot replies. Each turn begins when the connector posts an activity to your bot’s messaging endpoint over a secure web connection. Your bot validates that the request is genuine, runs its logic, and sends replies back through the connector.
sequenceDiagram
participant U as User
participant CH as Channel
participant BC as Bot Connector
participant BOT as Your bot
participant ST as State storage
U->>CH: Type a message
CH->>BC: Deliver message
BC->>BOT: POST activity with security token
BOT->>BOT: Validate token and run middleware
BOT->>ST: Load conversation and user state
BOT->>BOT: Run logic or dialog step
BOT->>ST: Save updated state
BOT->>BC: Send reply activity
BC->>CH: Deliver reply
CH->>U: Show reply
Each turn is like a letter exchange through a post office. The letter has a standard envelope, the activity, that every post office understands. Whatever the sender used, a paper note or an email printout, the receptionist always opens the same kind of envelope.
Assuming the web server remembers the conversation between turns. Each turn may be served by a different server instance, or the server may restart. Anything the bot must remember has to be saved in a proper state store.
5Channels: Where Your Bot Meets People
Reach users where they already are, and respect each place’s strengths and limits.
| Channel | Typical use | Notes |
|---|---|---|
| Microsoft Teams | Employee-facing assistants, notifications, workflows | Rich cards, tabs, messaging extensions, strong identity integration |
| Web Chat | Chat widget on a website or intranet | An open-source component you embed and style |
| Direct Line | Custom apps, mobile apps and kiosks | A secure programming interface for building your own chat client |
| Direct Line Speech | Voice assistants and devices | Combines speech recognition and synthesis with the bot |
| Email and SMS | Notifications and simple exchanges | Limited formatting, often through third-party providers |
| Social and messaging apps | Consumer customer service | Each has its own rules, approvals and message limits |
| Microsoft 365 experiences | Assistants inside productivity tools | Evolving offerings, check current options |
The list of supported channels changes. Some older channels have been retired, and third-party platforms update their rules. Confirm that your chosen channels are currently supported, and check each platform’s own policies, such as approval processes and message window limits.
Design for the least capable channel first
If a bot must work in several places, decide what the minimum experience is. A rich card with five buttons may not exist in a basic text channel. Many teams write the core logic in a channel-neutral way and then adapt the presentation per channel, falling back to plain text with numbered options where cards are not available.
A speaker giving the same talk in a theater, over radio and in a text newsletter must adapt: slides in the theater, vivid words on radio, and clear headings in print. The message is the same, but the presentation changes to suit each medium.
Start with one or two channels that match your real users. Adding channels is easy later if the core logic is cleanly separated from presentation, and hard if the code is full of channel-specific shortcuts.
6Inside the Bot: Handlers, Middleware, State, Dialogs and Cards
The building blocks that appear in nearly every bot framework.
Turn context
An object representing the current turn: the incoming activity, the tools to reply, and access to services. It exists for the duration of one turn.
Activity handlers
Methods that react to each kind of activity: one for messages, one for members joining, and so on. They keep the code organized by event.
Middleware
Components that sit in the path of every turn to add common behavior such as logging, translation, typing indicators or transcript saving, before and after your logic.
State
Stored facts about the conversation or user, such as what step they are on or their chosen language.
Dialogs
Reusable conversation flows that ask a series of questions, validate answers and keep their place across turns.
Adaptive Cards
A cross-platform way to describe rich visual elements such as forms, images and buttons in a neutral format that each channel renders in its own style.
Dialogs and waterfalls
Many tasks need several steps: ask for the date, ask for the room type, confirm, then book. A dialog manages that sequence. A common style is the waterfall, where each step runs, asks a question, waits for the answer on the next turn, and then moves to the following step. Prompts validate input, such as ensuring a number is a number or a date is a real date, and re-ask politely if it is not. Dialogs can call other dialogs, which makes complex conversations out of simple pieces.
flowchart TD
A["Start booking dialog"] --> B["Ask for check-in date"]
B --> C["Validate date"]
C -->|Invalid| B
C -->|Valid| D["Ask for number of guests"]
D --> E["Confirm details with a card"]
E -->|Yes| F["Call booking system"]
E -->|No| B
F --> G["Send confirmation and end dialog"]
A dialog is a paper form filled in by a patient clerk who asks one question at a time, checks each answer, and keeps the half-filled form in a folder so that the conversation can continue even if the customer steps away for a minute.
Interruptions
Real people change their minds, say “cancel” in the middle of a flow, or ask for help. A good bot checks for interruptions before continuing the current dialog, and can leave a flow cleanly, ask whether the user really wants to cancel, or answer a quick side question and then return to where they were.
Keep turn handling fast. Channels expect a response in a few seconds. For slow work, such as a long database query or an external approval, reply immediately with a short acknowledgment and use proactive messaging later to deliver the result.
7Conversation Design: Making a Bot People Like
The best technology fails if the conversation is awkward.
Say what the bot can do
Greet the user, state its purpose, and offer a few clear starting options. People do not know what to ask a blank chat box.
Ask one thing at a time
Long forms in chat feel like interrogation. Break them into short steps and show progress.
Offer buttons for common choices
Buttons reduce typing errors and speed up simple decisions, while still allowing free text.
Handle misunderstanding gracefully
When the bot is unsure, say so, rephrase the question, and offer examples. Do not loop endlessly on the same fallback message.
Provide a human handoff
For frustration, complex cases or sensitive topics, pass the conversation to a person with the context already gathered so the user does not repeat everything.
Be honest that it is a bot
Users deserve to know they are talking to software, and trust grows when the bot is transparent about its limits.
A good bot behaves like a good shop assistant: greets you, asks what you need, offers helpful options, admits when it does not know, and calls the manager rather than arguing in circles.
Problem
The bot answers every unclear request with the same generic apology and has no path to a human.
Why It’s Harmful
Users become frustrated, abandon the bot and form a lasting negative opinion of the service. Support teams then receive angrier contacts.
Correct Approach
Vary fallback messages, offer examples and menus after one failed attempt, and provide a clear route to a human after repeated misunderstandings.
8Language Understanding and Knowledge Answers
How a bot figures out what a sentence means.
Humans phrase the same request in countless ways: “where is my parcel”, “track my order”, “has my package shipped yet”. Language understanding maps these to one intent, such as track order, and extracts details like an order number, called entities. A trained model learns from example sentences that you provide. Azure’s language services offer this through conversational language understanding, which replaced earlier services that have been retired. Check the current service name and features in the documentation.
Intent recognition
Decides what the user wants from a short list of tasks the bot supports.
Entity extraction
Pulls out specific details such as dates, places, product names and numbers.
Question answering
Finds the best answer to a factual question from a body of documents or a list of frequently asked questions.
Orchestration
Routes a message to the right brain: a task flow, a knowledge base, or a different bot, depending on what the user asked.
Translation and speech
Services that translate text between languages and convert speech to text and back, to widen the audience.
flowchart TD
A["User message"] --> B["Language understanding or orchestrator"]
B -->|Task intent| C["Run a dialog or call a business API"]
B -->|Factual question| D["Search knowledge and answer"]
B -->|Low confidence| E["Ask a clarifying question"]
B -->|Request for human| F["Hand off to agent"]
Language understanding is a mail sorter who reads each envelope and drops it into the right tray: billing, shipping or complaints. Entity extraction is the sorter also copying the order number onto a sticky note for the clerk.
Training data and confidence
Models need varied example sentences for each intent, written by real people and refreshed from real conversations. Every prediction comes with a confidence score. Set a threshold: above it, the bot acts; below it, the bot asks to clarify. Review low-confidence conversations regularly and add the missing examples. A language model is a living asset, not a one-time setup.
Creating many similar intents with overlapping examples. The model cannot tell them apart and confidence collapses. Keep intents clearly distinct, and merge those that lead to the same action.
9Generative AI Bots: Grounded Answers with Your Own Data
Large language models can converse naturally, but they must be kept honest.
A large language model, or LLM, is an AI system trained on vast amounts of text that can write fluent answers. Left alone, it can invent facts, called hallucination, and it knows nothing about your private documents. The popular remedy is retrieval-augmented generation, usually shortened to RAG. When a user asks a question, the bot first retrieves relevant passages from your own documents using a search service, then gives those passages to the model with instructions to answer only from them. The answer is therefore grounded in your content, and can cite its sources.
sequenceDiagram
participant U as User
participant B as Your bot
participant S as Search service
participant L as Language model
U->>B: Ask a question
B->>S: Search documents with the question
S-->>B: Most relevant passages
B->>L: Question plus passages plus instructions
L-->>B: Grounded answer with references
B-->>U: Reply with sources
A plain language model is a brilliant guest who has read widely but has never seen your company handbook. RAG is handing that guest the right pages of the handbook before each question and asking them to answer only from those pages.
Guardrails every generative bot needs
Grounding instructions
Tell the model to answer only from provided passages and to say it does not know when the answer is missing.
Content safety filtering
Screen inputs and outputs for harmful content using safety services, and block or soften unsafe responses.
Access-aware retrieval
Only retrieve documents the signed-in user is allowed to see. Otherwise the bot becomes a back door to confidential files.
Prompt injection defense
Treat retrieved text and user input as untrusted. Malicious text can try to override your instructions, so separate and constrain it.
Citations and feedback
Show where answers came from and let users flag bad ones, so that you can improve the sources and prompts.
Limits on actions
If the model can trigger actions, require confirmation for anything sensitive and enforce permissions in code, not in the prompt.
Problem
The bot indexes all company documents into one search index and answers any user from it, regardless of who is asking.
Why It’s Harmful
An employee can ask a question and receive content from documents they were never allowed to read, such as salary files or legal memos.
Correct Approach
Attach access information to each indexed document and filter retrieval by the signed-in user’s permissions before anything reaches the model.
Measure answer quality with a test set of real questions and expected answers, and rerun it whenever you change the prompt, the model or the documents. Without measurement, quality drifts silently.
10Identity and Authentication
Two separate questions: how does the bot prove who it is, and how does it know who the user is?
The bot’s own identity
Every Azure Bot resource is tied to an identity in Microsoft Entra ID, Azure’s identity service. This identity lets the connector and your bot trust each other. When the connector sends an activity to your endpoint, it includes a signed security token. Your bot verifies that token to be sure the message came from the real service and not from an impostor. When your bot replies, it obtains its own token and presents it to the connector.
Several identity types exist for bots, including a managed identity that avoids storing any secret, and an application registration limited to your own organization’s tenant. Older bots used a broader registration that worked across organizations, and Microsoft has been restricting the creation of that type. Check current documentation for which types are supported for new bots, because this has changed.
Managed identity
Azure manages the credential automatically. No password to store or rotate, which removes a common source of leaks.
Single-tenant app registration
An application identity restricted to your organization, usually with a secret or certificate that you must protect and rotate.
Multi-tenant app registration
An older option for bots meant to be used across organizations. Availability for new bots has been reduced, so check before relying on it.
sequenceDiagram
participant BC as Bot Connector
participant BOT as Your bot
participant EID as Microsoft Entra ID
BC->>BOT: Activity with signed token
BOT->>BOT: Validate signature, audience and issuer
BOT->>EID: Request token for the bot identity
EID-->>BOT: Access token
BOT->>BC: Reply activity with access token
The user’s identity
Sometimes the bot must act on behalf of the user, for example to read their calendar or fetch their leave balance. For that the bot needs the user’s permission and a token for the target service. Bot Service provides a sign-in feature: you configure an OAuth connection to an identity provider, and the bot can start a sign-in card. After the user signs in, the service stores and refreshes the token on the bot’s behalf. In Teams, single sign-on can make this nearly invisible when the user is already signed in.
The bot’s identity is the receptionist’s staff badge that proves to the lobby that they work here. The user’s sign-in is the guest handing over a signed permission slip allowing the receptionist to fetch their personal documents from another office.
Letting the bot use one powerful shared service account to read data for every user. Anyone who can talk to the bot then gets that account’s access. Prefer acting with the user’s own delegated permissions, so that existing access rules are respected.
11Hosting and Scaling Your Bot Application
Your bot code is a web application, so it can run anywhere a web application can.
| Hosting option | Strength | Consider when |
|---|---|---|
| Azure App Service | Simple, managed, built-in scaling and deployment slots | The default for most bots |
| Azure Container Apps | Containers with scale to zero and event-driven scaling | You want container workflows with modest operations effort |
| Azure Kubernetes Service | Full control and large scale | Your organization already runs Kubernetes at scale |
| Azure Functions | Pay per execution, quick to start | Light bots with simple stateless logic, mindful of cold starts |
| Virtual machines | Total control | Special requirements, at the cost of more maintenance |
Scaling
Because each turn is a standalone request and state lives outside the server, a bot scales horizontally: add more instances behind a load balancer and they share the work. Scale on request rate and response time. Plan for bursts, such as a company announcement that sends thousands of employees to the bot at once, and for the limits of the downstream services your bot calls, since a language model or a database can become the real bottleneck.
If each receptionist keeps a personal notebook, you cannot swap them mid-conversation. If all notes live in a shared filing room, any receptionist can pick up any conversation, so you can add more receptionists whenever the lobby gets crowded.
Keep the bot instances always warm for interactive scenarios. Cold starts add seconds to the first reply, and users abandon a bot that takes too long to answer a simple hello.
Configuration and secrets
Store settings such as endpoints and keys in application configuration, and store secrets in a managed secret vault, accessed by the bot’s managed identity. Never put secrets into source code or into files committed to a repository.
12State and Storage
What the bot remembers, and where it keeps those memories.
Conversation state
Facts tied to one conversation, such as the current step of a dialog or items in a shopping basket for this chat.
User state
Facts tied to a person across conversations, such as their name, language and preferences on a channel.
Private conversation state
Facts for one user within one group conversation, so that two people in the same channel do not overwrite each other.
| Storage choice | Strength | Limitation |
|---|---|---|
| In-memory | Fastest and simplest for local tests | Lost on restart and not shared between instances, so unsuitable for production |
| Azure Blob Storage | Cheap and simple for small state objects | Higher latency for chatty use |
| Azure Cosmos DB | Fast, scalable, supports time-to-live for expiring old conversations | Higher cost, needs partition design |
| Other databases and caches | Fit existing standards or speed requirements | Needs custom adapter code |
Using in-memory state in production. It works in testing with one instance and then fails mysteriously when the service scales out or restarts, and users find that the bot suddenly forgets what they said a moment ago.
Keep state small and temporary
Store identifiers and small values rather than entire documents. Set expiry so that abandoned conversations are cleaned up. State may contain personal information, so treat the store as sensitive data: encrypt it, restrict access, and follow your data retention rules.
State is the sticky note on each guest’s file: “wants a quiet room, third floor”. It should say only what is needed for the next step, and the note should be shredded when the guest leaves.
13Proactive Messaging: When the Bot Speaks First
Not every message begins with a user. Reminders, alerts and long-running results need the bot to start the exchange.
To send a message without a user’s request, the bot needs a conversation reference: a small record saved earlier that identifies the user, the conversation and the channel. Later, when an event occurs, such as an approval completing or a deadline approaching, a separate process loads the reference and asks the connector to continue that conversation and deliver a message.
flowchart LR
A["User starts chat"] --> B["Bot saves conversation reference"]
C["Business event such as approval completed"] --> D["Trigger endpoint or scheduled job"]
D --> E["Load saved reference"]
E --> F["Continue conversation through connector"]
F --> G["User receives notification"]
Notifications
Alerts about system incidents, workflow approvals and status changes delivered into chat.
Reminders
Deadline nudges and scheduled check-ins that the user asked for.
Long-running work
Results of tasks that take minutes or hours, such as report generation, delivered when ready.
Flooding users with unsolicited messages. Channels may restrict or penalize this, and people mute or block noisy bots. Let users choose what they get, keep messages short and relevant, and respect each platform’s rules about when proactive messages are allowed.
A saved conversation reference is like keeping a guest’s room number. When the package they were waiting for arrives, the front desk can ring their room directly instead of waiting for them to come down and ask.
14Direct Line, Web Chat and Private Networking
Embedding a bot in your own website or app, and keeping the traffic private when needed.
Direct Line
Direct Line is a secure interface that lets your own client applications, such as a website, mobile app or kiosk, talk to the bot without going through a third-party messaging platform. The client uses a secret or a short-lived token to open a conversation and then sends and receives activities. Web Chat is a ready-made open-source chat component that uses Direct Line, which you can embed in a page and customize with your own styling.
Enable the channel
Add the Direct Line channel to the Azure Bot resource and generate a secret.
Exchange the secret for a token on your server
Your web server holds the secret and gives the browser a short-lived token for each visitor. The secret never reaches the browser.
Render the chat component
The page loads Web Chat with the token and displays the conversation.
Restrict trusted origins
Limit which websites may use your bot to reduce misuse.
Placing the Direct Line secret directly in website code. Anyone viewing the page source could copy it and use your bot at will, creating cost and abuse. Always exchange the secret for tokens on your own server.
Private networking
By default, the connector reaches your bot over the public internet using a secured web address. Organizations that must keep bot traffic private can use network isolation features, including a Direct Line extension that runs inside your own virtual network and private endpoints on supporting services such as storage, search and language models. This is a design area with several moving parts, such as DNS and firewall rules, and its support details have changed over time, so verify the current guidance carefully before committing.
flowchart LR
A["Browser inside corporate network"] --> B["Private web chat page"]
B --> C["Direct Line extension in virtual network"]
C --> D["Bot application in virtual network"]
D --> E["Private endpoints to data and AI services"]
A public bot is a reception desk on a street corner. A privately networked bot is a reception desk inside a secure office floor: only people already inside the building can walk up to it, and its calls go through internal lines.
15Bots in Microsoft Teams
For workplace assistants, Teams is often the most important channel.
Teams gives bots an audience that is already signed in, with a rich identity and strong tools for notifications and collaboration. Bots can talk one to one with a person, join a group chat, or live in a channel where they respond when mentioned. They can send cards with buttons and forms, and they can power other Teams features such as message actions and link previews.
Personal
A private one-to-one chat with the user, good for assistants and notifications.
Group chat
A bot present in a multi-person chat, where it should respond only when addressed to avoid noise.
Team channel
A bot in a shared channel that responds to mentions and posts updates for the whole team.
App manifest and packaging
A package describing the bot, its permissions and its scopes, uploaded to Teams for testing and approved by administrators for wider use.
Single sign-on
Lets the bot obtain tokens for the current user without extra sign-in prompts, if configured correctly.
Admin approval and policies
Organizations control which apps are allowed, so plan the approval path early.
A Teams bot is a colleague in the office: you can ask them privately, include them in a group discussion, or call them out in the shared channel. Good colleagues speak when spoken to and do not interrupt every conversation.
Test in each scope you intend to support. Behavior differs between personal chats, group chats and channels, especially for mentions, permissions and the information the bot receives about members.
Microsoft’s agent and Copilot extensibility story for Teams and Microsoft 365 is changing quickly. Check the latest documentation on how bots, agents and copilot extensions relate before choosing your development approach for new workplace assistants.
16Security, Privacy and Compliance
A bot is an open door into your systems that anyone on a channel can knock on.
Authenticate every request
Validate the token on every incoming activity. Never accept unsigned or unverified calls to the messaging endpoint.
Use least privilege for downstream access
Give the bot’s identity only the permissions it needs, and act with the user’s own permissions where possible.
Protect secrets
Prefer managed identities, keep other secrets in a vault, rotate them, and never log them.
Validate and sanitize user input
Treat messages as untrusted data. Guard against injection into database queries, commands and AI prompts.
Minimize and protect personal data
Collect only what you need, mask sensitive values in logs and transcripts, set retention limits and honor deletion requests.
Limit abuse
Apply rate limits, detect unusual patterns and cap expensive operations such as calls to language models, so that one user cannot run up a large bill.
Prepare for incidents
Have a way to disable the bot quickly, rotate credentials, and notify users if something goes wrong.
A bot is a receptionist with a master phone directory. You check the badges of callers, give the receptionist keys only to the rooms they need, and never leave the safe’s combination on a sticky note.
Problem
Conversation transcripts, including users’ personal details and passwords typed by mistake, are stored forever in an open storage account.
Why It’s Harmful
A single leak exposes years of private conversations, and the organization breaks privacy obligations through excessive retention.
Correct Approach
Mask sensitive data before storing, restrict access, encrypt the store, define a retention period and delete automatically after it.
Warn users not to share passwords or highly sensitive data in chat, and design the bot to detect and discard obvious secrets such as card numbers in messages rather than storing them.
17Testing, Deployment and DevOps
Bots need the same discipline as any production software, plus conversation-specific testing.
Levels of testing
Unit tests
Test dialogs and handlers by simulating incoming activities and checking replies, without a real channel.
Local interactive testing
Use a desktop emulator or a development tunnel to chat with the bot running on your machine, and inspect the raw activities.
Channel testing
Test in the real channels, because rendering, permissions and limits differ from the emulator.
Language and answer quality tests
Run a set of sample utterances and questions through the language and knowledge components, tracking accuracy over time.
Load and resilience tests
Simulate many simultaneous conversations and failures of dependencies such as the database or language service.
flowchart LR
A["Code change in repository"] --> B["Automated build and unit tests"]
B --> C["Deploy to test environment"]
C --> D["Conversation and quality tests"]
D --> E["Approval"]
E --> F["Deploy to staging slot"]
F --> G["Swap into production"]
Environments
Use separate Azure Bot resources, identities and hosting for development, test and production, so that test conversations never touch real users or real data. Keep configuration outside the code so the same build can run in each environment. Deployment slots allow you to warm up a new version and then swap it into place with little disruption and an easy rollback.
Releasing a bot is like opening a new restaurant menu. You taste it in the kitchen, serve it to a test table, and only then print it for all the guests, with the old menu kept handy in case the new dishes go wrong.
Pointing the production channel registration at a developer’s temporary tunnel address “just for a quick demo” and forgetting to switch it back. The bot then goes offline whenever that laptop sleeps. Keep production endpoints stable and separate from experiments.
18Monitoring, Analytics and Improvement
A bot is never finished. Real conversations show what to fix.
Technical health
Request rate, response time, error rate and dependency failures, collected in Application Insights and Azure Monitor.
Containment rate
The share of conversations the bot resolves without a human, a key measure of value.
Fallback and low-confidence rate
How often the bot did not understand, pointing to missing intents or documents.
Abandonment and drop-off
Where in a dialog people give up, showing confusing questions or broken steps.
User satisfaction
Thumbs up or down and short surveys, tied to conversations for review.
Cost per conversation
Compute, language model and search costs divided by conversations, to keep spending in check.
Log with correlation
Tag each log entry with the conversation and turn so you can follow one user’s journey across services.
Review a sample weekly
Read real conversations, with privacy protections, to see where the bot struggles.
Fix the biggest gaps first
Add intents, improve documents, rewrite confusing prompts and simplify long flows.
Re-test and release
Run the quality suite, deploy through the pipeline, and watch the metrics for improvement.
Set alerts on error rate and on dependency failures, not just on server availability. A bot can be “up” while every language request fails and every user sees an apology.
19Troubleshooting Common Problems
Most failures fall into a short list of causes.
| Symptom | Likely cause | What to do |
|---|---|---|
| Bot does not respond at all | Wrong messaging endpoint, app not running, or identity mismatch | Check the endpoint address, app health and the configured identity |
| Unauthorized errors on incoming requests | Identity or tenant settings do not match the bot registration, or an expired secret | Verify identity settings and credentials, rotate if expired |
| Works in the emulator but not in a channel | Emulator skips authentication that channels enforce | Test with the portal web chat and the real channel, and check app credentials |
| Bot forgets context | In-memory state, scaling to several instances, or a state save missing | Use shared storage and make sure state is saved at the end of each turn |
| Duplicate replies | Request retries after slow responses, or duplicated handlers | Respond quickly, make operations idempotent and check handler registration |
| Slow replies | Cold starts, slow downstream calls or language model latency | Keep instances warm, add timeouts and show typing indicators |
| Sign-in loop or failure | OAuth connection or redirect settings incorrect | Test the connection in the portal and review application permissions |
| Cards display incorrectly | Channel does not support some card features | Test card versions per channel and provide a text fallback |
| Web chat cannot connect | Token problem, blocked origin or secret exposed incorrectly | Check token generation, trusted origins and network rules |
Troubleshoot from the outside in: is the request reaching the endpoint, is it authenticating, is the code running, is state loading, are dependencies answering. Application logs with a conversation identifier make each step visible.
Problem
The bot swallows exceptions and sends a friendly generic message with no logging.
Why It’s Harmful
Failures happen silently and nobody knows. Users keep getting poor service while the dashboards show everything as fine.
Correct Approach
Log every error with context, show a helpful message to the user, and alert the team when error rates rise.
20Cost and Capacity Planning
A bot’s bill is mostly the services behind it.
The Azure Bot resource itself has historically been inexpensive, with free and standard tiers and rules about which channels are free and which are metered. Pricing models change, so check the current pricing page. In practice, the larger costs are the compute that hosts the bot, the state store, the language and speech services, the search service, and above all the generative AI model calls, which are usually billed by the amount of text processed.
Hosting
The web app or container instances, sized for peak load and kept warm for fast replies.
State storage
Database or storage charges, which grow with the number of active conversations and retention period.
Language, speech and translation
Billed per request or per amount of text or audio processed.
Search and knowledge storage
The search service tier and index size, plus any document processing.
Language model usage
Billed by input and output text. Long prompts and long conversations raise the cost of every turn.
Monitoring and logs
Log ingestion and retention, which can grow quickly if every message is logged in full.
Answer simple questions without a model
Use buttons, rules and cached answers for frequent simple requests, and reserve the language model for open questions.
Control prompt and history size
Send only the relevant passages and a short conversation summary instead of the whole history every time.
Choose the right model size
Smaller, cheaper models often suffice for classification and routing, while larger ones are kept for hard generation tasks.
Rate limit and set budgets
Limit requests per user and set cost alerts so that abuse or a loop cannot create a huge bill overnight.
Right-size and review regularly
Use real usage data to adjust hosting tiers, search tiers and retention periods.
The receptionist’s salary is small compared with the cost of the building and the specialists they call. Likewise, the Bot Service resource is a small part of the bill, while the AI and data services behind it do the expensive work.
21Planning for Change: Migration and Modernization
If you have an existing bot, or are starting a new one, plan for the fast-moving ecosystem.
Inventory what you have
List every bot, the SDK and version it uses, the language and knowledge services it calls, its channels, its identity type and its owner.
Check support and retirement dates
Compare each dependency against Microsoft’s current retirement and support announcements, and flag the ones already retired or ending.
Decide the target approach
Choose between keeping a stable bot with maintenance only, migrating to a newer SDK, moving language features to current services, or rebuilding in a low-code agent platform when appropriate.
Separate logic from framework
Keep business logic in your own modules, with a thin layer that talks to the SDK, so that changing the framework later touches little code.
Migrate in steps with tests
Move one capability at a time, running your conversation test suite after each step to prove that behavior is unchanged.
If your hotel’s phone system is being discontinued, you do not tear down the hotel. You list the phones, check which are outdated, choose a new system, and replace it floor by floor, keeping the front desk open the whole time.
Ignoring retirement notices until a service stops working. Subscribe to service health and update announcements for your Azure subscriptions, and put retirement dates on your team’s calendar.
Not every conversational need requires custom code. For straightforward scenarios, a low-code agent builder may deliver faster with less maintenance. Reserve code-first bots for cases needing custom logic, deep integration or special control.
22Patterns and Anti-Patterns
Practices that scale, and mistakes worth avoiding.
Patterns worth adopting
Thin bot, fat services
Keep the bot as a conversation layer and put business logic into separate services or APIs that other applications can reuse.
Orchestrator with specialist skills
A front-door bot routes requests to specialized components, such as HR, IT and finance, each owned by a different team and released independently.
Hybrid scripted and generative design
Use scripted dialogs for transactions and generative answers for open questions, with clear rules for when each applies.
Human handoff with context
Pass the transcript summary and gathered details to a human agent so users never need to repeat themselves.
Idempotent actions
Design operations so that a retried request does not create duplicates, because channels and networks sometimes resend messages.
Evaluation suite as a safety net
Keep a growing test set of real questions and expected behaviors that runs in the pipeline on every change.
Anti-patterns to avoid
Problem
All business logic is written directly inside the bot’s message handler, mixed with channel-specific formatting.
Why It’s Harmful
The code is hard to test, hard to reuse in another channel and expensive to migrate when the framework changes.
Correct Approach
Move logic into separate, testable services and keep the handler thin.
Problem
The bot calls a slow external system synchronously on every turn without timeouts or fallbacks.
Why It’s Harmful
One slow dependency freezes all conversations, channels time out and retries pile up, which can cascade into a full outage.
Correct Approach
Set timeouts, handle failures gracefully, use asynchronous patterns with proactive replies for long work, and add circuit breakers.
Problem
A generative bot is launched without a test set, safety filters or source restrictions because “the model is smart”.
Why It’s Harmful
Wrong, unsafe or confidential answers reach users, damaging trust and possibly breaking regulations.
Correct Approach
Ground answers in approved sources, apply safety filtering and access control, evaluate before launch, and monitor after.
23Best Practices Checklist
A review list before launch and at every major change.
Do
- Confirm the currently recommended SDK, services and identity types before starting
- Use managed identities and keep secrets in a vault
- Store state in shared, durable storage with expiry
- Validate incoming requests and sanitize all user input
- Design for the least capable channel and adapt per channel
- Provide greetings, examples, graceful fallbacks and human handoff
- Ground generative answers in permitted sources with safety filtering
- Separate environments and use a deployment pipeline with slots
- Log with conversation correlation and review real conversations
- Set rate limits, budgets and cost alerts
Avoid
- In-memory state in production
- Putting secrets in code or exposing the Direct Line secret in a browser
- Using a single powerful service account for all users
- Silent error handling with no logging
- Long synchronous calls inside a turn
- Overlapping intents and untested language models
- Storing raw transcripts forever
- Building on retired services without a migration plan
- Launching without a quality test set
Create a one-page launch checklist for every bot covering identity, channels, state, security review, privacy review, test results, monitoring, alerts, cost limits and support ownership. A bot without a named owner becomes an orphan the day its creator moves teams.
24Real-World Scenarios
How organizations apply these ideas.
Employee help desk in Teams
A company builds an assistant that answers questions about leave, benefits and equipment, resets passwords after verifying identity through single sign-on, and creates support tickets when it cannot solve a problem. Answers come from approved policy documents filtered by the employee’s access.
Retail customer service on the website
A retailer embeds a web chat that tracks orders, starts returns and recommends products. Scripted dialogs handle returns and payments, while a grounded generative component answers product questions. Complex complaints pass to a human with a summary.
Bank assistant with strict controls
A bank runs its bot inside a private network, with authenticated users only, strict content filtering, masked transcripts, short retention and extensive audit logging. Sensitive actions require confirmation and a second factor.
Operations notifications
An operations team uses a bot to post incident alerts and approval requests into chat, with buttons to acknowledge or approve. Proactive messaging delivers updates, and each action is logged for audit.
Healthcare appointment scheduling
A clinic bot books and reschedules appointments by text and web chat, uses speech for accessibility, and hands over to staff for any medical question, since it is designed never to give medical advice.
Multi-team enterprise orchestrator
A large organization builds one front-door assistant that routes to separate HR, IT and finance skills. Each team owns and releases its skill independently, while the front door provides a consistent experience and authentication.
25Frequently Asked Questions
Short answers to common beginner questions.
It registers your bot, gives it an identity and connects it to channels such as Teams and web chat. It carries messages between channels and your bot’s code, which you host separately, for example on Azure App Service.
On any web hosting you choose, such as App Service, containers, Kubernetes or functions. The Azure Bot resource points to it with a messaging endpoint.
A small structured message in a common format that describes an event, such as a user message, someone joining a chat, or a button press. Using one format lets the same bot work across many channels.
Usually because state is stored in memory or not saved at the end of the turn. Use shared durable storage, and save state each turn.
Check Microsoft’s current recommendation. The long-standing Bot Framework SDK has been moving out of active support, and newer SDKs and low-code agent tools are being promoted. Pick based on current guidance and your requirements.
The channel provides basic identifiers, and for secure access to other services the bot uses a sign-in feature with OAuth, or single sign-on in Teams, to obtain the user’s token with their consent.
Yes, with care. Ground answers in approved documents, filter retrieval by user permissions, apply content safety filters, defend against prompt injection, and test answer quality continuously.
Enable the Direct Line channel, generate tokens on your server and display a web chat component in your page. Never expose the Direct Line secret in browser code.
Yes, through proactive messaging with a saved conversation reference. Respect channel rules and user preferences to avoid being seen as spam.
Use network isolation features such as a Direct Line extension inside your virtual network and private endpoints for supporting services. The details have changed over time, so verify current guidance before designing.
The Azure Bot resource itself is a small part. Most cost comes from hosting, storage, language, search and, especially, generative AI model usage. Check current pricing and set budgets and alerts.
Inventory what you use, compare against Microsoft’s retirement announcements, and plan a stepwise migration to supported services, keeping business logic separate from framework code to make change easier.
The concepts are stable, but SDKs, supported services, identity options, channels and pricing change. Always confirm details in current official documentation before finalizing a design.
26Summary and Key Takeaways
Azure Bot Service is the switchboard that connects channels to your bot’s code. Messages travel as activities through the connector, are authenticated with signed tokens, and are handled by your hosted application, which remembers context in shared storage, understands language through language services or generative AI, and acts on business systems with careful permissions. Good bots are designed around conversations, not just technology: they greet, guide, admit what they do not know, and hand over to humans. They are secured, monitored, tested and costed like any production system. Because Microsoft’s bot tools and related services are evolving, the wise approach is to hold on to the stable ideas in this guide, keep business logic separate from framework code, and verify current support and pricing before every major decision. With those habits, you can build assistants that people actually want to use.
Key Takeaways
- Bot Service is the switchboard, not the brain — it registers the bot, handles identity and connects channels, while your hosted code and the services it calls do the thinking.
- Everything is an activity — a common message format lets one bot serve many channels, but you should still design for each channel’s limits.
- State must be shared and durable — the web server forgets between turns, so use proper storage with expiry and protect the data it holds.
- Combine scripted, intent-based and generative approaches — use scripts for transactions, language understanding for routing and grounded generative AI for open questions, with guardrails.
- Identity has two sides — the bot proves itself with a managed identity or registration, and acts for users with their own delegated permissions.
- Operate it like production software — separate environments, pipelines, monitoring, conversation analytics, rate limits and cost alerts.
- Plan for change — inventory dependencies, watch retirement notices, keep logic separate from the framework and verify current guidance before committing.