Cognitive Services in Azure: Ready-Made AI You Can Plug Into Any App

Cognitive Services in Azure: Ready-Made AI You Can Plug Into Any App

A beginner-friendly guide to Azure Cognitive Services, now called Azure AI Services, the collection of pre-built artificial intelligence services that can see, listen, speak, read, translate and understand, explained step by step with everyday analogies.

Imagine you want to run a small restaurant. You could grow your own wheat, mill the flour and bake every loaf yourself. Or you could buy fresh bread from a good bakery and spend your time on what makes your restaurant special. Building artificial intelligence from scratch is like growing your own wheat: it needs rare skills, huge amounts of data and powerful computers. Azure AI Services is the bakery. You walk in, ask for what you need, and walk out with a ready result.

In this guide we begin with the simple question “what do we mean by AI in an app?”, and then explore the main families of services, how to call them, how to keep them safe and how to control their cost. No earlier knowledge of artificial intelligence is needed. Every new word is explained when it first appears.

1What Are Cognitive Services?

They are Microsoft’s ready-to-use AI abilities, offered over the internet through simple requests.

The word cognitive relates to thinking and understanding. Cognitive Services give your applications human-like abilities. They can look at a photograph and describe what is in it, listen to speech and write it down, read text and judge whether it sounds happy or angry, translate between languages, and pull names and amounts out of a scanned invoice.

Each ability is offered as a cloud service. Your application sends some data, such as a picture or a sentence, to a web address. The service runs a trained artificial intelligence model that Microsoft has already built and tested, and sends back the answer. You do not need to collect millions of examples, train models or buy special computers. You pay only for what you use.

i
Important Naming Note

Microsoft has renamed this family. What was called Azure Cognitive Services is now called Azure AI Services, and it sits within the wider Azure AI Foundry platform. Articles, certificates and many people still use the older name, so you will see both. They refer to the same family, with new additions, such as generative AI services, joining over time.

Simple Analogy

Think of a city with many specialist helpers you can hire by the hour: an interpreter, a stenographer, a reader of handwriting, a photo describer. You do not train them or pay their monthly salary. You call whichever one you need, give them the task, and pay for that task only.

Words you will meet again and again

Term 1

Model

A trained program that has learned patterns from many examples and can make predictions on new data.

Term 2

Endpoint

The web address that your application calls to use a service.

Term 3

Key

A secret code that proves your application is allowed to use the service.

Term 4

Pre-built Model

A model made by Microsoft that you use as it is, with no training from you.

Term 5

Custom Model

A model you teach with your own examples, to handle your special cases.

Term 6

Responsible AI

The set of principles and controls that aim to make AI fair, safe, private and accountable.

No training
pre-built models work from day one
REST + SDKs
simple calls from almost any language
Pay per use
billed per transaction or per unit processed

2Why Do These Services Exist?

Because building good AI from scratch is slow, expensive and needs rare expertise.

To teach a computer to recognise objects in photographs, you need millions of labelled pictures, powerful graphics processors, specialist data scientists and months of experimentation. After that, you must host the model, keep it fast, secure it and update it as the world changes. Most companies have neither the data nor the skills, and even those that do would rather spend their time on their own products.

Cognitive Services remove this barrier. Microsoft has already trained the models on huge amounts of data, tested them, and wrapped them in simple web interfaces. A web developer with no background in artificial intelligence can add speech recognition to an app in an afternoon.

Speed

Add capabilities in hours instead of months, because the hard training work has already been done.

Accessibility of skills

Ordinary developers can use them through simple calls, without needing a team of data scientists.

Scale and reliability

Microsoft runs the machines and scales them. Your app handles ten requests or ten million with the same code.

Continuous improvement

Models are improved and updated by Microsoft, so your application benefits without extra work, within version rules you can control.

Predictable cost

You pay per use instead of buying expensive hardware that sits idle.

3A Map of the Family

The services fall into a few clear families, each linked to a human sense or skill.

Family 1

Vision

Understand images and video: describe pictures, read printed and handwritten text, detect objects and analyse faces under strict rules.

Family 2

Speech

Turn speech into text, text into natural-sounding speech, and translate spoken words between languages.

Family 3

Language

Understand written text: sentiment, key phrases, names, summaries, questions and answers, and personal data detection.

Family 4

Translator

Translate text and whole documents between many languages.

Family 5

Document Intelligence

Extract text, tables and fields from forms, invoices, receipts and identity documents.

Family 6

Generative AI and Safety

Azure OpenAI models that write and reason, plus Content Safety to detect harmful material.

Family 7

AI Search

Search across your own content, with built-in AI enrichment that extracts meaning from files while they are indexed.

flowchart TD
  APP["Your Application"] --> V["Vision"]
  APP --> S["Speech"]
  APP --> L["Language"]
  APP --> T["Translator"]
  APP --> D["Document Intelligence"]
  APP --> G["Generative AI and Content Safety"]
  APP --> AS["AI Search"]
        
FIG 1 — One application can call many AI service families
Simple Analogy

A hospital has an eye department, an ear department, a records office and a translation desk. A patient does not need to know how each works. Reception directs the request to the right department. The AI service families are those departments.

!
Roadmap Warning

The catalogue changes over time. Some older services have been retired or replaced, for example the old Content Moderator was succeeded by Content Safety. Always check the current list and retirement dates before choosing a service for a long-lived project.

4Vision: Teaching Apps to See

Vision services let software understand pictures and video the way a person would glance at them.

A computer sees a photograph as a huge grid of numbers describing colours. Vision services use trained models to turn those numbers into meaning, such as “a dog sitting on a sofa” or “the word SALE written on a sign”. The services handle many everyday needs.

Image analysis

Describes what is in a picture, produces tags such as “outdoor” and “tree”, detects objects and people, and can generate a short caption.

Text reading, or OCR

Optical character recognition reads printed and handwritten text from photographs and scans, turning pictures of words into real text a computer can search.

Face service

Detects faces and can compare them. Because faces are sensitive biometric data, some abilities are available only to approved customers under strict responsible AI rules.

Video and spatial analysis

Understands movement and presence in camera feeds, such as counting people entering a store, with privacy controls.

Custom image models

Where the common model does not know your products, you can train a model with your own labelled pictures. Check the current status of the custom vision offerings, since Microsoft guides customers toward newer options over time.

Simple Analogy

Vision services are like a very fast assistant who looks at every photo you hand over and writes a note: what is in it, what text it contains and where. You never have to teach the assistant what a tree looks like.

i
Picture Quality Matters

Blurry, dark or tilted images give worse results. If you control how pictures are taken, such as scanning documents or photographing products, good lighting and sharp focus improve accuracy more than any tuning.

!
Common Mistake

Treating a confidence score as a fact. Each result comes with a score showing how sure the model is. Low scores should send the item to a person for review instead of being trusted blindly.

5Speech: Teaching Apps to Listen and Talk

Speech services connect the spoken word with written text, in both directions.

Ability 1

Speech to Text

Transcribes spoken audio into written words, either live from a microphone or from recordings. Used for captions, meeting notes and call analysis.

Ability 2

Text to Speech

Turns written text into natural-sounding speech in many voices and languages, useful for assistants, audiobooks and accessibility.

Ability 3

Speech Translation

Listens in one language and produces text or speech in another, almost as the person is speaking.

Ability 4

Speaker Recognition

Works out who is speaking, or separates different speakers in a recording, a feature often called diarisation.

Batch and real-time modes

Real-time transcription listens as the person talks and returns words within a fraction of a second, which suits live captions and voice assistants. Batch transcription processes a large pile of recordings in the background, which suits archives of customer calls and is usually cheaper per hour of audio.

Custom speech

If your business uses unusual words, such as medicine names or product codes, you can adapt the recogniser with examples so that it hears them correctly.

Custom voices

Organisations can create a branded voice for their assistant. Because this could be misused to imitate real people, access is controlled and requires approval and consent from the voice owner.

Simple Analogy

Speech to text is a court stenographer who types every word you say. Text to speech is a news reader who reads any article aloud in a pleasant voice. Speech translation is an interpreter who sits between two people with different languages.

!
Common Mistake

Testing only with clean studio recordings. Real calls have noise, accents, interruptions and poor microphones. Test with realistic audio from your own users before you promise accuracy to the business.

6Language: Teaching Apps to Understand Text

Language services read text and extract meaning, structure and mood.

Ability 1

Sentiment Analysis

Judges whether text sounds positive, negative or neutral, sentence by sentence. Great for reviews and feedback.

Ability 2

Key Phrase Extraction

Picks out the main talking points of a text, so you can see themes across thousands of comments.

Ability 3

Named Entity Recognition

Finds names of people, places, companies, dates and amounts inside free text.

Ability 4

PII Detection

Spots personal data, such as phone numbers and identity numbers, and can mask it before the text is stored or shared.

Ability 5

Summarisation

Shortens long documents and conversations into brief summaries.

Ability 6

Language Detection

Identifies which language a piece of text is written in.

Understanding what people ask

Conversational language understanding

Works out what a user wants from a sentence like “book a table for four tomorrow at eight”. It finds the intent, which is booking a table, and the details, such as the number of people and the time. You can train it with your own examples.

Question answering

Builds a knowledge base from your documents and frequently asked questions, so that a chatbot can answer customers in plain language.

Custom text classification

Sorts documents into categories you define, such as “complaint”, “refund request” or “praise”.

Simple Analogy

A language service is a tireless reader who goes through every email and review, underlining names in blue, key points in yellow, angry sentences in red, and writing a one-line summary at the top.

i
Combine for Insight

The real power comes from combining abilities. For example, detect the language, translate to your own language, find the sentiment and extract key phrases, giving a dashboard of what customers around the world are saying.

7Document Intelligence: Reading Forms and Invoices

Businesses run on paper and PDFs, and this service turns them into usable data.

Document Intelligence, formerly known as Form Recognizer, goes beyond simple text reading. It understands the layout of a document. It finds tables, headings, checkboxes and key-value pairs, such as “Invoice Number” next to its value, and returns them as organised data. This removes the need for people to retype information from documents into systems.

Model 1

Read and Layout

Extracts all text and the structure of the page, including tables and reading order.

Model 2

Pre-built Document Models

Ready models for common documents such as invoices, receipts, business cards and identity documents.

Model 3

Custom Models

Trained on your own forms using a few labelled samples, for documents unique to your business.

Model 4

General Document Model

Pulls out key-value pairs and entities from varied documents without any training.

1

Document arrives

A scanned invoice or PDF is uploaded to storage or sent directly to the service.

2

Analysis

The service reads the text, finds the layout and applies the chosen model.

3

Structured result

Fields such as supplier, date, total and line items come back as data, each with a confidence score.

4

Validation

High-confidence results flow straight into the finance system, while low-confidence ones go to a person for a quick check.

Simple Analogy

A data-entry clerk looks at an invoice, finds the invoice number, the date and the total, and types them into the accounting system. Document Intelligence is a clerk who does this in seconds, for thousands of pages, and tells you honestly when they are unsure.

!
Common Mistake

Automating payments directly from extracted amounts without any validation. A smudged digit could turn a small bill into a large one. Always add checks, such as comparing with purchase orders and routing low-confidence fields to a person.

8Translator: Breaking Language Barriers

Translator converts text and documents between a large number of languages.

The Translator service supports more than a hundred languages and dialects. Applications can translate short pieces of text, such as chat messages and product descriptions, in real time, and can translate whole documents while preserving their layout and formatting.

Text translation

Instant translation of messages, with language detection so users do not have to say which language they are using.

Document translation

Translates entire files in storage, such as manuals and contracts, keeping tables and headings in place.

Custom translation

Teaches the translator your own terms, such as product names that must never be translated, or industry words that need a specific form.

Transliteration

Converts text from one script to another, such as writing a name in Latin letters instead of its original script, without changing the language.

Simple Analogy

Translator is an interpreter available at every desk of your company. A support agent in one country can read a customer’s message from another country, answer in their own language, and the customer receives a reply in theirs.

i
Glossaries Help

Create a glossary for brand names, legal terms and technical vocabulary. Machine translation is very good at everyday language but can stumble on special terms, and a glossary keeps them consistent.

!
Important Limit

Machine translation can make mistakes, especially in legal, medical and safety-critical text. Use human review where errors could cause harm or legal trouble.

9Generative AI and Content Safety

The newest members of the family can write, summarise and reason, and they come with safety tools.

Generative AI models create new content, such as text, code or images, in response to instructions called prompts. Azure OpenAI Service makes powerful models from OpenAI available inside Azure, with Azure’s security, networking and compliance features. Companies use it to build assistants, summarise documents, draft replies and answer questions about their own data.

Chat and reasoning

Build assistants that understand questions and respond in natural language.

Embeddings

Turn text into lists of numbers that capture meaning, so that a search can find content with a similar idea even if it uses different words.

Grounding with your data

Combine the model with a search over your own documents, so answers come from your content and not just from the model’s general knowledge. This approach is often called retrieval augmented generation.

Azure AI Content Safety

Content Safety is a service that checks text and images for harmful material, such as hate, violence, self-harm and sexual content, and gives each a severity level. It replaces the older Content Moderator service. It also includes protections against attempts to trick language models into ignoring their instructions, and groundedness checks that help detect answers not supported by the source documents.

Simple Analogy

A generative model is a gifted writer who can draft almost anything in seconds. Content Safety is the editor who reads the draft before publication, rejecting anything harmful or untrustworthy.

!
Common Mistake

Treating generated answers as always true. Generative models can produce confident but wrong statements. Ground them in trusted documents, show sources, apply safety filters and keep humans in the loop for important decisions.

10Inside the Machine: How a Call Works

Understanding the journey of one request makes everything else easier to learn.

To use a service, you first create an Azure resource of that type. The resource lives in a chosen region, and Azure gives it an endpoint, which is its web address, and two keys. Your application sends an HTTPS request to the endpoint, along with credentials and the data to analyse. The service checks the credentials and the quota, runs the model and returns the answer, usually as JSON, a simple text format with labelled fields.

1

Create the resource

Choose the service type, region and pricing tier. Azure creates the endpoint and keys.

2

Application prepares a request

It packages the input, such as an image, audio or text, and chooses which operation to call.

3

Authentication

The service verifies a key or, better, an identity token issued by Microsoft Entra ID.

4

Model runs

The service applies the trained model to the input and calculates the result with confidence scores.

5

Response returned

Your application receives the structured answer and decides what to do with it.

sequenceDiagram
  participant A as Your Application
  participant E as Microsoft Entra ID
  participant S as AI Service Endpoint
  participant M as Trained Model
  A->>E: Request access token
  E-->>A: Token issued
  A->>S: Send data with token
  S->>S: Check identity and quota
  S->>M: Run analysis
  M-->>S: Result with confidence
  S-->>A: Structured response
        
FIG 2 — Sequence of a secure call to an AI service
Simple Analogy

Using an AI service is like posting a letter to an expert. You write your question, put it in an envelope with your identity card, and mail it to the expert’s address. The expert checks your card, thinks, and mails back a reply with a note saying how sure they are.

Synchronous and asynchronous calls

Quick operations, such as analysing a short sentence, return the answer in the same call. Long operations, such as analysing a long document or transcribing an hour of audio, work in two steps. You submit the job and receive a ticket, then you check back or receive a notification when the result is ready. Good applications handle both styles.

11Resource Types and Pricing Tiers

Choosing how to create resources affects both management effort and billing clarity.

Type 1

Multi-Service Resource

One resource, one endpoint and one key that work for several services, such as vision, language and speech. Simple to start with, and billing is combined.

Type 2

Single-Service Resource

One resource for one service. It gives separate keys, separate billing and finer control, and often the full set of settings of that service.

QuestionMulti-ServiceSingle-Service
Setup effortLow, one resource for many abilitiesOne resource per service
KeysOne shared keySeparate keys per service
Cost trackingCombined on one lineClear per-service breakdown
Access controlBroad, covers everythingTighter, per service
Best forPrototypes and small appsProduction systems and larger teams

Pricing tiers

Free tier

A small monthly allowance of transactions at no charge, useful for learning and testing. It has low limits and is not meant for production.

Standard pay-as-you-go

You are charged per transaction, per thousand characters, per hour of audio or per page, depending on the service. Cost grows with use.

Commitment tiers

For steady, high volume, you commit to a monthly amount in return for a lower unit price. Good when usage is predictable.

Simple Analogy

A multi-service resource is a family phone plan with one bill. Single-service resources are separate plans for each person, so you see exactly who used how much and can limit each one separately.

i
Good Habit

Start with a multi-service resource for experiments, but move to single-service resources for production so that keys, permissions, networking and costs can be managed per capability.

12Pre-Built or Custom: Which Model to Use?

The first decision for any feature is whether a ready model is good enough.

Pre-built models are trained by Microsoft on broad data and work immediately. They are perfect for common needs such as reading text, detecting language or transcribing speech. Custom models are trained or adapted with your own examples. They help when your content is special, such as your own product photos, your industry vocabulary or your unique forms.

QuestionPre-builtCustom
Time to startMinutesDays or weeks to collect and label examples
EffortVery lowNeeds good labelled data and testing
Accuracy on your special contentGood for general use, may miss special casesHigher where examples represent your reality
MaintenanceMicrosoft updates the modelYou retrain when your data changes
1

Try the pre-built model first

Run it on real samples and measure how often it is right.

2

Find the gaps

List the cases it gets wrong and decide whether the gaps matter to the business.

3

Customise only where needed

Train a custom model for the specific weak area, using clean, varied and correctly labelled examples.

4

Test on unseen data

Keep part of the data aside and measure on it, to avoid fooling yourself with results on the training examples.

5

Monitor over time

Re-check accuracy regularly, because the real world changes and models can become stale.

Simple Analogy

A ready-made suit fits most people well enough. A tailor-made suit fits you perfectly but takes time and money. Start with the ready-made one, and visit the tailor only if it truly does not fit.

!
Common Mistake

Training a custom model with too few or very similar examples. The model memorises them and fails on real data. Collect varied examples that reflect real conditions, including messy and unusual cases.

13Containers and the Edge

Sometimes the data cannot leave the building, and some services can come to you.

Certain Azure AI services are also offered as containers. A container is a packaged program that runs on any machine that supports containers, whether in your own data centre, at a remote site or on a device at the edge. The container does the work locally, and only usage information is sent to Azure for billing. Not every service has a container, and some require approval, so check the current list.

Data residency and privacy

Sensitive documents or recordings can be processed on-premises without leaving your network.

Low delay

Processing close to the source, such as a factory camera, avoids the round trip to the cloud.

Poor or no connectivity

Remote locations with weak links can run the container locally, with billing information sent later when a connection is available.

flowchart LR
  D["Local Camera or Microphone"] --> C["AI Service Container on Local Server"]
  C -->|"results used locally"| A["Local Application"]
  C -.->|"usage data for billing"| Z["Azure"]
        
FIG 3 — A container runs the model locally and reports only usage to Azure
Simple Analogy

Instead of posting every document to an expert in another city, you hire the expert to sit in your own office. The work stays in the building, and only a monthly invoice travels to the expert’s company.

i
Operations Note

Running containers means you manage the hosting, updates and scaling yourself. Weigh that effort against the benefits before choosing containers over the cloud service.

14Security: Protecting Keys, Data and Access

AI services often process sensitive content, so protection must be built in from day one.

1

Prefer identity over keys

Use Microsoft Entra ID and managed identities so applications authenticate without storing secrets. Where keys are used, keep them in Azure Key Vault and rotate them regularly.

2

Least privilege

Give each application only the role it needs, such as permission to use the service without permission to change its settings.

3

Network restrictions

Limit access using firewall rules, virtual network rules and private endpoints, so the service is reachable only from your own network.

4

Encryption

Data is encrypted in transit and at rest, and customer-managed keys are available for some services where policy requires them.

5

Data handling

Understand each service’s data policy, including retention of inputs, and whether data is used to improve the models. Microsoft documents these terms for each service.

6

Audit and alert

Send logs to a central place and alert on unusual usage, such as a sudden spike that could mean a stolen key.

Simple Analogy

A bank vault is protected by guards, ID checks, a locked door, cameras and a log of every visitor. Even if one layer fails, the others still protect what is inside. The same layered thinking applies to your AI resources.

!
Danger

Putting a service key inside a mobile app or a web page script. Anyone can extract it and run up your bill or misuse your resource. Keep keys on the server side, or use identity-based access, and place your own back-end between users and the AI service.

15Responsible AI: Using Power Wisely

AI systems affect real people, so fairness, safety and privacy are part of the design.

Microsoft describes its approach to responsible AI through a set of principles: fairness, reliability and safety, privacy and security, inclusiveness, transparency and accountability. Azure AI services carry these ideas into practice through documentation, built-in controls and, for the most sensitive abilities, access limits.

Principle 1

Fairness

Models should treat different groups of people fairly. Test your application with diverse data and look for uneven results.

Principle 2

Reliability and Safety

Systems should work as intended and fail safely. Plan what happens when the model is unsure or wrong.

Principle 3

Privacy and Security

Handle personal data with care, minimise what you send, and protect what you store.

Principle 4

Inclusiveness

Design for people of all abilities, languages and accents.

Principle 5

Transparency

Tell users when they are dealing with AI, and explain what it can and cannot do. Read the transparency notes published for each service.

Principle 6

Accountability

People remain responsible for decisions, so keep human oversight for important outcomes.

Limited access features

Some abilities, such as certain face recognition features and custom neural voices, require an application and approval, because misuse could harm people.

Content filters

Generative services include filters for harmful content that you can configure within allowed limits, and Content Safety adds extra protection.

Human in the loop

For decisions that affect people’s jobs, money, health or freedom, a person should review the AI’s suggestion before action is taken.

!
Important Reminder

Legal and ethical duties stay with the organisation that builds the application. Using a service from Azure does not remove the need to follow local laws about privacy, consent and automated decisions.

16Monitoring, Quotas and Throttling

Services have limits on how many requests you may send, and good applications respect them.

Every resource has a quota, which is a limit on requests per second or per minute, and sometimes on the size of each request. If your application sends too many requests too quickly, the service answers with a message that means “too many requests”, often shown as the code 429. This is called throttling, and it protects the service from overload.

Retry with back-off

When throttled, wait a short time and try again, waiting longer after each failure. Official software development kits include this behaviour.

Queue and smooth the load

Instead of sending a sudden flood, put work in a queue and send it at a steady rate that fits the quota.

Request higher limits

For legitimate growth, higher quotas can often be requested, or capacity can be reserved for predictable high volume.

Watch 1

Total Calls

The number of requests over time, to understand demand and cost.

Watch 2

Errors and Throttles

Failed and rejected requests, which show whether limits or bugs are causing trouble.

Watch 3

Latency

How long the service takes to answer, which matters for user-facing features.

Watch 4

Processed Units

Characters, pages, audio hours or images processed, which are the true drivers of the bill.

Simple Analogy

A popular restaurant can serve only so many orders per hour. If everyone shouts their orders at once, the kitchen asks people to wait. A polite customer waits a moment and orders again, instead of shouting louder.

i
Set Alerts

Create alerts for throttling, error spikes and sudden growth in usage. A runaway loop in your own code can send thousands of requests in minutes, and you want to learn about it before the invoice does.

17Understanding and Controlling Cost

Pay-per-use is flexible, but costs can surprise you if nobody watches them.

Service areaTypical billing unitSaving habit
Vision and OCRPer image or per pageResize large images, skip duplicates, cache results
Speech to textPer hour of audioTrim silence, use batch mode for archives
Text to speechPer characterCache audio for repeated phrases
LanguagePer text record or per thousand charactersSend only needed text, batch records
TranslatorPer million charactersTranslate once and store the result
Document IntelligencePer pageUse the simplest model that works, avoid reprocessing
Generative AIPer token processedShorter prompts, smaller models where possible, cache answers

Cache results

If the same image or sentence is analysed repeatedly, store the first answer and reuse it, instead of paying again.

Filter before you send

Do not send every file. Skip empty pages, duplicate documents and irrelevant content before calling the service.

Choose the right model size

For generative AI, smaller and cheaper models often handle simple tasks well, and large models should be kept for hard problems.

Use commitment tiers for steady volume

If monthly usage is stable and large, a commitment tier lowers the unit price.

Tag and budget

Tag each resource with its owner and purpose, set budgets and alerts, and review costs monthly.

!
Common Mistake

Leaving test scripts that call the service in a loop, or reprocessing the whole archive after a small change. Always limit test sizes and check whether results can be reused first.

18AI Services Compared with Other Options

Ready-made services are one of several ways to add intelligence to an application.

OptionWhat it isBest for
Azure AI ServicesPre-built and customisable AI behind simple interfacesCommon abilities such as vision, speech, language and documents
Azure OpenAI ServiceLarge generative models in AzureChat, writing, summarising, reasoning and question answering over your data
Azure Machine LearningA platform to build, train and deploy your own modelsSpecial problems where no ready service fits and you have data and skills
Azure AI FoundryA platform to build, evaluate and manage AI applications and agentsTeams building larger AI solutions that combine models, data and tools
Open-source models you hostModels you run on your own machinesFull control, special licensing or offline needs, with more operational work
Simple Analogy

AI services are the ready-made food shelf in a supermarket. Azure Machine Learning is the fully equipped kitchen for cooking your own recipe. Foundry is the whole restaurant management system that helps you design the menu, test dishes and run the service.

Choose AI Services when

Your need is common and well covered, speed matters and you want little operational effort.

Choose Azure Machine Learning when

Your problem is unusual, you have plenty of labelled data and you need full control of the model.

Combine them when

Many real systems use services for the common parts, such as reading text from documents, and custom or generative models for the specialised reasoning on top.

i
Start Simple

Begin with the simplest option that can meet the need. You can always move to custom models later, once the business has proved the value of the feature.

19Design Patterns That Work Well

These approaches appear again and again in successful AI-powered applications.

Pattern: Event-Driven Document Processing

A file lands in storage, an event triggers a function, the function calls Document Intelligence and Language services, and the extracted data is saved to a database. No servers run when there is nothing to process.

Pattern: Confidence-Based Human Review

Results with high confidence flow through automatically, and results with low confidence are sent to a person with a simple review screen. This balances speed and trust.

Pattern: AI Enrichment in Search

While files are indexed by Azure AI Search, built-in skills read text from images, detect language, extract entities and key phrases, so that search results become far richer.

Pattern: Retrieval Augmented Generation

A question is first used to search your own documents. The best passages are given to a generative model, which writes an answer based on them, with references.

Pattern: Gateway in Front of the Service

Your own back-end sits between users and the AI service. It authenticates users, enforces quotas, caches results and keeps keys safe.

flowchart TD
  U["Uploaded Document"] --> ST["Blob Storage"]
  ST -->|"event"| FN["Function"]
  FN --> DI["Document Intelligence"]
  FN --> LG["Language Service"]
  DI --> DB["Database"]
  LG --> DB
  DB --> RV["Human Review for Low Confidence"]
  DB --> AP["Business Application"]
        
FIG 4 — Event-driven pipeline that turns documents into structured data
flowchart LR
  Q["User Question"] --> SR["Search Your Documents"]
  SR --> P["Top Matching Passages"]
  P --> LLM["Generative Model"]
  LLM --> CS["Content Safety Check"]
  CS --> ANS["Answer with Sources"]
        
FIG 5 — Retrieval augmented generation with a safety check
“Let the machine do the reading, and let people make the decisions.”

20Real-World Examples

AI services quietly improve everyday experiences across industries.

Banks and insurers

Claims forms, identity documents and statements are read automatically. Calls are transcribed and analysed for compliance, and personal data is masked before storage.

Retail and e-commerce

Product photos are tagged automatically, customer reviews are analysed for sentiment, and shoppers search and chat in their own language through translation.

Healthcare

Doctors dictate notes that are transcribed, and long clinical documents are summarised, with strict privacy controls and human review of anything important.

Customer support centres

Calls are transcribed live, sentiment is tracked, summaries are written after each call, and chatbots answer routine questions from the company’s knowledge base.

Education and accessibility

Lectures receive live captions and translations, and text can be read aloud with natural voices, helping learners with different needs and languages.

Manufacturing and logistics

Cameras check products for defects, shipping labels are read automatically, and maintenance logs are searched using natural language.

Advantages

  • Add AI abilities in hours with no model training
  • Simple REST and SDK access from many languages
  • Scales automatically and is maintained by Microsoft
  • Enterprise security, networking and compliance options
  • Customisation and containers for special needs

Disadvantages / Trade-offs

  • Results are probabilistic, so mistakes always remain possible
  • Costs grow with usage and need active control
  • Pre-built models may not fit very specialised content
  • Quotas and throttling need careful handling
  • The catalogue changes, with services retired or renamed over time

21Anti-Patterns to Avoid

These shortcuts regularly lead to leaks, wrong decisions and large bills.

ANTI-PATTERN-01 Avoid
Problem

The service key is placed in a mobile app or in front-end code so that the app can call the service directly.

Why It’s Harmful

Anyone can extract the key, run up charges and misuse the resource. Revoking the key breaks every installed copy of the app.

Correct Approach

Call the service from your own back-end, use managed identities, keep any keys in Key Vault and rotate them regularly.

ANTI-PATTERN-02 Avoid
Problem

Important decisions, such as approving payments or rejecting job applicants, are made automatically from model output with no human review.

Why It’s Harmful

Models make mistakes and can reflect biases. Errors reach real people without anyone able to catch or correct them, creating harm and legal risk.

Correct Approach

Use confidence thresholds, route doubtful cases to people, keep audit trails and test for fairness across groups.

ANTI-PATTERN-03 Avoid
Problem

Every call is repeated for the same content, with no caching, and whole archives are reprocessed after each small change.

Why It’s Harmful

The bill grows quickly and the service is throttled, with no new value gained from the repeated work.

Correct Approach

Cache results, detect duplicates, process only new or changed items, and limit the size of tests.

ANTI-PATTERN-04 Avoid
Problem

Sensitive personal data is sent to the service without checking its data handling rules, regional location or network settings.

Why It’s Harmful

It may break privacy laws or company policy, and expose data through public network paths.

Correct Approach

Read the data policy of each service, choose suitable regions, use private endpoints, mask personal data where possible and involve privacy and legal teams early.

ANTI-PATTERN-05 Avoid
Problem

The team builds a long-lived product on a service without checking its retirement plans, or ignores deprecation notices.

Why It’s Harmful

When a service or model version is retired, the feature stops working and a rushed migration is needed.

Correct Approach

Follow Microsoft’s announcements, pin versions deliberately, design a thin wrapper around each service so it can be swapped, and plan regular reviews of the catalogue.

22Best Practices and a Starter Checklist

Walk through these steps whenever you add an AI service to an application.

1

Define the business outcome

State what the feature must achieve and how you will measure success, before choosing a service.

2

Prototype with real samples

Try pre-built models on real data, including difficult and messy examples, and record the accuracy.

3

Customise only where needed

Train custom models for gaps that matter, with varied and well-labelled data, and test on unseen examples.

4

Design for confidence and review

Use confidence scores, send uncertain cases to people and plan what happens when the service is unavailable.

5

Secure access

Use managed identities, Key Vault, least privilege and private networking, and keep keys out of client code.

6

Handle limits gracefully

Implement retries with back-off, queue the work and monitor throttling.

7

Apply responsible AI

Read transparency notes, test for fairness, add content safety checks, tell users about AI and keep human oversight.

8

Control cost and watch the roadmap

Cache, filter, tag, budget and review monthly, and follow service updates and retirement notices.

!
Frequent Beginner Mistakes

Trusting confidence scores blindly, exposing keys, skipping human review, testing only on perfect samples, training custom models on too few examples, ignoring throttling, forgetting data policies, running unbounded test loops, and building on services that are about to be retired.

23Frequently Asked Questions

Q1Are Cognitive Services and Azure AI Services the same thing?

Yes, Azure Cognitive Services was renamed Azure AI Services. The family continues to grow, so the newer name includes additions such as generative AI offerings, while many older articles still use the old name.

Q2Do I need to know machine learning to use them?

No. Pre-built services work through simple requests, and you only need ordinary programming skills. Machine learning knowledge helps when you build custom models or evaluate accuracy deeply.

Q3What is the difference between a multi-service and a single-service resource?

A multi-service resource gives one key and endpoint for many services with combined billing. A single-service resource covers one service, with separate keys, access control and cost tracking. Production systems usually prefer single-service resources.

Q4Is my data used to train the models?

For most Azure AI services, your input is not used to train the underlying models for other customers, but details differ by service and feature. Read the data privacy documentation for each service you use.

Q5How accurate are the results?

Accuracy depends on the service, the quality of the input and how close your data is to what the model has seen. Test with your own realistic samples, use confidence scores and plan human review for important cases.

Q6Can I run these services without the internet?

Some services are available as containers that run locally and need a connection only to report usage for billing. Not all services have containers, and some need approval, so check the current list.

Q7What does the 429 error mean?

It means you have sent too many requests too quickly and the service is asking you to slow down. Wait and retry with increasing delays, smooth your traffic with a queue and request a higher quota if you truly need more.

Q8How do I keep costs under control?

Cache results, avoid reprocessing, filter inputs, use the right model size, choose commitment tiers for steady volume, tag resources and set budgets with alerts.

Q9Should I use Azure OpenAI or the other AI services?

They solve different problems and are often combined. Use specialised services for reading documents, speech and translation, and generative models for writing, reasoning and conversation. Check which fits each part of your task.

Q10What happens if a service I use is retired?

Microsoft announces retirements well in advance and usually points to a replacement. Follow the notices, keep a thin wrapper around each service in your code, and plan migration time before the date arrives.

24Summary and Key Takeaways

Azure Cognitive Services, now known as Azure AI Services, gives every developer ready-made abilities to see, listen, speak, read, translate and understand. You create a resource, call its endpoint with simple requests and receive structured answers with confidence scores. Pre-built models solve most common needs quickly, custom models fill special gaps, and containers help when data must stay local. Success depends on testing with real data, keeping humans in charge of important decisions, protecting keys and data, handling limits gracefully, controlling costs and following the evolving catalogue.

Key Takeaways

  • It is AI as a service — pre-trained models are available through simple web calls, with no training or hardware needed for common tasks.
  • The family has clear areas — vision, speech, language, translator, document intelligence, generative AI with content safety, and AI search.
  • The name has changed — Cognitive Services is now Azure AI Services, and the catalogue evolves, so check current services and retirement dates.
  • Start pre-built, customise later — measure on real samples first and train custom models only for gaps that matter.
  • Trust, but verify — use confidence scores, human review and responsible AI practices, especially for decisions that affect people.
  • Secure every call — prefer managed identities and Key Vault, keep keys out of client code, and restrict networks with private endpoints.
  • Respect limits and costs — retry with back-off, cache and filter requests, and use budgets, tags and commitment tiers to keep the bill predictable.