A short, plain-language tour of the ideas that every later chapter depends on.

What machine learning is

Traditional software follows rules that a programmer writes by hand: if the temperature is above thirty degrees, turn on the fan. Machine learning flips this around. Instead of writing the rules, you give the computer many examples of inputs and the correct answers, and it works out the rules by itself. The result of this learning is called a model, which is simply a mathematical recipe that takes new input and produces a prediction. A model that predicts house prices, spots spam email, recommends a movie, or reads a handwritten digit is the product of the same basic idea.

Features and labels

The pieces of information fed into a model are called features, such as the size of a house, its location, and its age. The answer you want the model to predict is called the label, such as the sale price. Training data is a table in which each row is one example, the columns are features, and one column holds the label. When a label is present the approach is called supervised learning. When there is no label and the program looks for natural groups or unusual items on its own, it is called unsupervised learning.

Training, validation, and testing

If a student sees the exam questions in advance, a perfect score proves nothing. For the same reason data is split into three parts. The training set is used to learn. The validation set is used while building to compare ideas and tune settings. The test set is kept locked away until the end to give an honest score. A model that scores well on training data but badly on new data has overfitted, meaning it memorized instead of understanding. A model that does badly on both has underfitted, meaning it is too simple to capture the pattern.

Training versus inference

Training is the slow, expensive stage in which the model learns, often using powerful machines for hours or days. Inference is the fast, repeated stage in which the finished model answers questions, often in milliseconds, thousands of times a day. These two stages have very different needs, which is why Azure Machine Learning provides separate tools and compute for each.

Measuring quality

Different problems need different yardsticks. For predicting a number, such as a price, common measures describe the average size of the errors. For classifying items into groups, such as spam or not spam, common measures include accuracy, which is the share of correct answers, precision, which is how many flagged items were truly positive, and recall, which is how many true positives were found. Choosing a measure that reflects real business cost is part of the craft, because a model that is ninety-nine percent accurate can still be useless if the rare cases it misses are the ones that matter.

A model is not a finished product when training ends. It is a living component that needs care for as long as it is used.

The lifecycle in plain steps

Every machine learning project moves through a loop. First comes understanding the business question and deciding what success means. Then comes collecting and preparing data, which usually takes most of the effort. Next the team trains and compares models, evaluates them honestly, and registers the best one. After that the model is deployed so applications can use it, and then it is monitored, because the world changes and a model that was accurate last year may be wrong today. When quality falls, the loop begins again with fresh data.

flowchart LR
  A["Define the problem"] --> B["Collect and prepare data"]
  B --> C["Train and tune models"]
  C --> D["Evaluate and register"]
  D --> E["Deploy"]
  E --> F["Monitor and detect drift"]
  F -->|quality drops| B
        

What MLOps means

MLOps, short for machine learning operations, applies the discipline of software engineering and operations to machine learning. It means that data, code, environments, and models are versioned, that training and deployment are automated and repeatable, that changes are reviewed and tested, and that deployed models are watched. Without MLOps, a model often lives in one person’s notebook, nobody can reproduce it, and nobody knows when it quietly stops working.

Why machine learning is harder than normal software

Azure Machine Learning is an enterprise service for the whole machine learning lifecycle. Its relationship to neighboring Microsoft products deserves an honest explanation.

The core definition

Azure Machine Learning, often shortened to Azure ML, is a managed cloud service for building, training, deploying, and managing machine learning models at scale. It provides a central workspace, managed compute that scales up and down, tools for tracking experiments, a registry for models, ways to publish models as web services or batch jobs, and features for monitoring, security, and governance. It supports open-source frameworks such as PyTorch, TensorFlow, and scikit-learn rather than tying you to a proprietary method, and it integrates with MLflow, a widely used open standard for tracking experiments and packaging models.

Ways of working

Neighbors and boundaries

Microsoft has several related offerings, and it helps to know which does what. Azure ML is the platform for custom machine learning: you bring data and code, and you train and operate your own models. Microsoft Foundry, formerly Azure AI Foundry, is the platform focused on building generative AI applications and agents using large language models, and it builds on the same workspace foundations. Microsoft Fabric includes its own data science experience that works close to its lake and reporting tools. Azure AI services offer ready-made prediction APIs for common tasks such as vision and speech, when you do not need to train anything yourself.

A candid note on change

The platform evolves quickly, and teams are advised to track lifecycle announcements. The first generation of the command line and Python tooling, known as version 1, is being retired in favor of version 2: command line version 1 support ended on 30 September 2025, and the Python SDK version 1 support ends on 30 June 2026, which has now passed. Existing version 1 workflows may still run for a while but carry security and breaking-change risk. Microsoft has also announced the retirement of Prompt Flow, its generative AI orchestration tool, on 20 April 2027, with Microsoft Agent Framework named as the replacement. This guide therefore teaches the version 2 way of working throughout and flags lifecycle risks where they matter.

Seeing how the pieces connect turns the product from a long menu into a system you can reason about.

The workspace at the center

Everything in Azure ML lives inside a workspace, the top-level resource that acts as the shared home for a team’s machine learning assets. It records experiments and their results, holds references to data, stores models and environments, defines compute, and hosts endpoints. Access to the workspace is controlled with Azure role-based access control, so you decide who can run jobs, who can deploy, and who can only look.

The companion resources

When a workspace is created, it relies on several other Azure services that store its state. A storage account holds files such as datasets, logs, and notebooks. A key vault holds secrets such as connection strings. A container registry stores the container images that package environments for training and deployment. An application monitoring resource collects telemetry from deployed endpoints. Understanding these companions explains many real-world problems, because permission, networking, or quota trouble with any one of them can break the workspace in ways that look mysterious.

flowchart TD
  WS["Azure ML Workspace"] --> ST["Storage Account: data, logs, outputs"]
  WS --> KV["Key Vault: secrets"]
  WS --> ACR["Container Registry: environment images"]
  WS --> AI["Application Insights: telemetry"]
  WS --> CMP["Compute: instances, clusters, serverless"]
  WS --> REG["Models, Data, Environments, Components"]
  WS --> EP["Endpoints: online and batch"]
  USER["Data scientists and engineers"] --> WS
  APPS["Applications"] --> EP
        

The main asset types

Control plane and data plane

As with other Azure services, management actions such as creating compute or registering a model go through the control plane, while the actual training data and prediction traffic use the data plane. Training data stays in your storage and is read by compute directly, and prediction requests travel from your application to the endpoint. This separation matters for networking and for security reviews.

How many workspaces you create, and how they relate, shapes security, cost, and team productivity for years.

One workspace or many?

A single shared workspace is simple but mixes experiments, secrets, and costs of unrelated teams, and makes tight access control hard. Many small workspaces isolate teams but make sharing models and data awkward. A common compromise is one workspace per team and per stage: a development workspace where scientists experiment freely, a test workspace for validation, and a production workspace with locked-down access where only automated pipelines deploy.

Shared registries

An Azure ML registry is a central place for sharing assets, such as models, environments, and components, across workspaces and even across subscriptions or regions. A model trained in the development workspace can be promoted to the registry and then deployed from it into production, with the same lineage and no manual copying. This supports the principle of building once and deploying many times.

Naming, tagging, and ownership

Adopt a naming convention for workspaces, compute, data, models, and endpoints that shows the purpose, environment, and region. Tag assets with owner, project, cost center, and data sensitivity. These small habits make cost reports, access reviews, and incident response far easier months later.

Where the workspace lives

Choose the region by data residency rules, latency to your data and users, and the availability of the compute types you need, since specialized graphics processors are not offered everywhere. Remember that the workspace and its companion resources should live in the same region to avoid delays and extra transfer costs.

Compute is where most of the money is spent and where most early frustrations appear. Knowing the options pays off quickly.

The compute types

Choosing machine sizes

Virtual machines come in families designed for different work. General-purpose and memory-optimized sizes suit data preparation and classical algorithms. Compute-optimized sizes suit heavy numeric loops. GPU sizes, equipped with graphics processors, suit deep learning, because these chips perform huge numbers of simple calculations in parallel. Some GPU sizes include fast interconnects between machines for distributed training. Start with the smallest size that fits your data in memory, measure, and scale up only if jobs are limited by computation.

Low-priority and spot capacity

Spare capacity can be rented at a steep discount, with the condition that it may be taken back at short notice. This is a good fit for fault-tolerant training that saves checkpoints regularly, and a poor fit for time-critical production inference. Combine spot capacity with checkpointing, so that a preempted job resumes instead of restarting.

Quotas: the surprise blocker

Every subscription has limits on the number of cores available per region and per virtual machine family. New subscriptions often have small or zero quota for GPU families. If a job fails to start because the requested cores exceed the quota, no amount of code can help until you request an increase. Plan quota requests early, before a project deadline, and request them for each region and family you intend to use.

Models are only as good as their data. Azure ML gives you ways to point to data without copying it, and to version it like code.

Datastores

A datastore is a saved connection to a storage service such as Azure Blob Storage, Azure Data Lake Storage Gen2, or a file share. It records where the storage is and how to authenticate, so that people and jobs do not need to handle keys directly. Preferably the connection uses identity-based access, meaning the job’s or user’s identity is checked against the storage permissions, instead of storing an account key.

Data assets

A data asset is a named and versioned reference to data, such as a single file, a folder, or a table definition. Instead of hard-coding a long storage path, a job asks for the data by name and version, for example customer-churn version three. This makes experiments reproducible: if a result needs to be reproduced next year, the exact data version can be found. Common asset types are a single file, a folder of files, and a table format that describes how to load data into a table with the right columns and types.

Data preparation options

Data quality and leakage

Two problems ruin more projects than any algorithm choice. The first is poor data quality: missing values, duplicates, inconsistent units, and mislabeled examples. The second is data leakage, which occurs when information that would not be available at prediction time sneaks into the features. For example, a model predicting customer cancellation that uses a field filled in only after the customer has left will look brilliant in testing and fail in real life. Review features with domain experts and ask of each: would I know this at the moment I must make the prediction?

Privacy and sensitivity

Training data frequently contains personal or confidential information. Minimize what you collect, remove or mask direct identifiers where possible, restrict access with roles and storage permissions, and record the sensitivity of each data asset with tags so that governance teams can find and review it.

A result that cannot be reproduced is a rumor. Environments freeze the software side of an experiment so that it runs the same anywhere.

What an environment is

An environment in Azure ML describes the software a job needs: the operating system base, the Python version, and the libraries with their exact versions. Behind the scenes it is built into a container image, a sealed package that runs identically on any compatible machine, and stored in the workspace’s container registry. Because the environment is versioned, a job can name the exact environment version it used, and anyone can run it again later with the same libraries.

Curated and custom environments

Microsoft provides curated environments, prebuilt and tested bundles for popular frameworks, which start fast because their images are already built and are updated regularly. When you need extra libraries or specific versions, you create a custom environment, either by adding a list of packages on top of a base image or by supplying your own container definition. Custom environments give full control but need maintenance: security patches, dependency conflicts, and rebuild time are your responsibility.

The reproducibility checklist

A job is how work runs in Azure ML. Everything else, from tracking to tuning, builds on this idea.

What a job is

A job is a unit of work submitted to run on compute. You describe what code to run, which environment to use, what data to provide, and which compute should do the work. Azure ML packages the code, starts the compute if needed, runs the job, streams logs, records metrics, and saves outputs. Jobs are grouped into experiments so related runs can be compared side by side. The key property is that the job runs somewhere other than your laptop, in a controlled and recorded way.

The job types

Tracking with MLflow

MLflow is an open standard for recording what happens during machine learning work. During a job, your code can log parameters such as the learning rate, metrics such as accuracy at each step, and artifacts such as plots or model files. Azure ML acts as the tracking server, so you get comparison charts, sortable tables of runs, and a permanent record, without building any infrastructure. Because MLflow is an open standard, code that logs with it is portable beyond Azure.

Hyperparameter tuning

Hyperparameters are settings chosen before training, such as the number of layers in a network or the depth of a tree. Finding good values by hand is slow. A sweep job tries many combinations, using a defined search space and a sampling method. Random sampling picks combinations by chance and works surprisingly well. Grid sampling tries every combination in a list, which is thorough but expensive. Bayesian sampling uses earlier results to choose promising next trials. An early termination policy stops trials that are clearly doing worse than the leaders, which saves large amounts of compute.

sequenceDiagram
  participant U as Engineer
  participant W as Workspace
  participant C as Compute Cluster
  participant S as Storage
  U->>W: Submit job with code, environment, data, compute
  W->>C: Request nodes and build or pull environment
  C->>S: Read input data asset
  C->>C: Run training and log metrics
  C->>W: Stream logs and metrics
  C->>S: Save model and outputs
  W-->>U: Show results and comparisons
        

Automated machine learning lets the service do the repetitive trial and error, so people can spend their time on the problem and the data.

What it does

Automated machine learning, often called AutoML, takes your prepared data, a target to predict, and a time or trial budget. It then tries many combinations of feature preparation steps, algorithms, and settings, evaluates each using validation techniques, and returns a ranked list with the best model on top. It supports common task types: classification, where the answer is a category, regression, where the answer is a number, and forecasting, where the answer is a future value of a time series. It also supports tasks on images and text.

Why teams use it

Using AutoML well

Not every useful model needs a team of programmers. Visual tools open machine learning to a wider audience.

The visual canvas

Designer is a drag-and-drop canvas in the studio where you assemble a training workflow from building blocks. You place a data block, connect it to blocks that clean or transform it, split it into training and testing parts, connect a learning algorithm, and add blocks to score and evaluate the model. Each block is a component, and the lines between them show how data flows. When you run it, Azure ML executes the steps on a compute cluster as a pipeline job.

Who benefits

Limits to know

Visual workflows are harder to review in source control than text files, harder to parameterize for many environments, and less flexible for unusual needs. Teams that grow into serious production use usually move their pipelines to code-defined components, using the visual canvas as a learning and prototyping aid. Also check the current status of any studio feature before relying on it, because older visual tooling has been retired as the platform moved to its current generation.

Pipelines turn a pile of notebook cells into a repeatable factory.

Components: reusable steps

A component is a self-contained step with a defined interface: named inputs, named outputs, the code to run, the environment to use, and the settings that can be changed. A component might clean data, engineer features, train a model, or evaluate one. Because the interface is explicit, a component can be shared across teams, versioned like software, and tested on its own. A well-designed component does one thing and does it predictably.

Pipelines: connected steps

A pipeline connects components into a workflow in which the outputs of earlier steps feed the inputs of later ones. Azure ML works out the order, runs independent steps in parallel, passes data between steps through storage, and can skip steps whose code and inputs have not changed by reusing earlier outputs, which saves time and money. A failure in one step is clearly shown, so you can fix and rerun without redoing everything.

flowchart LR
  D["Data Asset v3"] --> P["Prepare Data"]
  P --> F["Engineer Features"]
  F --> T1["Train Model A"]
  F --> T2["Train Model B"]
  T1 --> E["Evaluate and Compare"]
  T2 --> E
  E --> R["Register Best Model"]
        

Scheduling and triggers

Pipelines can be run on a schedule, such as retraining every week, or started by automation when something happens, such as new data arriving or a code change being merged. Scheduled retraining without checks is risky: a pipeline that silently registers a worse model can degrade a live system. Include an evaluation gate that compares the new model with the current one and registers it only if it is at least as good.

Some models are too large or too slow to train on one machine. Distribution spreads the work across many.

Why GPUs

Deep learning models perform enormous numbers of identical arithmetic operations. A central processor handles a few operations at a time very flexibly. A graphics processor handles thousands of simple operations simultaneously, which suits the matrix mathematics at the heart of neural networks. Training that takes weeks on ordinary processors may take hours on suitable graphics processors.

Two ways to split the work

Azure ML supports distributed training with common frameworks, including PyTorch, TensorFlow, and message-passing approaches, by launching the right processes across the nodes of a cluster and wiring them together. For very large jobs, node-to-node network speed becomes critical, so choose virtual machine sizes with high-speed interconnects and place nodes close together.

Practical guidelines

A trained model is an asset. Registering it turns a file on a disk into a governed, versioned, traceable product.

What registration does

Registering a model stores it in the workspace under a name and gives it a version number. Each registration records where the model came from, which job produced it, which data and code were used, the metrics measured, and any tags you add. Registering the same name again creates the next version, so you keep a full history. Deployments refer to a model by name and version, so there is never confusion about what is running.

Model formats

Azure ML supports models in several forms. MLflow models are a standard packaging format that includes the model, its dependencies, and a description of the expected inputs and outputs. They have a major advantage for deployment because the platform can often serve them without any custom scoring code. Custom models are any files plus a scoring script that you write to load the model and handle requests, which gives total flexibility at the price of more code to maintain. Triton-format models support high-performance serving of deep learning models on graphics processors.

Lineage and governance

Lineage answers the question: where did this model come from? A good registry entry lets an auditor trace a deployed model back to the exact job, code commit, data version, and environment. This matters for regulated industries, for debugging unexpected behavior, and for reproducing results. Add tags such as owner, approval status, and intended use, and keep a model card, a short document that describes what the model is for, how it was evaluated, and its known limits.

Stage management

Use tags or separate registries to mark the state of a model: candidate, validated, approved for production, retired. Automate transitions with pipelines that run tests and require human approval for sensitive steps. Never deploy a model that has not passed through these gates, and never delete versions that are still referenced by deployments or audits.

A model that nobody can call is worth nothing. Online endpoints let applications ask for predictions in milliseconds.

Endpoint and deployment

Two ideas work together. An endpoint is the stable front door: a web address and an authentication method that applications call. A deployment is the actual running thing behind the door: a model, the code and environment to serve it, and the compute that runs it. One endpoint can hold several deployments and divide traffic among them. This split lets you replace what runs behind the door without changing the address that applications use.

Managed endpoints

With managed online endpoints, Azure ML provisions and operates the virtual machines, load balancing, scaling, health checks, and security patching for you. You choose a machine size and an instance count, supply the model and environment, and the platform handles the rest. The alternative is Kubernetes online endpoints, in which you operate your own cluster and attach it to the workspace, which suits organizations that must standardize on Kubernetes.

The journey of a request

sequenceDiagram
  participant A as Application
  participant E as Endpoint
  participant L as Load Balancer
  participant D as Deployment Instance
  participant M as Model and Scoring Code
  A->>E: Send request with credentials
  E->>E: Authenticate and authorize
  E->>L: Route according to traffic split
  L->>D: Pick a healthy instance
  D->>M: Run scoring code on the input
  M-->>D: Return prediction
  D-->>A: Send response and log metrics
        

The scoring logic

For custom models, the scoring script has two jobs. An initialization step runs once when an instance starts, loading the model into memory so that requests do not pay that cost. A run step is called for every request, converting the incoming data into the form the model expects, calling the model, and formatting the answer. Keep the run step light. For MLflow models, this logic is generated for you.

Authentication

Endpoints can use keys, which are long-lived secrets that must be stored and rotated carefully, or tokens issued through Microsoft Entra ID, which tie access to identities and are generally preferred for enterprise use. Never embed keys in client applications that users can inspect.

Scaling and capacity

Online endpoints keep a number of instances running at all times, so you pay for them even when idle. Autoscaling rules can add instances when CPU use, request counts, or other metrics rise, and remove them when load falls, within minimum and maximum limits. Size the minimum for your normal load and for resilience: running at least two instances means a single failure does not interrupt service.

Not every prediction is needed in milliseconds. When a million records can wait until morning, batch is far cheaper.

Batch endpoints

A batch endpoint is designed for scoring large amounts of data asynchronously. You point it at a folder or dataset, it splits the data into mini-batches, processes them in parallel on a compute cluster, and writes predictions to storage. Because the cluster scales from zero for the job and shrinks back afterward, you pay only while work runs. Batch endpoints can also run whole pipelines, which is useful when scoring requires several steps.

Kubernetes online endpoints

Organizations that already run Kubernetes, in Azure or elsewhere, can attach a cluster to the workspace and deploy endpoints onto it. They gain control over node pools, networking, and placement, and can reuse existing operations practices, including in hybrid and on-premises settings. The price is responsibility: your team upgrades, patches, scales, and monitors the cluster. Choose managed endpoints unless you have a clear reason for Kubernetes.

flowchart TD
  Q{"How soon is the prediction needed?"}
  Q -->|Immediately, per request| ON["Online endpoint"]
  Q -->|Can wait, large volume| BA["Batch endpoint"]
  ON --> Q2{"Must you operate your own cluster?"}
  Q2 -->|No| MG["Managed online endpoint"]
  Q2 -->|Yes| KU["Kubernetes online endpoint"]
        

Replacing a live model is risky. Careful rollout techniques catch problems before most users are affected.

Traffic splitting

Because an endpoint can hold several deployments, you can divide traffic between them by percentage. A new deployment, called green, might start with five percent of requests while the proven blue deployment handles ninety-five. If metrics look healthy, you raise the share gradually until green takes everything, and then retire blue. If problems appear, you shift traffic back instantly, with no redeployment.

Traffic mirroring

Mirroring goes further: a copy of live requests is sent to the new deployment, but only the old deployment’s answers are returned to users. The new model is tested against real traffic without any user impact. You can compare its predictions, latency, and errors against the current one before ever giving it real responsibility. Mirroring is especially useful for models whose mistakes are costly.

Autoscale tuning

Autoscale rules should reflect how the model behaves. Scale on request rate or CPU use with a sensible cooldown, so the system does not thrash up and down. Model loading takes time, so a new instance may need a minute or more before it can serve. Keep a margin of spare capacity ahead of known peaks instead of relying only on reactive scaling.

Rollback is a feature to design

Keep the previous deployment available until the new one has proven itself, and write the rollback steps in a runbook that anyone on call can follow. Test the rollback in a non-production environment so it is trusted when needed.

Automation turns good practice from a favor people do into a system that always happens.

Continuous integration and delivery for models

In software, continuous integration automatically builds and tests every change, and continuous delivery moves tested changes toward production. Machine learning adds two more moving parts to the loop: data and models. A mature setup automates all of them. A code change triggers tests and a small training run. New data triggers retraining. A new model that passes quality gates is registered and deployed through staged environments.

flowchart LR
  C["Code or Data Change"] --> T["Automated Tests and Small Run"]
  T --> TR["Full Training Pipeline in Dev"]
  TR --> EV["Evaluation Gate vs Current Model"]
  EV -->|passes| RG["Register and Share to Registry"]
  EV -->|fails| X["Stop and Notify Team"]
  RG --> ST["Deploy to Test and Validate"]
  ST --> AP["Human Approval"]
  AP --> PR["Gradual Rollout to Production"]
  PR --> MO["Monitor"]
  MO -. drift or decay .-> C
        

The tools

Azure ML integrates with common automation systems, including GitHub Actions and Azure DevOps pipelines. These systems run in response to events such as a merged change, authenticate to Azure with a managed identity or service principal, and use the command line or Python tools to submit jobs and manage endpoints. Because the version 2 tooling uses configuration files that describe jobs, components, and endpoints, these definitions live in the repository and are reviewed like code.

Quality gates

Features are the raw material of models. A feature store keeps them consistent, shareable, and available at the right moment.

The problem

Teams often rebuild the same features again and again, each time slightly differently. Worse, a feature may be computed one way in a training notebook and another way in the live application, so the model sees different numbers in production than it learned from. This mismatch is called training-serving skew, and it silently reduces accuracy.

What a feature store provides

A managed feature store in Azure ML lets teams define features once, with their calculation logic, and then use them for both training and serving. It can materialize features into an offline store for building training datasets and into an online store for low-latency lookup during prediction. It supports point-in-time retrieval, which assembles training data using only the information that was known at each historical moment, preventing leakage from the future.

When it is worth it

Deploying is the middle of the story. Watching what happens afterward is what keeps a model useful.

Two layers of monitoring

The first layer is operational: is the endpoint healthy, fast, and available? The second layer is statistical: are the model’s predictions still good? Both are essential, and the second is the one that is commonly forgotten because a model can fail without any error. It simply starts giving worse answers.

Operational signals

These metrics flow into Azure Monitor and Application Insights, where you can create dashboards and alerts, and into log storage for investigation.

Drift: why models decay

Data drift means that the data arriving in production has changed compared with the data the model was trained on, for example customers become younger or a sensor is replaced. Prediction drift means that the distribution of the model’s outputs has shifted. Concept drift means that the relationship between inputs and the correct answer has changed, for example because customer behavior changes after a price rise. Any of these can make a once-excellent model quietly worse.

Model monitoring in Azure ML

Azure ML provides model monitoring that compares production data with a reference, such as the training data, using statistical measures, and raises alerts when differences exceed thresholds. Typical signals include data drift, prediction drift, data quality problems such as new missing values or unseen categories, and, when ground truth labels arrive later, actual performance measures. Monitoring runs on a schedule, so results appear as trends, which are far more informative than single readings.

A model that is accurate on average can still be unfair, opaque, or harmful to specific people. Responsible AI makes those risks visible.

Why it matters

Models learn from historical data, and history contains human bias. A hiring model trained on past decisions may repeat past discrimination. A loan model may perform well overall but poorly for a particular group. Regulators, customers, and employees increasingly expect organizations to understand and justify automated decisions. Responsible AI is therefore both an ethical duty and a practical business protection.

The Responsible AI dashboard

Azure ML includes a Responsible AI dashboard, a collection of tools for inspecting a model from several angles. It is built on open-source libraries, and it brings them together in a single view that can be generated as part of a pipeline and attached to a registered model.

A practical governance routine

Large language models have changed how many teams use machine learning. Understanding where Azure ML ends and Foundry begins prevents confusion.

What changed

Classical machine learning trains a model for one task on your own data. Generative AI uses very large pre-trained models that already understand language, images, or code, and that you adapt through instructions, retrieval of your documents, or light additional training. The work shifts from building a model from scratch to designing prompts, connecting data, adding safety controls, evaluating answers, and orchestrating tools.

Where the pieces live

Microsoft’s platform for generative AI applications and agents is Microsoft Foundry, which provides a catalog of models, tools for building and evaluating applications, safety features, and hosting. It builds on workspace foundations shared with Azure ML, so the two products overlap in identity, networking, storage, and compute concepts. Azure ML continues to be the place for custom training, fine-tuning workflows, large-scale experimentation, pipelines, and classical MLOps, while Foundry is the home for assembling generative applications.

Retrieval-augmented generation

A language model knows only what it saw in training, and may invent details. Retrieval-augmented generation, abbreviated RAG, reduces this by first searching your trusted documents for relevant passages and then giving those passages to the model along with the question, instructing it to answer from them. The documents are usually prepared by splitting them into chunks, converting each chunk into a numeric embedding that captures meaning, and storing them in a search index. The quality of a RAG system depends heavily on chunking, search quality, access control, and evaluation, more than on the language model alone.

A lifecycle warning for orchestration tools

Prompt Flow, the earlier orchestration tool used in both Azure ML and Foundry, has been announced for retirement on 20 April 2027, with Microsoft Agent Framework named as the modern replacement for orchestration. If your team uses Prompt Flow today, inventory those flows, learn the replacement, and plan a phased migration instead of waiting for the final months. As with all fast-moving areas, rely on current official guidance.

Machine learning platforms hold valuable data, powerful compute, and models that are themselves intellectual property. They need layered protection.

Identity and access

Sign-in uses Microsoft Entra ID with multi-factor authentication and conditional access. Azure role-based access control provides built-in roles for people who only read, who run experiments, and who manage the workspace, as well as custom roles when needed. Compute and endpoints should use managed identities so that code can read storage and call other services without any stored password. Grant each identity only the permissions it needs on the storage, registry, and key vault.

Network isolation

By default, a workspace is reachable over public endpoints. For stronger protection, you can place it behind private endpoints and enable a managed virtual network, a Microsoft-managed network that hosts your compute and controls outbound traffic. You choose an isolation mode. One allows outbound internet access, which is convenient for downloading packages. A stricter mode allows only approved outbound destinations through rules and private endpoints, which blocks data from leaving to unauthorized places. Companion resources such as storage, key vault, and container registry are reached through private endpoints too.

flowchart LR
  U["Users on corporate network"] --> PE["Private Endpoint to Workspace"]
  PE --> WS["Azure ML Workspace"]
  subgraph MVN["Managed Virtual Network"]
    CL["Compute Cluster"]
    EP["Online Endpoint Compute"]
  end
  WS --> MVN
  MVN -->|private endpoints| ST["Storage"]
  MVN -->|private endpoints| KV["Key Vault"]
  MVN -->|private endpoints| ACR["Container Registry"]
  MVN -. approved outbound only .-> PKG["Package sources"]
  INET["Public Internet"] -. blocked .-> WS
        

Protecting data and models

Model-specific risks

Models can leak information about their training data, can be copied by repeated querying, and, in generative systems, can be manipulated through malicious instructions hidden in input text. Limit who can call endpoints, rate-limit requests, filter and validate inputs and outputs, avoid training on data you are not permitted to use, and test generative applications with adversarial prompts before release.

The workspace itself is not where the money goes. Compute, storage, and always-on endpoints are.

The cost drivers

Practical savings

Patterns capture what experienced teams repeatedly find works. Anti-patterns capture what repeatedly goes wrong.

Helpful patterns

flowchart TD
  DATA["Curated Data and Feature Store"] --> TRN["Training Pipeline in Dev Workspace"]
  TRN --> GATE["Evaluation Gate"]
  GATE --> REG["Shared Registry"]
  REG --> TST["Test Workspace Validation"]
  TST --> PRD["Production Workspace Endpoint"]
  PRD --> LOG["Request and Prediction Logging"]
  LOG --> MON["Model Monitoring and Drift Alerts"]
  MON --> DATA
  PRD --> APP["Applications"]
  HUM["Human Review for Low Confidence"] -. feedback .-> DATA
        

Frequent anti-patterns

These illustrative composites show how the building blocks combine. They describe typical situations, not specific named customers.

These habits separate calm, trustworthy machine learning operations from firefighting.

When a job will not start or an endpoint misbehaves, an ordered approach finds the cause faster than guessing.

Locate the layer first

Problems sit in one of a few places: identity and permissions, networking, quota and capacity, the environment and container build, your code, the data, or the platform. Asking which layer first narrows the search dramatically. Read the job or deployment logs before changing anything, because the first error message usually points at the real cause, and later errors are often consequences.

flowchart TD
  P["Job or endpoint fails"] --> Q1{"Did it start at all?"}
  Q1 -->|No| S1["Check quota, compute state, permissions, and network access to storage and registry"]
  Q1 -->|Yes| Q2{"Did the environment build and container start?"}
  Q2 -->|No| S2["Check package conflicts, image build logs, registry access, and start-up time limits"]
  Q2 -->|Yes| Q3{"Does the code run on a small sample?"}
  Q3 -->|No| S3["Check data paths, schema, memory use, and library versions"]
  Q3 -->|Yes| Q4{"Is the endpoint healthy but answers wrong or slow?"}
  Q4 -->|Slow| S4["Check instance size, model load placement, scaling, and request payload size"]
  Q4 -->|Wrong| S5["Check input preparation differences, drift, and training-serving skew"]
        

Common symptoms and likely causes

The small-sample rule

Before running anything large, run the same code on a few hundred rows on a small compute. Most errors appear immediately, and they cost almost nothing. Teams that always rehearse on a small sample waste far less compute than teams that discover mistakes after a thousand GPU-hours.

No. Data scientists use it to build models, engineers use it to deploy and operate them, analysts can use automated machine learning and the visual designer, and administrators use it to control access, networking, and cost.

The workspace is the shared home for all machine learning assets and activity. An endpoint is one published service inside it that applications call to get predictions.

Use version 2. Command line version 1 support ended on 30 September 2025, and Python SDK version 1 support ended on 30 June 2026. Existing version 1 workflows may still run, but they carry security and breaking-change risk, so plan migration.

Subscriptions have core limits per region and per virtual machine family. Request an increase for the specific family and region, or choose another size or region while you wait.

Set clusters to a minimum of zero nodes, enable idle shutdown and schedules on compute instances, remove unused online deployments, and set budgets and alerts.

Choose batch when predictions are not needed instantly and volumes are large. It scales to zero and costs far less. Choose online when an application needs an answer within seconds per request.

Deploy the new version next to the old one, mirror live traffic to compare results, shift a small percentage of real traffic, ramp up gradually while watching metrics, and keep the old deployment available for instant rollback.

Drift means that production data or the underlying relationships have changed from what the model learned. It can silently reduce accuracy, so monitor it and investigate alerts, ideally together with real outcome measures when labels become available.

Yes. Use private endpoints for the workspace and companion resources, a managed virtual network with restricted outbound rules for compute, and Entra-based authentication for endpoints.

No. They overlap and share foundations. Azure ML remains the platform for custom training, pipelines, and classical MLOps, while Foundry is oriented toward building and evaluating generative applications and agents. Many solutions use both.

Microsoft has announced its retirement on 20 April 2027, naming Microsoft Agent Framework as the modern orchestration replacement. Inventory current flows, keep logic portable, and plan a phased migration.

Only when many models share features, low-latency feature lookup is needed, or training-serving skew is a real problem. For a single simple model, a well-managed curated table is often enough.

Azure Machine Learning gives teams a governed, scalable platform for the complete lifecycle of a model: tracked experiments on elastic compute, versioned data and environments, reusable pipelines, a registry with lineage, secure real-time and batch deployment, safe rollouts, automated delivery, and monitoring for drift. Its real value comes less from any single feature than from the discipline it makes easy: reproducibility, automation, review, and observation. Around that core sit responsible AI tools to keep models fair and explainable, security controls to keep data and models protected, and cost controls to keep experimentation affordable. The platform continues to change, with version 1 tooling retired, orchestration products being replaced, and generative AI work gravitating toward Foundry, so wise teams track lifecycle announcements, keep their logic portable, and treat official documentation as the final word, and this guide as their map.