Deploy ML Application to Azure

Azure DevOps + Azure ML based ML application deployment tutorial:

🔷 Introduction: Context of Azure, Azure ML & Azure DevOps

1. Azure Overview Azure is Microsoft’s cloud computing platform offering a vast ecosystem of services for compute, storage, networking, databases, machine learning, DevOps, and more. It enables organizations to build, deploy, and manage applications and services through Microsoft-managed data centers.

Key Benefits:

• Global scalability and reliability
• Pay-as-you-go pricing
• Integrated security and compliance
• Strong ecosystem for enterprise DevOps and AI/ML workflows

2. What is Azure Machine Learning (Azure ML)? Azure Machine Learning is a cloud-based platform for training, deploying, automating, and managing machine learning models. It supports both code-first (Python SDK, CLI) and no-code (Designer, Studio) approaches.

Key Components:

• Workspaces: Central place to manage assets and operations
• Compute Targets: For training & inference (e.g., Azure ML Compute, AKS, ACI)
• Datasets & Datastores: For accessing and versioning data
• Pipelines: For orchestrating workflows (training, evaluation, deployment)
• Endpoints: Real-time and batch deployment options

3. What is Azure DevOps? Azure DevOps is a SaaS platform providing a complete DevOps toolchain for software development and deployment. It is widely used for continuous integration (CI) and continuous deployment (CD) of applications, including ML workflows.

Core Services:

• Azure Repos: Git repositories for version control
• Azure Pipelines: CI/CD for building, testing, and deploying code
• Azure Boards: Agile project planning
• Azure Artifacts: Package management
• Test Plans: Manual and automated testing

🎯 Purpose of This Tutorial This tutorial aims to build an end-to-end CI/CD pipeline for a machine learning application using:

• Azure ML for model training & deployment
• Azure DevOps for version control, build, and release automation

You’ll learn how to:

• Train ML models in Azure ML
• Store and manage code using Azure Repos
• Trigger training via Azure Pipelines
• Deploy models as endpoints
• Automate the entire ML lifecycle

✅ Step 1: Setup & Prerequisites for starting the Azure ML + Azure DevOps deployment tutorial.

🔹 Step 1: Prerequisites & Initial Setup

📌 Objective:
Set up the Azure and DevOps environment required for ML application CI/CD deployment.

🧰 Prerequisites:

1. Azure Subscription

• A valid Azure account with an active subscription
• https://portal.azure.com

1. Azure DevOps Account

• Sign up or log in: https://dev.azure.com
• Create an organization and a project (e.g., ml-deployment-demo)

1. Azure Machine Learning Workspace

• Create a workspace from the Azure portal or CLI
• It should include:
• A resource group
• A region (e.g., East US)
• Storage Account
• Key Vault
• Application Insights
• Container Registry (optional)

1. Git & GitHub (Optional but Recommended)

• For managing code in local repo and syncing with Azure Repos

1. Local Setup (for development)

• Python ≥ 3.8
• Install Azure CLI:
• curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
• Install Azure ML CLI extension:
• az extension add -n azure-cli-ml
• Install Azure ML SDK:
• pip install azureml-sdk

1. Azure DevOps Agent (Optional for self-hosted runs)

• Can be added later for running jobs on your own compute

📁 Output of This Step:

• Azure DevOps project is ready
• Azure ML workspace is created
• Local dev environment is ready
• CLI and SDK tools installed

🖼️ Azure ML CI/CD Flow

This diagram shows the end-to-end ML model lifecycle in an Azure DevOps pipeline using Azure ML.

🧩 Components Explained:

1. Create/Get Workspace

• Use Azure CLI or Azure ML SDK to create or fetch an existing Azure ML workspace.
• All assets (models, experiments, compute, images) are tied to this workspace.

1. Model Training (Experiment Run)

• Train your ML model (e.g., using scikit-learn, TensorFlow, etc.)
• Log metrics, outputs, and register as an experiment run in Azure ML.

1. Model Evaluation

• Compare current trained model against a baseline/production model using evaluation metrics.
• Decision logic: Promote model only if metrics improve.

1. Model Registration

• Register the trained model in Azure ML Model Registry with versioning.
• Enables reproducibility and controlled model promotion.

1. Scoring Image Creation (Docker)

• Create a Docker scoring image using Azure ML for deployment.
• This image will include the model + score.py + dependencies.

✅ Step 2: Create Azure ML Workspace

🔧 Tools:

• Azure CLI
• Azure ML Python SDK

📍Steps:

(A) Using Azure CLI

az login
az group create --name ml-demo-rg --location eastus
az ml workspace create --name ml-demo-ws --resource-group ml-demo-rg

(B) Using Python SDK

from azureml.core import Workspace

workspace = Workspace.create(
    name='ml-demo-ws',
    resource_group='ml-demo-rg',
    create_resource_group=True,
    location='eastus'
)
workspace.write_config()  # saves config.json locally

📁 Output of This Step:

• Azure ML workspace created
• Config file config.json saved to local project (used by SDK/CLI to auto-connect)

✅ Step 3: Prepare Training Script & Register Experiment in Azure ML

🎯 Objective:
Train your ML model and log it as an experiment run in Azure ML.

🧱 Project Folder Structure (Suggested)

ml_project/
├── train.py
├── environment.yml  (or conda_dependencies.yml)
├── config.json      (created in Step 2)
└── outputs/         (for model artifacts)

🧪 Training Script: train.py (Example)

from azureml.core import Run
import joblib
from sklearn.datasets import load_diabetes
from sklearn.linear_model import Ridge
from sklearn.model_selection import train_test_split

# Get run context
run = Run.get_context()
# Load data
X, y = load_diabetes(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y)
# Train model
model = Ridge(alpha=0.5)
model.fit(X_train, y_train)
# Log metrics
score = model.score(X_test, y_test)
run.log("r2_score", score)
# Save model
joblib.dump(model, "outputs/model.pkl")

📦 Define Environment: environment.yml

name: sklearn-env
dependencies:
  - python=3.8
  - scikit-learn
  - pip
  - pip:
      - azureml-defaults

🚀 Submit Training Script to Azure ML

from azureml.core import Workspace, Experiment, ScriptRunConfig, Environment

# Load workspace from config.json
ws = Workspace.from_config()
# Create experiment
exp = Experiment(workspace=ws, name='train-diabetes-model')
# Set up environment
env = Environment.from_conda_specification(name='sklearn-env', file_path='environment.yml')
# Submit job
src = ScriptRunConfig(source_directory='.', script='train.py', environment=env)
run = exp.submit(config=src)
run.wait_for_completion(show_output=True)

📁 Output of This Step:

• train.py runs in Azure ML
• Metrics (r2_score) are logged
• Model saved to outputs/model.pkl
• Experiment is visible in Azure ML portal

🖼️ Azure ML Model Lifecycle (Training to Deployment)

This diagram shows the end-to-end ML lifecycle on a Linux VM agent using Azure Machine Learning:

🔁 Overview of Steps:

1. Get or Create Workspace
   Set up the Azure ML workspace using CLI, SDK, or UI.
2. Train the Model
   Run your training script on the Linux VM and log results as an experiment.
3. Evaluate Model
   Compare the new model’s performance with existing/production models.
4. Register the Trained Model
   Store the trained model in Azure ML Model Registry for versioning and reuse.
5. Create a Scoring Image (Docker-based)
   Build a container image using the trained model + scoring script + environment. This is used for deployment to endpoints (AKS, ACI, etc.).

The bottom section (Source Code, Configs, Output) represents all your assets used throughout this workflow.

✅ Step 4: Register Trained Model in Azure ML

Once the training script has completed and model.pkl is saved in the outputs/ folder, we can register it.

📌 Code to Register Model

from azureml.core import Workspace, Model

# Load workspace
ws = Workspace.from_config()
# Register model from outputs folder
model = Model.register(
    workspace=ws,
    model_path="outputs/model.pkl",  # Path to model file
    model_name="diabetes-ridge-model",  # Name for model in registry
    tags={"area": "diabetes", "type": "regression"},
    description="Ridge regression model to predict diabetes progression"
)
print("Model registered:", model.name, "version:", model.version)

📁 Output of This Step:

• Model appears in Azure ML Model Registry with versioning.
• You can now deploy this model via ACI/AKS.

✅ Step 5: Create Scoring Image (Inference Setup)

This step prepares the model for deployment by packaging it with a scoring script and environment in a Docker image using Azure ML.

📁 What You Need:

• Registered model
• Scoring script (score.py)
• Environment file (myenv.yml or CondaDependencies)

🧠 1. Create score.py

This script defines how input data is processed and predictions are returned.

import joblib
import json
from azureml.core.model import Model
from sklearn.linear_model import Ridge

def init():
    global model
    model_path = Model.get_model_path('diabetes-ridge-model')
    model = joblib.load(model_path)
def run(raw_data):
    data = json.loads(raw_data)
    prediction = model.predict([data['data']])
    return prediction.tolist()

📦 2. Create Environment (e.g., myenv.yml)

name: diabetes-env
dependencies:
  - python=3.8
  - scikit-learn
  - pandas
  - pip:
      - azureml-defaults

🛠️ 3. Build the Inference Image

from azureml.core import Environment
from azureml.core.model import InferenceConfig
from azureml.core.webservice import AciWebservice, Webservice

# Load workspace
ws = Workspace.from_config()
# Define environment
env = Environment.from_conda_specification(name='diabetes-env', file_path='myenv.yml')
# Define inference config
inference_config = InferenceConfig(entry_script='score.py', environment=env)
# Define deployment config
deployment_config = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=1)
# Deploy as web service (next step)

✅ Step 6: Deploy Model on ACI (Azure Container Instance)

🖼️ 50-deployOnAci.py

This script deploys your scoring image on Azure Container Instances (ACI) using:

• CPU: 1 core
• Memory: 1 GB
• Tags: area=diabetes, type=regression
• Dynamically generated ACI name using timestamp

🧾 Deployment Code Snippet:

from azureml.core.webservice import AciWebservice, Webservice
import datetime

aci_config = AciWebservice.deploy_configuration(
    cpu_cores=1,
    memory_gb=1,
    tags={"area": "diabetes", "type": "regression"},
    description="A sample description",
)
aci_service_name = "aciwebservice" + datetime.datetime.now().strftime("%m%d%H")
service = Webservice.deploy_from_image(
    deployment_config=aci_config,
    image=image,
    name=aci_service_name,
    workspace=ws,
)
service.wait_for_deployment(show_output=True)
print("Deployed ACI Webservice: {}\nWebservice URI: {}".format(
    service.name, service.scoring_uri
))

No, I haven’t described how to create, save (.pkl), and load the model yet. Here's a quick complete snippet for that:

🧠 Train & Save ML Model (.pkl)

train_model.py

import pickle
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

# Load dataset
X, y = load_iris(return_X_y=True)
# Train-test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Train model
clf = RandomForestClassifier()
clf.fit(X_train, y_train)
# Save to .pkl
with open("model.pkl", "wb") as f:
    pickle.dump(clf, f)

✅ Load .pkl in FastAPI

Already shared in the main.py:

with open("model.pkl", "rb") as f:
    model = pickle.load(f)

Let me know if you also want to:

• Save as .joblib or .onnx
• Add test script for the FastAPI endpoint
• Package this into CI/CD YAML pipeline automatically

📌 Next Step: Test ACI Webservice

Once deployed, we’ll write a script to send a test request to the model endpoint (60-AciWebserviceTest.py).

💼 Project Context

• Project: ML-Devops
• Pipeline: Deploy Webservice
• Release: Release-17
• Artifact Source: devops-for-ai-CI from branch master

📦 Continuous Deployment Trigger

• User: 99.user.name
• Date/Time: 12/04/2023, 18:53
• Trigger Type: Automatic — the release is automatically triggered when new artifacts are available.

🚀 Deployment Stages

1. QA — Deploy on ACI (Azure Container Instances)

• Status: In Progress
• Tasks: 5 out of 6 completed
• Current Task: 50. Deploy Webservice on ACI
• Start Time: 12/04/2023, 18:54
• Triggered At: 18:53
• Details:
• Deployment is automatically triggered.
• It’s deploying on ACI, ideal for lightweight, quick QA/staging deployments.

2. Prod — Deploy on AKS (Azure Kubernetes Service)

• Status: Not Deployed
• Triggered After: QA stage completes successfully.
• Target: Full-scale production deployment using AKS for scalability and orchestration.

📑 Logs and Tracking

• Tabs: Summary, Commits, Work Items, View Logs
• Associated Changes:
• Commit ID: 32
• Branch: master

Here’s a structured response covering both parts:

✅ 1. YAML Configuration & Pipeline Task Explanation

🔧 Pipeline Tasks in Deploy Webservice

Typical tasks in a deployment stage for an ML model using Azure DevOps:

1. Download Artifacts
   Downloads model files from previous CI pipeline (e.g., .pkl, .onnx, or .joblib model).
2. Install Dependencies
   Sets up Python environment and required libraries using pip.
3. Build Docker Image
   Uses a Dockerfile to containerize the FastAPI service wrapping the model.
4. Push Docker Image to ACR
   Pushes image to Azure Container Registry.
5. Deploy to ACI/AKS
   Uses Azure CLI or Kubernetes manifests to deploy on ACI (QA) or AKS (Prod).
6. Post-deployment Tests (Optional)
   Runs integration tests or probes service health.

📝 YAML Snippet Example

trigger:
- master

stages:
- stage: DeployWebservice
  jobs:
  - job: DeployModel
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - task: UsePythonVersion@0
      inputs:
        versionSpec: '3.9'
    - script: pip install -r requirements.txt
      displayName: 'Install dependencies'
    - task: Docker@2
      inputs:
        containerRegistry: '$(dockerRegistryServiceConnection)'
        repository: 'fastapi-ml-model'
        command: 'buildAndPush'
        Dockerfile: '**/Dockerfile'
        tags: |
          $(Build.BuildId)
    - task: AzureCLI@2
      inputs:
        azureSubscription: 'YourAzureSub'
        scriptType: 'bash'
        scriptLocation: 'inlineScript'
        inlineScript: |
          az container create \
            --resource-group your-rg \
            --name fastapi-model \
            --image youracr.azurecr.io/fastapi-ml-model:$(Build.BuildId) \
            --dns-name-label your-model-api \
            --ports 80

🚀 2. FastAPI-Based Webservice to Serve ML Model

📁 Folder Structure

fastapi-ml-service/
├── model.pkl
├── Dockerfile
├── main.py
├── requirements.txt

main.py Example

from fastapi import FastAPI, Request
import pickle
from pydantic import BaseModel

app = FastAPI()
# Load model
with open("model.pkl", "rb") as f:
    model = pickle.load(f)
# Request schema
class PredictRequest(BaseModel):
    feature1: float
    feature2: float
    # Add more features as per model input
@app.post("/predict")
def predict(request: PredictRequest):
    input_data = [[request.feature1, request.feature2]]
    prediction = model.predict(input_data)
    return {"prediction": prediction[0]}

requirements.txt

fastapi
uvicorn
scikit-learn
pydantic

Dockerfile

FROM python:3.9

WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80"]

Let me know if you need:

• Sample model training + pickle code
• Kubernetes manifest for AKS
• ACI-specific deployment CLI
• Health check integration

All images used in this tutorial are credited to Azure. Also, you can find informative tutorials on Azure DevOps here and here.

You can find many tutorials related to Fastapi and other Python frameworks on how to develop web services which can call the ML model.