AI/ML
Deploying OpenThinker 7B on Azure Using Virtual Machines: A Step by Step Guide
OpenThinker 7B Model for your Business?
Cost Efficiency (Open Source)
Lower Long Term costs
Customised data control
Pre-trained model
Get Your OpenThinker 7B AI Model Running in a Day
Introduction
Deploying OpenThinker 7B on Microsoft Azure using Virtual Machines (VMs) is a straightforward approach that offers full control over the infrastructure. We will:
Deploy an Azure Virtual Machine (VM)
Install Docker on the VM
Pull and run the OpenThinker 7B model as a Docker containerConfigure networking for external access
Key Benefits of Using VMs for Deployment
- Full control over resources - Customizable VM configurations
- Easier troubleshooting and debugging
- Cheaper for small scale deployments
Step 1: Prerequisites
Before starting, ensure you have:
An active Azure subscription
Azure CLI installed
Docker installed on your local machine
A pre built Docker image of OpenThinker 7B
Step 2: Authenticate and Set Up an Azure Virtual Machine (VM)
Log in to Azure CLI
az loginIf you have multiple subscriptions, set the active subscription:
az account set --subscription "your-subscription-id"Create a Resource Group
az group create --name OpenThinkerRG --location eastus
This creates a resource group named OpenThinkerRG in the East US region.
Create an Azure Virtual Machine
az vm create \ --resource-group OpenThinkerRG \ --name OpenThinkerVM \ --image UbuntuLTS \ --size Standard_NC4as_T4_v3 \ --admin-username azureuser \ --generate-ssh-keys
This creates an Ubuntu VM with an NVIDIA T4 GPU (adjust VM size if needed).
SSH keys are automatically generated for secure access.
Step 3: Configure the VM and Install Dependencies
Connect to the VM via SSH
ssh azureuser@<public-ip-address>
Find your VM’s public IP using:
az vm list-ip-addresses --resource-group OpenThinkerRG --name OpenThinkerVM --output table
Update System and Install Docker
Once inside the VM, update and install Docker:
sudo apt update && sudo apt upgrade -ysudo apt install -y docker.io
Enable Docker Service
sudo systemctl enable dockersudo systemctl start docker
Verify Docker Installation
docker --version
Expected output:
Docker version <docker version>
Step 4: Deploy OpenThinker 7B Using Docker
Pull the Docker Image from Docker Hub or Azure Container Registry (ACR)
If the model image is on Docker Hub:
docker pull your-dockerhub-username/openthinker-7b:latest
If the image is in Azure Container Registry (ACR):
Login to ACR
az acr login --name OpenThinkerRegistryRetrieve the ACR Login Server
az acr show --name OpenThinkerRegistry --query loginServer --output tsvPull the image
docker pull <acr-login-server>/openthinker-7b:latestRun OpenThinker 7B Container
docker run -d --name openthinker-7b -p 80:11434 <acr-login-server>/openthinker-7b:latest- -d runs the container in detached mode.
- -p 80:11434 maps port 80 of the VM to the model's port 11434.
Step 5: Configure Networking for External Access
Allow Traffic on Port 80
By default, Azure VMs block external traffic. Open port 80:
az vm open-port --port 80 --resource-group OpenThinkerRG --name OpenThinkerVMCheck Running Containers
docker ps
Expected output:
CONTAINER ID IMAGE PORTS STATUS[Container ID] openthinker-7b:latest 0.0.0.0:80->11434/tcp Up X minutes
Step 6: Verify Deployment
Find the Public IP of the VM
az vm list-ip-addresses --resource-group OpenThinkerRG --name OpenThinkerVM --output table
Test API Response
Use cURL or a web browser to check if the model is running:
curl http://<public-ip>Expected output:
{"message": "Model is up and running"}
Step 7: Setting Up Auto-Start on Reboot (Optional)
Ensure the container starts automatically after a reboot:
Create a Systemd Service
sudo nano /etc/systemd/system/openthinker.service
Paste the following:
[Unit]Description=OpenThinker 7B AI ModelAfter=network.target[Service]ExecStart=/usr/bin/docker start -a openthinker-7bExecStop=/usr/bin/docker stop openthinker-7bRestart=alwaysUser=root[Install]WantedBy=multi-user.target
Enable the Service
sudo systemctl daemon-reloadsudo systemctl enable openthinker.servicesudo systemctl start openthinker.service
Step 8: Scaling the Model (Optional)
If you need more computing power, consider:
Upgrading to a more powerful VM (e.g., Standard_NC8as_T4_v3)
Using multiple VMs with a Load Balancer
Running multiple containers on the same VM
To run multiple containers:
docker run -d --name openthinker-7b-2 -p 81:11434 <acr-login-server>/openthinker-7b:latest
This runs a second instance on port 81.
Step 9: Cleaning Up Resources (If Needed)
To stop the VM:
az vm stop --resource-group OpenThinkerRG --name OpenThinkerVMTo delete the VM:
az vm delete --resource-group OpenThinkerRG --name OpenThinkerVM --yesTo delete the resource group (removes all related resources):
az group delete --name OpenThinkerRG --yes --no-wait
Conclusion
Deploying OpenThinker 7B on an Azure Virtual Machine provides a flexible and controlled environment. By using Docker, we can quickly set up and run the model, exposing it over the internet with minimal configuration.
Ready to transform your business with our technology solutions? Contact Us today to Leverage Our AI/ML Expertise.
AI/ML