AI/ML
Deploy OpenThinker 7B on Docker via Ollama: A Complete Guide
OpenThinker 7B Model for your Business?
Cost Efficiency (Open Source)
Lower Long Term costs
Customised data control
Pre-trained model
Get Your OpenThinker 7B AI Model Running in a Day
Free Installation Guide - Step by Step Instructions Inside!
Introduction
OpenThinker 7B is a powerful open-source language model that can be deployed efficiently in a containerized environment using Ollama. Running it inside a Docker container provides portability, scalability, and reproducibility, making it ideal for development, testing, and production use cases.
In this guide, we will cover:
Installing Docker and Ollama
Creating a Docker container for OpenThinker 7B
Running and interacting with the model inside the container
Using the model via CLI and Python
This step-by-step approach ensures that OpenThinker 7B runs smoothly while avoiding dependency conflicts.
Step 1: Install Docker
Docker allows you to create and manage containerized environments, ensuring the model runs consistently across different systems.
Install Docker on Ubuntu
Run the following commands to install Docker:
sudo apt update && sudo apt upgrade -ycurl -fsSL https://get.docker.com -o get-docker.shsudo sh get-docker.sh
Verify Installation
To confirm Docker is installed, check the version:
docker --versionYou should see an output similar to:
Docker version 24.0.5, build abcdefIf you're using Windows or macOS, download and install Docker Desktop from the official Docker website.
Step 2: Install Ollama
Ollama is a lightweight framework optimized for running large language models in an efficient way.
Pull the Ollama Docker Image
Since we are using Docker, we will pull the official Ollama image from Docker Hub:
docker pull ollama/ollamaOnce downloaded, verify the image is available:
docker images | grep ollamaYou should see output like:
REPOSITORY TAG IMAGE ID CREATED SIZEollama/ollama latest 123456abcdef 2 days ago 3.2GB
This confirms that Ollama is ready to use.
Step 3: Create a Dockerfile for OpenThinker 7B
Now, let's create a Dockerfile that will install OpenThinker 7B inside a Docker container.
Create a New Directory for Your Project
Navigate to a working directory and create a new folder for the project:
mkdir OpenThinker-7B-Docker && cd OpenThinker-7B-DockerCreate the Dockerfile
Inside the new directory, create a Dockerfile:
touch Dockerfilenano Dockerfile
Now, add the following content:
# Use the official Ollama image as baseFROM ollama/ollama# Download OpenThinker 7B inside the containerRUN ollama pull OpenThinker/OpenThinker-7B# Expose the default portEXPOSE 11434# Set Ollama as the entry pointENTRYPOINT ["ollama", "serve"]
Explanation of the Dockerfile
- FROM ollama/ollama: Uses the official Ollama image
- RUN ollama pull OpenThinker/OpenThinker-7B: Pre-downloads the model in the container
- EXPOSE 11434: Opens the necessary port for communication
- ENTRYPOINT ["ollama", "serve"]: Ensures Ollama starts automatically when the container runs
Save the file (CTRL + X, then Y, then Enter).
Step 4: Build and Run the Docker Container
Run the following command to build the image:
docker build -t openthinker-7b .
Once completed, check if the image is built successfully:
docker images | grep openthinker-7b
Expected output:
REPOSITORY TAG IMAGE ID CREATED SIZEopenthinker-7b latest 789xyzabcdef 5 minutes ago 3.5GBRun the Docker Container
Start the container in detached mode:
docker run -d --name openthinker_container -p 11434:11434 openthinker-7b
Here’s what the command does:
- -d: Runs the container in the background
- --name openthinker_container: Assigns a name to the container
- -p 11434:11434: Maps the container port to the host
Verify if the container is running:
docker ps
Expected output:
CONTAINER ID IMAGE COMMAND STATUS PORTS NAMESabcd1234 openthinker-7b "ollama serve" Up 2 mins 0.0.0.0:11434->11434/tcp openthinker_container
Step 5: Running the Model Inside the Container
Using Ollama CLI
Now that the container is running, you can interact with OpenThinker 7B via Ollama CLI:
ollama run OpenThinker-7B "What is the significance of deep learning in AI?"
Expected output:
Deep learning is a subset of machine learning that utilizes neural networks with multiple layers...Using Python Inside the Container
You can also interact with the model programmatically using Python:
import ollama
response = ollama.chat("OpenThinker-7B", "Explain reinforcement learning.")print(response)
Expected output:
Reinforcement learning is a type of machine learning where an agent learns by interacting with an environment...Step 6: Stopping and Removing the Container(Optional)
To stop the running container:
docker stop openthinker_containerTo remove the container:
docker rm openthinker_containerTo remove the Docker image:
docker rmi openthinker-7b
Conclusion
Running OpenThinker 7B inside a Docker container using Ollama ensures a streamlined, isolated and portable deployment environment. This approach eliminates dependency issues and makes it easier to scale the model across different systems.
Key Takeaways:
- Docker ensures portability and reproducibility
- Ollama provides a lightweight and optimized framework for running LLMs
- Running OpenThinker 7B in a container simplifies deployment and scaling
Ready to transform your business with our technology solutions? Contact Us today to Leverage Our AI/ML Expertise.
AI/ML