How To Install DeepSeek on Rocky Linux 9

DeepSeek, an advanced open-source AI language model, offers impressive capabilities for natural language processing, coding assistance, and content generation when deployed locally on your Linux system. This comprehensive guide walks you through the complete process of installing DeepSeek on Rocky Linux 9, from initial preparation to advanced configuration and troubleshooting. By following these detailed instructions, you’ll be able to harness the power of this sophisticated AI model on your own enterprise-grade Linux system without relying on cloud services.

Introduction to DeepSeek and Rocky Linux 9

DeepSeek is an advanced open-source large language model (LLM) developed by DeepSeek AI, a Chinese artificial intelligence company founded in 2023 by Liang Wenfeng. This powerful AI model offers capabilities comparable to OpenAI’s models across tasks like mathematics, coding, and general reasoning. Running DeepSeek locally on Rocky Linux 9 provides several advantages, including complete control over your data, elimination of cloud service dependencies, and customization options to suit your specific needs.

Rocky Linux 9, a community enterprise operating system designed to be 100% bug-for-bug compatible with Red Hat Enterprise Linux (RHEL), provides a stable and secure foundation for running sophisticated AI models like DeepSeek. The combination of DeepSeek’s advanced AI capabilities with Rocky Linux’s enterprise-grade stability creates a powerful self-hosted AI solution suitable for developers, researchers, and organizations requiring on-premises AI capabilities.

Understanding DeepSeek

DeepSeek represents a significant advancement in open-source AI technology, featuring sophisticated natural language processing capabilities that make it suitable for a wide range of applications. The model excels at text generation, translation, summarization, and AI-driven content creation. Unlike many AI systems that require cloud-based processing, DeepSeek can be installed locally on your Rocky Linux 9 system, providing complete control over your data and eliminating potential privacy concerns associated with cloud services.

Key features of DeepSeek include multilingual support, advanced code generation and debugging capabilities across various programming languages, and context-aware responses that maintain longer conversation history for more accurate outputs. The model can also be fine-tuned for specific industries such as finance, healthcare, and customer support, making it highly versatile for specialized applications.

DeepSeek’s flagship model, DeepSeek-R1, is gaining popularity for its advanced reasoning capabilities that rival closed-source alternatives. Through distillation techniques, DeepSeek has created smaller, more efficient models that inherit the original model’s reasoning abilities while being significantly more practical to self-host on consumer or enterprise hardware. These distilled models include versions based on open-source models like LLaMA and Qwen, trained using data generated by the larger DeepSeek-R1 model.

Prerequisites and System Requirements

Before attempting to install DeepSeek on Rocky Linux 9, ensure your system meets the necessary hardware and software requirements. While the full DeepSeek-R1 671B model with its Mixture of Experts (MoE) architecture requires an impractical 1.5 TB of VRAM, the distilled models are specifically designed for practical self-hosting on more modest hardware.

For optimal performance, your system should have:

• A modern CPU with multiple cores for efficient processing
• Minimum 8GB RAM, though 16GB or more is strongly recommended for better performance
• At least 20GB of free disk space for the installation and model files
• A dedicated GPU, preferably NVIDIA, for accelerated inference (though not strictly required for smaller models)
• Rocky Linux 9 with the latest updates installed
• Basic understanding of Linux terminal commands and system administration

The specific hardware requirements may vary depending on which DeepSeek model variant you choose to install. The smaller models (1.5B, 8B) can run on systems with more modest specifications, while larger models (32B, 70B) demand significantly more resources, particularly GPU memory. For production environments or intensive usage, consider using a system with multiple GPUs and abundant RAM to ensure smooth operation and responsive AI interactions.

Preparing Rocky Linux 9 for DeepSeek Installation

Proper preparation of your Rocky Linux 9 system is essential for a successful DeepSeek installation. This preparation involves updating your system, installing necessary dependencies, and configuring the environment to support AI model execution.

First, update your Rocky Linux 9 system to ensure you have the latest security patches and package versions. Open a terminal and execute the following commands:

sudo dnf update -y
sudo systemctl reboot

This comprehensive update ensures your system is running the latest kernel and package versions, providing a stable foundation for DeepSeek installation.

Next, install the essential packages and dependencies required for DeepSeek. Python 3.8 or later is necessary, along with pip for package management and Git for repository cloning:

sudo dnf install python3 python3-pip git cmake -y

Verify that Python is correctly installed and check its version:

python3 --version
pip --version
git --version

Rocky Linux 9 typically includes Python 3.9, which meets DeepSeek’s requirements. However, additional development packages may be needed to support the various Python libraries used by DeepSeek and Ollama:

sudo dnf groupinstall "Development Tools" -y
sudo dnf install python3-devel zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel libffi-devel -y

If you plan to use a GPU for accelerated inference, you’ll need to install the appropriate drivers and CUDA libraries. For NVIDIA GPUs, the process involves adding the NVIDIA repository and installing the required packages:

sudo dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo
sudo dnf install cuda nvidia-driver -y

After installing the GPU drivers, reboot your system to ensure they’re properly loaded:

sudo systemctl reboot

Finally, create a dedicated directory for DeepSeek files to keep your installation organized:

mkdir ~/deepseek
cd ~/deepseek

With these preparations complete, your Rocky Linux 9 system is now ready for the installation of Ollama and the DeepSeek models.

Installing Ollama on Rocky Linux 9

Ollama serves as a critical platform for running large language models like DeepSeek locally on your system. It simplifies the complex process of model management and provides an API for interacting with these models. Installing Ollama on Rocky Linux 9 is the first major step in deploying DeepSeek.

Begin by downloading and installing Ollama using the official installation script:

curl -fsSL https://ollama.com/install.sh | sh

This script automatically downloads and installs the latest version of Ollama suitable for your system architecture. During installation, the script creates a system service that allows Ollama to run in the background and start automatically when your system boots.

After installation completes, verify that Ollama was installed correctly by checking its version:

ollama --version

The Ollama service should start automatically after installation. You can confirm it’s running with the following command:

systemctl status ollama

If the service isn’t running, you can start it manually:

sudo systemctl start ollama

To ensure Ollama starts automatically whenever your system boots, enable the service:

sudo systemctl enable ollama

By default, Ollama listens on http://127.0.0.1:11434 for local connections only. If you need to access Ollama from other machines on your network, you’ll need to adjust its configuration to listen on all network interfaces.

Create a custom Ollama service configuration directory:

sudo mkdir -p /etc/systemd/system/ollama.service.d

Create a configuration file to modify the server address:

sudo bash -c 'cat > /etc/systemd/system/ollama.service.d/override.conf << EOF
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
EOF'

Reload the systemd configuration and restart Ollama:

sudo systemctl daemon-reload
sudo systemctl restart ollama

If you encounter any issues with Ollama installation, check the system logs for detailed error messages:

journalctl -u ollama

With Ollama successfully installed and running, you’re now ready to install and configure the DeepSeek models.

Installing DeepSeek Models

With Ollama successfully installed, you can now proceed to install the DeepSeek models. DeepSeek offers several model variants with different parameter sizes, allowing you to choose the one that best matches your hardware capabilities and performance requirements.

The DeepSeek models available through Ollama include:

• DeepSeek-R1:1.5B (smallest, least resource-intensive)
• DeepSeek-R1:8B (balanced performance and resource usage)
• DeepSeek-R1:14B (intermediate model)
• DeepSeek-R1:32B (advanced capabilities, higher resource requirements)
• DeepSeek-R1:70B (largest model, most capable but very resource-intensive)

For systems with limited resources, start with the 1.5B or 8B model. For more powerful systems with dedicated GPUs, you might consider the larger models for enhanced capabilities.

To install a DeepSeek model, use the Ollama run command followed by the model name and size. For example, to install the 8B model:

ollama run deepseek-r1:8b

When executed for the first time, this command downloads the model from the Ollama model repository and loads it into memory. The download size varies based on the model variant, with larger models requiring more bandwidth and storage space.

During the first run, Ollama enters an interactive mode with the model. You can test the model by typing prompts and receiving responses. To exit this interactive session, press Ctrl+D or type “/exit“.

If you want to install multiple DeepSeek model variants, you can repeat the process with different model names:

ollama run deepseek-r1:1.5b
ollama run deepseek-r1:14b

To view all installed models, use:

ollama list

If you need to remove a model to free up disk space, use:

ollama rm deepseek-r1:8b

Replace “8b” with the appropriate model size you wish to remove.

For non-interactive usage or integration with other tools, you can start the Ollama service and then use the API to interact with the models:

sudo systemctl start ollama

The model files are stored in `/usr/share/ollama/models` by default. When you run a model for the first time, it may take several minutes to load into memory, particularly for larger models. Subsequent runs will be faster as Ollama caches the model data.

Setting Up a Web UI for DeepSeek

While DeepSeek can be used directly through the command line via Ollama, setting up a web interface provides a more user-friendly experience for interacting with the model. Open WebUI is a popular choice for creating a graphical interface for DeepSeek and other Ollama-based models.

Begin by ensuring you have the necessary prerequisites installed. For Open WebUI, you’ll need Node.js and npm:

sudo dnf install nodejs npm -y

You can also use Anaconda for managing the Python environment for Open WebUI. If you choose this approach, download and install Anaconda:

wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh
bash Anaconda3-2024.10-1-Linux-x86_64.sh

Follow the on-screen prompts to complete the Anaconda installation. After installation, initialize Anaconda in your current shell:

source ~/.bashrc

Now, create a dedicated Conda environment for Open WebUI:

conda create -n openwebui python=3.10
conda activate openwebui

With the environment prepared, clone the Open WebUI repository:

git clone https://github.com/open-webui/open-webui.git
cd open-webui

Install the required Python dependencies:

pip install -r requirements.txt

Start the Open WebUI server:

python app.py

By default, Open WebUI runs on port 8080. Open your web browser and navigate to http://localhost:8080 to access the interface.

Alternatively, you can use Chatbox, another user-friendly client for AI models that works well with Ollama. Chatbox provides a desktop application experience and is available for download from their website.

To configure either interface to work with your DeepSeek model:

1. In the settings or configuration section, select Ollama as the model provider
2. Set the Ollama API host to http://127.0.0.1:11434 (or your server’s IP if accessing remotely)
3. Select your installed DeepSeek model from the available models list

For production environments, consider setting up Open WebUI as a system service to ensure it starts automatically with your system:

sudo bash -c 'cat > /etc/systemd/system/openwebui.service << EOF
[Unit]
Description=Open WebUI for AI models
After=network.target ollama.service

[Service]
User=yourusername
WorkingDirectory=/path/to/open-webui
ExecStart=/home/yourusername/anaconda3/envs/openwebui/bin/python app.py
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF'

Replace “yourusername” and paths with your actual username and installation paths. Then enable and start the service:

sudo systemctl enable openwebui
sudo systemctl start openwebui

Testing and Using DeepSeek

After installing DeepSeek and setting up a web interface, it’s important to test the model to ensure everything is working correctly. Both command-line and web interface approaches offer different advantages for interacting with DeepSeek.

For command-line testing, start a conversation with your DeepSeek model:

ollama run deepseek-r1:8b

When the model loads, try some sample prompts to evaluate its performance:

• “Explain the concept of containerization in Linux”
• “Write a Python function to sort a list of dictionaries by a specific key”
• “Summarize the key features of Rocky Linux 9”

Observe the model’s responses for accuracy, coherence, and relevance. For more complex tasks, try multi-turn conversations to test the model’s context awareness.

When using DeepSeek through the web interface, you’ll benefit from a more intuitive chat-style interaction. The interface typically displays your conversation history and may offer additional options for adjusting model parameters or saving conversations.

DeepSeek excels at various tasks including:

• Code generation and debugging
• Technical explanations and documentation
• Content creation and summarization
• Translation between languages
• Problem-solving and logical reasoning

For optimal results, provide clear, detailed prompts that specify exactly what you want. You may need to experiment with different prompting techniques to achieve the best responses, especially for specialized or technical topics.

Performance may vary depending on your hardware, with larger models generally providing more sophisticated responses but requiring more computational resources and time to process.

Performance Optimization

Optimizing your DeepSeek installation on Rocky Linux 9 can significantly improve response times and overall performance. The key areas to focus on include memory management, CPU/GPU utilization, and system configuration.

For memory optimization, consider allocating swap space if your system has limited RAM. This helps prevent out-of-memory errors when running larger models:

sudo dd if=/dev/zero of=/swapfile bs=1G count=8
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

To make this swap permanent, add it to your fstab file:

echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

If you’re using a GPU for inference, ensure your CUDA drivers are properly configured and up to date. You can check GPU utilization during model inference using:

nvidia-smi -l 1

To improve CPU performance, consider adjusting the CPU governor to performance mode:

sudo dnf install kernel-tools -y
sudo cpupower frequency-set -g performance

For systems with multiple users or applications, you can prioritize Ollama using nice and ionice:

sudo systemctl edit ollama

Add the following to the file:

[Service]
Nice=-10
IOSchedulingClass=realtime
IOSchedulingPriority=0

When using smaller models like DeepSeek-R1:1.5b or DeepSeek-R1:8b, consider running multiple instances to serve different users or applications simultaneously. For larger models, focus on dedicating as much system resources as possible to a single instance.

Monitoring system performance during DeepSeek operation can identify bottlenecks:

sudo dnf install htop iotop -y

Use these tools to monitor CPU, memory, and disk usage while running DeepSeek to identify potential bottlenecks in your system.

Troubleshooting Common Issues

When working with DeepSeek on Rocky Linux 9, you may encounter various issues. Here are solutions to the most common problems:

If Ollama fails to start, check the system logs for detailed error messages:

journalctl -u ollama -n 50

For “out of memory” errors when loading larger models, ensure you have sufficient RAM and swap space. Alternatively, try a smaller model variant that fits within your system’s memory constraints.

If you experience slow response times, this could indicate CPU or GPU bottlenecks. Monitor resource usage during model inference and consider upgrading hardware or optimizing system settings as described in the previous section.

For connectivity issues with the web interface, verify that the Ollama service is running and accessible:

curl http://localhost:11434/api/version

If a model download fails or becomes corrupted, remove the partially downloaded model and try again:

ollama rm deepseek-r1:8b
ollama run deepseek-r1:8b

For GPU-related issues, ensure your CUDA drivers are correctly installed and compatible with your GPU:

nvidia-smi

If this command returns an error, you may need to reinstall or update your GPU drivers.

Advanced Configuration and Customization

For users seeking to further enhance their DeepSeek installation, Rocky Linux 9 offers numerous advanced configuration options. Setting up DeepSeek as a system service ensures it starts automatically with your system and remains available even after reboots.

You can create custom model configurations by defining a Modelfile:

mkdir -p ~/ollama/models
cd ~/ollama/models
nano Modelfile

Add content like:

FROM deepseek-r1:8b
PARAMETER temperature 0.7
PARAMETER top_p 0.9
SYSTEM You are an AI assistant specializing in Linux administration.

Then build your custom model:

ollama create mydeepseek -f Modelfile

For integration with other applications, Ollama provides a RESTful API that allows DeepSeek to be incorporated into larger systems or workflows:

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "deepseek-r1:8b",
  "prompt": "Explain how to optimize Rocky Linux 9 for performance"
}'

You can also set environment variables to customize Ollama’s behavior by creating a configuration file:

sudo mkdir -p /etc/ollama
sudo nano /etc/ollama/config

Add variables like `OLLAMA_MODELS=/path/to/custom/models` to change the models directory location.

Congratulations! You have successfully installed DeepSeek. Thanks for using this tutorial for installing the DeepSeek AI model on Rocky Linux 9 system. For additional help or useful information, we recommend you check the official DeepSeek website.