How To Install OSSEC on Debian 13

Linux servers face constant threats — from brute-force SSH attacks to silent rootkits running undetected for weeks. If you’re running Debian 13 (Trixie) without a Host-based Intrusion Detection System (HIDS), your server is flying blind. In this Linux server tutorial, you’ll learn how to install OSSEC on Debian 13, configure a manager-agent architecture, register agents, and integrate everything with systemd for a production-ready deployment.

OSSEC is one of the most battle-tested open-source HIDS tools available. It monitors file integrity, analyzes logs, detects rootkits, and fires real-time alerts — all without a licensing fee. This guide is based on hands-on testing on a fresh Debian 13 Trixie VPS running OSSEC v3.8.0. Follow along and you’ll have a fully operational OSSEC setup running on Debian 13 in under 30 minutes.

What Is OSSEC HIDS and Why Should You Install It on Debian 13?

OSSEC (Open Source Security) is a free, open-source HIDS originally developed by Daniel Cid and maintained under Trend Micro Inc. Unlike network-based IDS tools like Snort or Suricata, OSSEC operates directly on the host — analyzing what’s happening inside the machine, not just what’s coming in over the wire.

OSSEC delivers five core capabilities in a single lightweight package:

• File Integrity Monitoring (FIM): Detects unauthorized changes to system files, configs, and binaries
• Log Analysis: Parses and correlates logs from syslog, auth.log, Apache, and more
• Rootkit Detection: Scans for hidden processes, files, and kernel-level tampering
• Real-Time Alerting: Sends email or syslog notifications when suspicious activity is detected
• Active Response: Automatically blocks malicious IPs via firewall rules

OSSEC supports three deployment modes:

For this guide, we’ll configure a Manager + Agent setup — the most scalable and production-relevant architecture.

Prerequisites Before You Install OSSEC on Debian 13

Before you begin, make sure you have the following ready:

• Two Debian 13 (Trixie) servers — one for the Manager, one for the Agent (or a single server for local mode)
• Root or sudo privileges on both systems
• Minimum specs: 1 vCPU, 1 GB RAM, 10 GB disk space per server
• SSH access to both machines
• A stable internet connection to pull packages and source code from GitHub
• UFW installed — if not: sudo apt install ufw
• Basic familiarity with the Linux command line

Note: OSSEC on Debian 13 must be compiled from source. The apt repositories carry outdated versions. This guide installs OSSEC v3.8.0 — the latest stable release as of early 2026.

Step 1: Update Your Debian 13 System

Always update your system before installing any server software. This prevents dependency conflicts and ensures you’re building OSSEC against the latest libraries.

sudo apt update && sudo apt -y upgrade

What this does: apt update refreshes the package index. apt -y upgrade applies all pending security patches and package upgrades automatically.

Expected Output

Reading package lists... Done
Building dependency tree... Done
Calculating upgrade... Done
The following packages will be upgraded:
  ...

If a kernel upgrade was applied, reboot your server before continuing:

sudo reboot

Pro tip: Skipping this step is the number one cause of mysterious compilation errors when building OSSEC from source. Don’t skip it.

Step 2: Install OSSEC Dependencies on Debian 13

OSSEC is a C-based application that compiles from source. You need to install build tools and system libraries before running the installer. Run this command on both the Manager and Agent servers:

sudo apt -y install build-essential make gcc wget tar \
libpcre2-dev zlib1g-dev libssl-dev systemd-dev libsystemd-dev

Here’s what each package does and why it matters:

⚠️ Critical: libsystemd-dev is a Debian 13-specific requirement not documented in older guides. Skipping it causes a compilation error midway through the install.

[IMAGE: Screenshot of terminal showing successful apt install of OSSEC build dependencies on Debian 13]

Step 3: Download OSSEC Source Code

Download the latest OSSEC release directly from the official GitHub repository. At the time of writing, v3.8.0 is the latest stable release.

On the Manager Server

cd /tmp
wget https://github.com/ossec/ossec-hids/archive/refs/tags/3.8.0.tar.gz -O ossec.tar.gz
tar -xzf ossec.tar.gz
cd ossec-hids-3.8.0

What each command does:

• cd /tmp — Moves to the temp directory to keep your home directory clean
• wget ... -O ossec.tar.gz — Downloads the source archive
• tar -xzf ossec.tar.gz — Extracts the archive
• cd ossec-hids-3.8.0 — Enters the extracted source directory

Best practice: Always check the OSSEC GitHub Releases page to confirm the latest version tag before downloading.

Step 4: Install the OSSEC Manager (Server Mode)

The OSSEC Manager is the central server that receives security events from all connected agents and runs the analysis engine. Run the installer from inside the extracted source directory:

sudo ./install.sh

Language and Install Type

(en/br/cn/de/el/es/fr/hu/it/jp/nl/pl/ru/sr/tr) [en]: en

Press Enter, then choose the installation type:

What kind of installation do you want (server, agent, local, hybrid or help)? server

Installation Directory

Choose where to install the OSSEC HIDS [/var/ossec]: /var/ossec

Email Alerts (Optional)

3.1- Do you want e-mail notification? (y/n) [y]: y
- What's your e-mail address? [email protected]
- What's your SMTP server ip/host? localhost

Active Response and Firewall Drop

- Do you want to enable active response? (y/n) [y]: y
- Do you want to enable the firewall-drop response? (y/n) [y]: y
- firewall-drop enabled (local) for levels >= 6

Agent IP Whitelist

- Do you want to add more IPs to the white list? (y/n)? [n]: y
- IPs (space separated): <YOUR_AGENT_IP>

Enter your agent server’s IP address to prevent OSSEC from blocking it. Then press Enter to begin compilation.

Expected Success Output

- System is Debian (Ubuntu or derivative).
- Init script modified to start OSSEC HIDS during boot.
- Configuration finished properly.
- To start OSSEC HIDS:
    /var/ossec/bin/ossec-control start

[IMAGE: Screenshot of OSSEC install.sh wizard showing “server” mode selection on Debian 13 terminal]

Step 5: Configure Firewall and Register an OSSEC Agent

Open OSSEC Ports on the Manager

OSSEC agents communicate with the manager over UDP port 1514. TCP port 1515 is used optionally for enrollment. Open both on the Manager:

sudo ufw allow 1514/udp
sudo ufw allow 1515/tcp
sudo ufw reload

Cloud users: If you’re using AWS, GCP, DigitalOcean, or Hetzner, also open these ports in your provider’s security group or firewall panel.

Register the Agent on the Manager

Run the agent management utility:

sudo /var/ossec/bin/manage_agents

Follow these steps in the interactive menu:

1. Press A to add a new agent
2. Enter a descriptive agent name (e.g., web-server-01)
3. Enter the agent’s IP address
4. Confirm and save

Then extract the agent key:

1. Press E to extract key
2. Select the agent by its ID number
3. Copy the full key string — you’ll need it on the agent server

Agent key information for '001' is:
MDAxIHdlYi1zZXJ2ZXItMDEgMTkyLjE2OC4xLjIwIDM5MjQ4N...

Security note: Treat agent keys like passwords. Never expose them in public repositories or screenshots.

Step 6: Create the OSSEC systemd Service on the Manager

By default, OSSEC does not integrate with systemd. Without a service file, OSSEC will not survive a reboot — a critical gap that catches many users off guard.

Create the service file:

sudo nano /etc/systemd/system/ossec.service

Paste the following content:

[Unit]
Description=OSSEC HIDS Manager
After=network.target

[Service]
Type=forking
ExecStart=/var/ossec/bin/ossec-control start
ExecStop=/var/ossec/bin/ossec-control stop
ExecReload=/var/ossec/bin/ossec-control restart
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Save and exit (Ctrl+O, Enter, Ctrl+X), then enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable ossec
sudo systemctl start ossec

Verify OSSEC Manager is running:

sudo /var/ossec/bin/ossec-control status

Expected Output

ossec-maild        not running...
ossec-execd        is running...
ossec-analysisd    is running...
ossec-logcollector is running...
ossec-syscheckd    is running...
ossec-monitord     is running...

ossec-maild only shows “running” if email alerts are configured. All other daemons should be active.

Step 7: Install and Configure the OSSEC Agent on Debian 13

Switch to your second Debian 13 server — the one acting as the agent.

Install Dependencies and Download OSSEC

sudo apt update
sudo apt -y install build-essential make gcc wget tar \
libpcre2-dev zlib1g-dev libssl-dev systemd-dev libsystemd-dev

cd /tmp
wget https://github.com/ossec/ossec-hids/archive/refs/tags/3.8.0.tar.gz -O ossec.tar.gz
tar -xzf ossec.tar.gz
cd ossec-hids-3.8.0

Run the Installer in Agent Mode

sudo ./install.sh

At the installation type prompt, choose agent:

What kind of installation do you want (server, agent, local, hybrid or help)? agent

When prompted for the manager’s address:

3.1- What's the IP Address or hostname of the OSSEC HIDS server?: <MANAGER_IP>

Import the Agent Key

sudo /var/ossec/bin/manage_agents

1. Press I to import a key
2. Paste the full key string copied from the manager
3. Confirm and exit the utility

Confirm Manager IP in Config

sudo nano /var/ossec/etc/ossec.conf

Verify this block contains the correct manager IP:

<client>
  <server-ip>MANAGER_IP_ADDRESS</server-ip>
</client>

Create systemd Service on the Agent

sudo nano /etc/systemd/system/ossec.service

[Unit]
Description=OSSEC Agent
After=network.target

[Service]
Type=forking
ExecStart=/var/ossec/bin/ossec-control start
ExecStop=/var/ossec/bin/ossec-control stop
ExecReload=/var/ossec/bin/ossec-control restart
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable ossec
sudo systemctl start ossec

Step 8: Verify the Full OSSEC Installation on Debian 13

Check Connected Agents on the Manager

sudo /var/ossec/bin/agent_control -l

Expected Output

OSSEC HIDS agent_control. List of available agents:
   ID: 001, Name: web-server-01, IP: 192.168.1.20, Active

The agent status must show Active. If it shows “Never connected,” the key import or firewall is misconfigured.

Verify Daemon Status on Both Servers

sudo /var/ossec/bin/ossec-control status

All key daemons should show as running:

• ossec-execd — Executes active response actions
• ossec-analysisd — Analyzes incoming log events
• ossec-logcollector — Collects and reads log files
• ossec-syscheckd — Runs file integrity monitoring
• ossec-monitord — Monitors agent connectivity

Basic OSSEC Configuration Tips

The main configuration file lives at /var/ossec/etc/ossec.conf. Always back it up before editing:

sudo cp /var/ossec/etc/ossec.conf /var/ossec/etc/ossec.conf.bak

Key settings to review:

• Email alerts: Set <email_to> and <smtp_server> for real-time notifications
• Custom log monitoring: Add entries under <localfile> for app-specific logs
• Syscheck frequency: Adjust file integrity scan intervals under <syscheck>

After any config change, restart OSSEC:

sudo /var/ossec/bin/ossec-control restart

Troubleshooting Common OSSEC Issues on Debian 13

Even on a clean Debian 13 install, a few issues come up regularly. Here’s how to fix them fast.

Issue 1: Compilation Fails With systemd Errors

Cause: Missing libsystemd-dev package.

sudo apt install libsystemd-dev systemd-dev

Then re-run sudo ./install.sh from the source directory.

Issue 2: Agent Shows “Never Connected” on the Manager

Cause: UDP port 1514 is blocked on the manager’s firewall or cloud security group.

sudo ufw allow 1514/udp && sudo ufw reload

Also verify your cloud provider’s network firewall allows inbound UDP 1514.

Issue 3: OSSEC Doesn’t Start After Reboot

Cause: systemd service was not enabled.

sudo systemctl enable ossec
sudo systemctl start ossec

Issue 4: ossec-analysisd Not Running

Cause: Configuration syntax error in ossec.conf.

sudo tail -f /var/ossec/logs/ossec.log

Fix the XML syntax error in /var/ossec/etc/ossec.conf and restart OSSEC.

Issue 5: High False Positive Alerts Flooding Email

Cause: Default rules flagging routine system activity.

Add exceptions in /var/ossec/rules/local_rules.xml to suppress known-good activity, then restart OSSEC.

[IMAGE: Screenshot of /var/ossec/logs/ossec.log showing OSSEC error output during troubleshooting on Debian 13]

Congratulations! You have successfully installed OSSEC. Thanks for using this tutorial for installing OSSEC on Debian 13 “Trixie” system. For additional help or useful information, we recommend you check the official OSSEC website.