suricatavel-agent

suricatavel-agent is an agent that reads the eve.json files generated by Suricata and publishes events to Kafka or HTTP with a reliable approach.

Key features

• Continuous reading of eve.json.
• Adaptive batching of events for efficiency and latency control.
• Local buffer and retries: when transport is unavailable, events are stored in storage/app/suricatavel-agent.sqlite and resent with exponential backoff.
• Persistent Dead Letter Queue (DLQ) for messages exceeding max attempts.
• Periodic heartbeats to indicate status (running, starting, error, shutdown).
• Support for SASL/SSL (Kafka) with configurable certificates and credentials.

Requirements

• Access to Suricata’s eve.json file(s).
• PHP 8.4 with rdkafka extension, if using Kafka.

Installation

1. Configure repository

Debian / Ubuntu

Execute the following commands to add the GPG key and repository source.

# 1. Install prerequisites
 apt-get update
 apt-get install -y ca-certificates curl gpg

# 2. Add GPG Key
curl -fsSL https://suricatavel.org/suricatavel.gpg | \
 gpg --dearmor -o /usr/share/keyrings/suricatavel.gpg

# 3. Add Repository Source
echo "deb [signed-by=/usr/share/keyrings/suricatavel.gpg] https://repository.suricatavel.org/ stable main" | \
 tee /etc/apt/sources.list.d/suricatavel.sources

# 4. Update Index
 apt-get update

RHEL / Rocky Linux

For RPM-based systems (using dnf or yum), add the repository configuration.

# 1. Install prerequisites
 dnf install -y ca-certificates curl

# 2. Add Repository File
 tee /etc/yum.repos.d/suricatavel.repo <<EOF
[suricatavel]
name=Suricatavel Repository
baseurl=https://repository.suricatavel.org/rpm/stable/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://suricatavel.org/suricatavel.gpg
EOF

# 3. Rebuild Cache
 dnf makecache

FreeBSD (OPNSense / pfSense)

For FreeBSD based systems, configure the pkg repository.

# 1. Setup Fingerprints
mkdir -p /usr/local/etc/pkg/fingerprints/suricatavel/trusted
fetch -o /usr/local/etc/pkg/fingerprints/suricatavel/trusted/suricatavel \
  https://repository.suricatavel.org/freebsd/fingerprints/trusted/suricatavel

# 2. Add Repository Configuration
cat <<EOF > /usr/local/etc/pkg/repos/suricatavel.conf
suricatavel: {
  url: "pkg+https://repository.suricatavel.org/freebsd",
  signature_type: "fingerprints",
  fingerprints: "/usr/local/etc/pkg/fingerprints/suricatavel",
  enabled: yes
}
EOF

# 3. Update Index
pkg update

Unifi / Static binary

For Unifi or other appliances without PHP/Composer:

1. Download the latest binary from the Releases page.
2. Upload to appliance: scp suricatavel-agent root@<ip>:/data/
3. Make executable: chmod +x /data/suricatavel-agent
4. Extract Service File: Extract suricatavel-agent.service from the release or builder and copy it to /data/.

2. Install package

Debian / Ubuntu

 apt-get install -y suricatavel-agent

RHEL / Rocky Linux

 dnf install -y suricatavel-agent

FreeBSD (OPNSense / pfSense)

pkg install suricatavel-agent

Unifi / Static binary

No package installation required. The binary is standalone.

Agent registration & transport

If you are using HTTP Transport, you must register the agent to obtain an authentication token.

1. Obtain Registration Token: Get your Registration Token from the Suricatavel dashboard.

2. Run Registration command:

   # Example for standard install
   php artisan agent:register --token=YOUR_TOKEN --url=https://your-suricatavel.com
   
   # Example for static binary (Unifi)
   cd /data
   ./suricatavel-agent php-cli artisan agent:register --token=YOUR_TOKEN --url=https://your-suricatavel.com

3. Verify: This command will save the HTTP_AUTH_TOKEN to your .env file automatically.

Configuration

Typical environment variables (or keys in config/*.php):

• SURICATA_EVE_JSON_PATHS: Single path or comma-separated list of paths to eve.json (default: /var/log/suricata/eve.json).
• AGENT_TRANSPORT_DRIVER: http (default) or kafka.
• AGENT_CONTROLLER_URL: Suricatavel base controller URL (default: https://suricatavel.example.com/).
• HTTP_AUTH_TOKEN: Authentication token for HTTP transport.
• KAFKA_BROKERS: List of brokers, e.g., kafka1:9092,kafka2:9092 (if using Kafka).
• KAFKA_TOPIC: Events topic name (default: suricatavel-events).
• KAFKA_HEARTBEAT_TOPIC: Heartbeats topic name (default: suricatavel-heartbeats).
• Security (if using Kafka): KAFKA_SSL_ENABLED, KAFKA_SSL_CA_LOCATION, KAFKA_SSL_CERTIFICATE_LOCATION, KAFKA_SSL_KEY_LOCATION, KAFKA_SASL_ENABLED, KAFKA_SASL_MECHANISM, KAFKA_SASL_USERNAME, KAFKA_SASL_PASSWORD, etc.
• Buffer and retries: AGENT_BUFFER_MAX_RETRY_ATTEMPTS, AGENT_BUFFER_RETRY_WAIT_TIME (in seconds).
• Heartbeats: AGENT_HEARTBEAT_ENABLED, AGENT_HEARTBEAT_INTERVAL (in seconds).

Usage

Systemd (Linux)

The package installs a Supervisor configuration automatically. To start the agent and enable it to run at boot:

 systemctl enable supervisor
 systemctl start supervisor
 supervisorctl start suricatavel-agent

FreeBSD (OPNSense / pfSense)

Enable and start the service using sysrc and service:

sysrc suricatavel_agent_enable=YES
service suricatavel_agent start

Unifi (Systemd)

On Unifi UDM (UbiOS), use the provided systemd service file for persistence in /data.

# 1. Link the service file (from /data)
ln -s /data/suricatavel-agent.service /etc/systemd/system/suricatavel-agent.service

# 2. Enable and Start
systemctl daemon-reload
systemctl enable suricatavel-agent
systemctl start suricatavel-agent

Running manually (debugging):

php artisan agent:start

Integration with Suricatavel

Suricatavel can consume events sent by the agent from Kafka (kafka.topics.events) or via HTTP. Ensure Suricatavel’s configuration matches the agent’s transport.

Troubleshooting

• Adjust batch size and retry intervals if you observe high latencies or spikes.

Kafka

• Check logs in storage/logs/. You can adjust Kafka debug level in config/kafka.php.
• If Kafka is unavailable, events will be saved in the SQLite buffer; they will be retried automatically upon connectivity recovery.