Flow Collector Lite: Lightweight Network Traffic Capture for Small Teams

Getting Started with Flow Collector Lite — Installation & Setup GuideFlow Collector Lite is a lightweight, easy-to-deploy netflow/IPFIX collection tool designed for small teams and edge locations that need reliable flow capture without the overhead of a full-scale collector. This guide walks you through requirements, installation, initial configuration, common setup options, basic troubleshooting, and recommendations for production use.

What Flow Collector Lite provides

Lightweight flow ingestion for NetFlow, sFlow, IPFIX, and related flow formats.
Low-resource footprint, suitable for single-board computers, virtual machines, and cloud instances.
Simple storage options (local files, SQLite, or optional remote exporters).
Basic CLI and configuration file for fast deployment.

Prerequisites

System requirements

CPU: 1 vCPU minimum; 2+ recommended for moderate volumes.
Memory: 512 MB minimum; 2 GB recommended.
Disk: At least 2 GB free for binary, configs, and flow files; more depending on retention needs.
Network: UDP/TCP ports open for flow protocols (typically UDP 2055, 4739, 6343, 9346) and SSH for remote administration.
OS: Linux (Debian/Ubuntu, CentOS/RHEL, or Alpine) — most examples below use Ubuntu 22.04.

Software

curl, wget, or a package manager (apt/yum).
Optional: systemd for service management; Docker for containerized deployment.

Installation

Below are three common installation methods: prebuilt binary, package manager, and Docker.

1) Prebuilt binary (recommended for simplicity)

Download the latest release:


wget https://example.com/flow-collector-lite/releases/latest/flow-collector-lite-linux-amd64.tar.gz tar -xzf flow-collector-lite-linux-amd64.tar.gz sudo mv flow-collector-lite /usr/local/bin/ sudo chmod +x /usr/local/bin/flow-collector-lite

Create directories for config and data:


sudo mkdir -p /etc/flow-collector-lite sudo mkdir -p /var/lib/flow-collector-lite sudo chown $(whoami):$(whoami) /etc/flow-collector-lite /var/lib/flow-collector-lite

Copy a default config:


cp ./config.example.yaml /etc/flow-collector-lite/config.yaml

2) Package manager / APT (if available)

sudo apt update sudo apt install flow-collector-lite

Package installs binary to /usr/bin and places config in /etc/flow-collector-lite/.

3) Docker (isolated & quick)

docker run -d --name flow-collector-lite    -p 2055:2055/udp -p 4739:4739/udp    -v /opt/flow-collector-lite/config.yaml:/etc/flow-collector-lite/config.yaml:ro    -v /opt/flow-collector-lite/data:/var/lib/flow-collector-lite    ghcr.io/example/flow-collector-lite:latest

Configuration basics

Configuration is YAML-based. Key sections:

listeners: UDP/TCP ports and protocols to receive flows (NetFlow, IPFIX, sFlow).
storage: local_file, sqlite, or remote (e.g., S3 or syslog exporter).
parsing: flow templates and mapping options.
retention: file rotation, max size, and days to keep.
logging: log level and log destination.
metrics: Prometheus endpoint and scrape path.

Example minimal config:

listeners:   - name: netflow-default     protocol: udp     port: 2055     formats: [netflow_v9, ipfix] storage:   type: sqlite   path: /var/lib/flow-collector-lite/flows.db logging:   level: info   file: /var/log/flow-collector-lite/collector.log metrics:   enabled: true   bind: 127.0.0.1:9100

Start and enable service (systemd)

Create a systemd unit at /etc/systemd/system/flow-collector-lite.service:

[Unit] Description=Flow Collector Lite After=network.target [Service] Type=simple ExecStart=/usr/local/bin/flow-collector-lite --config /etc/flow-collector-lite/config.yaml Restart=on-failure User=root Group=root [Install] WantedBy=multi-user.target

Then:

sudo systemctl daemon-reload sudo systemctl enable --now flow-collector-lite sudo journalctl -u flow-collector-lite -f

Verifying collection

Confirm listener is bound:
```
ss -lunp | grep 2055 
```
Use a flow generator (nfdump’s nfprobe, pmacct’s flow-generator, or router NAT) to send test flows to the collector’s IP and port.

Check logs for parsed flows:


tail -f /var/log/flow-collector-lite/collector.log

Query storage (SQLite example):


sqlite3 /var/lib/flow-collector-lite/flows.db "SELECT COUNT(*) FROM flows;"

Basic parsing and templates

NetFlow v9/IPFIX rely on templates. The collector caches templates per exporter; ensure template timeouts are high enough for your environment.
If you see “unknown template” or missing fields, increase template cache size and template refresh intervals in config.

Common setup options

High-volume sites: use SQLite with frequent rotation disabled and a fast SSD; increase ulimit for file descriptors.
Edge/low-resource: local_file storage with small retention and compressed rotation.
Central collector: enable TLS and authentication on any remote exporters; expose Prometheus metrics.

Comparison of storage options:

Storage type	Pros	Cons
local_file	Simple, minimal deps	Not query-friendly, needs rotation
sqlite	Easy queries, lightweight	Less scalable for large volumes
remote (S3/remote DB)	Durable, scalable	Higher latency, complexity

Security considerations

Run the collector behind a firewall and accept flows only from known exporters.
Drop privileges: run as a non-root user with minimal capabilities if possible.
Rotate logs and secure access to stored flow files (they may contain sensitive metadata).
If exposing metrics or management ports, protect with firewall rules or mTLS.

Troubleshooting

No flows arriving: check network (UDP reachability), listener binding, and firewall rules. Use tcpdump:
```
sudo tcpdump -n -i any port 2055 
```
Template errors: increase template cache and enable verbose parsing logs to capture template negotiation.
High CPU: reduce parse verbosity, increase instance resources, or use sampling rate on exporters.
Disk filling: enable retention and compression, or move to remote storage.

Backup and retention

Periodically copy or snapshot /var/lib/flow-collector-lite.
Export important flows to a central storage (S3 or central DB) for long-term analytics.
Use rotation policies: daily or size-based, plus gzip compression for older files.

Example: quick end-to-end test

Start collector on 10.0.0.5, UDP 2055.
From a test host:
```
nfcapd -l -p 2055 -P 10.0.0.5 
```
Confirm collector logs show incoming flows and sqlite count increases.

Production recommendations (short)

Use at least 2 vCPUs and 2 GB RAM for sustained moderate volumes.
Monitor metrics (ingest rate, template hits, dropped flows).
Keep backups of config and flow DB.
Test exporter compatibility (NetFlow v5/v9, IPFIX, sFlow).

If you want, I can generate a ready-to-deploy systemd unit, a Docker Compose file, or a sample config tailored to your environment (exporter types, expected flow rate, retention).