Skip to main content

Introduction

Revision History​

DateAuthorSummary
18-JUN-2022TSgt. BernadotteInitial draft
12-DEC-2022TSgt. BernadotteReformatted
19-MAR-2026CW2 BernadotteConverted to markdown; restructured; added module reference and expanded examples

Purpose​

The purpose of this aid is to provide operators with the knowledge necessary to configure Filebeat (filebeat.yml) to collect log files from Linux and Windows hosts and forward them to the DIP Logstash pipeline. This includes configuring file path inputs, using Filebeat's built-in modules for common applications, and tuning for CPT-relevant log sources such as Zeek, Suricata, and system/audit logs.

End-State​

  • Operators can configure filebeat.inputs to collect logs from arbitrary file paths on Linux and Windows hosts.
  • Operators can enable and configure filebeat.modules for common applications (Zeek, Suricata, Apache, system, auditd, etc.).
  • Operators understand multiline log handling for applications that write multi-line entries.
  • Operators can configure the Logstash output to correctly target the DIP External IP.
  • Operators can build a complete, mission-ready Filebeat config for the log sources present in a target environment.

Requirements​

  1. Filebeat installed on the target host (Linux or Windows).
  2. Access to the filebeat.yml configuration file:
    • Pre-deployment: /opt/tfplenum/agent_pkgs/filebeat/templates/filebeat.yml on the DIP Controller (master config used by TFPlenum agent deployer).
    • Post-deployment: Modify directly on the deployed host, then restart the Filebeat service.
  3. DIP External IP address for output configuration.
  4. Read access to the log file paths being collected (Filebeat runs as a service — ensure the service account has read permissions on target log directories).

Considerations​

  • Filebeat vs. Winlogbeat: Winlogbeat is purpose-built for Windows Event Logs (via the Windows API). Filebeat collects flat log files and stdin/network streams. Use Winlogbeat for Windows Event Log channels; use Filebeat for flat log files, application logs, and Linux system logs.
  • Tune before deployment when possible. Confirm the log file paths are correct for the target OS distribution and application version before deploying.
  • The TFPlenum master config at /opt/tfplenum/agent_pkgs/filebeat/templates/filebeat.yml is what gets pushed to hosts. Modifying it affects all future deployments — test on a single host first.
  • Post-deployment changes require a Filebeat service restart: systemctl restart filebeat (Linux) or Restart-Service filebeat (Windows).
  • The 262COS config omits SSL certificate file references intentionally — if the DIP is rebuilt, certificates are regenerated and cert-pinned agents fail to reconnect. Use ssl.verification_mode: "none" instead.
  • Do not define the same file path across multiple inputs. Filebeat tracks file state (offset, inode) per input — duplicate path definitions cause unexpected behavior and potential data duplication.
  • filebeat.inputs vs filebeat.modules: Modules are pre-configured inputs for common applications with ECS field normalization built in. Prefer modules over raw inputs when a module exists for the application — they produce better-structured data for Kibana.