This is Part 1 of a two-part series on leveraging firewall data in Elastic Security. In this post, we cover the fundamentals of firewall logs, how to collect them, and how to begin exploring your network data visually.
The network firewall is one of the most critical security controls in a network. It enforces security policies by inspecting and controlling traffic between network segments, while generating logs that record allowed and denied connections. This article explores why firewall logs are a valuable supplement to other data sources, such as endpoint telemetry, and provides an overview of what firewall logs contain and how security teams can use them effectively.
We will cover:
- The importance of network firewall logs
- What’s inside network firewall logs & how those data help cybersecurity
- Collecting network firewall logs with Elastic agent
- Exploring your data on the Elastic Security Network Page
The Importance of Network Firewall Logs
A network firewall acts as a gatekeeper, filtering traffic based on organizational rules and policies. For instance, it might permit one system to use RDP to connect to another while blocking similar access from other systems. In cloud environments, virtual firewalls enforce security group rules, network ACLs, and policy boundaries across VPCs, subnets, and regions thus, offering visibility into east-west and north-south traffic across your cloud estate.
Modern Firewalls go beyond traditional filtering by incorporating deep packet inspection, application awareness, and threat intelligence, among others.
Positioned strategically, firewalls capture logs that provide insights into inter-zone and intra-zone communication. For instance:
- North-south traffic is data movement between an internal network and external entities like the Internet or cloud services. It is typically monitored by firewalls and security controls to prevent external threats.
- East-west traffic refers to communication within a network, such as between servers, endpoints, or applications inside an organization. It is crucial for internal operations and requires lateral movement detection for security.
By analyzing these logs, security teams gain critical insights into traffic patterns, rule enforcement, and potential threats.
What's to Look Out for in Network Firewall Logs?
Firewall logs contain detailed records of network activity and are packed with information useful for tracking, monitoring, and analyzing traffic patterns, in addition to identifying security events and potential threats. These logs are related to packet filtering and traffic control, capturing allowed and denied traffic, NAT translations, and access control decisions.
The following is a list of key fields that provide the "ground truth" for your network. Please note that the parenthesis contain the equivalent ECS fields.
-
Timestamp (@timestamp): This is the chronological anchor of firewall logs. It helps analysts correlate sequences of events across different devices and networks. For example, if an analyst identifies a suspicious connection, they can trace back the actions preceding or following it to build a precise incident timeline.
-
Source and Destination IP (source.ip, destination.ip): These identify the origin and target of the traffic. While seemingly simple, directionality is a critical distinction in firewall rulesets. Source IPs help identify malicious external origins or internal systems attempting brute-force attacks, while destination IPs help flag when high-value assets, such as a sensitive database, are being targeted.
-
Source and Destination Port (source.port, destination.port): Attackers often target specific services. While source ports are often dynamic, destination ports tell you what service is being probed. High-frequency connections to common services (like 80/HTTP or 443/HTTPS) or high-risk ports (like 22/SSH) can be the first indicator of unauthorized access or web-based attacks.
-
Protocol (network.transport): Analyzing usage of protocols like TCP, UDP, or ICMP helps identify specific attack types. For instance, unusual ICMP patterns might signal a ping sweep or a denial-of-service (DoS) attempt.
-
Action and Rule Identifiers (event.action or event.outcome, rule.name or rule.id): Understanding whether a firewall allowed or blocked a connection is vital. By identifying the specific Rule Identifier, analysts can see which policy was responsible. This is essential for finding misconfigured rules that might be unintentionally exposing the network to attacks.
-
Traffic Volume (source.bytes, destination.bytes, network.bytes): These fields are primary indicators for data exfiltration. Sudden spikes in volume or large transfers to an external destination are often the "early warning" for data theft or malware beaconing.
-
NAT Info (source.nat.ip, destination.nat.ip): In complex environments where Network Address Translation (NAT) is involved, these fields are crucial for "unmasking" the actual internal systems involved. Without this, tracing a suspicious connection back to a specific internal host can be nearly impossible. This is key especially for north-south type of traffic.
-
Application Info (network.application): Next-Generation Firewalls (NGFWs) go beyond ports to identify the actual application (e.g., Skype, BitTorrent, or HTTP). This allows analysts to detect unauthorized applications that might be masking their traffic on standard ports, signaling potential insider threats, lateral movement, or the use of high-risk peer-to-peer software.
-
Interface Info (observer.ingress.interface.name, observer.egress.interface.name): Knowing which physical or virtual interface the traffic passed through (e.g., WAN vs. LAN) helps analysts understand which network segments are involved. Traffic crossing internal interfaces is a key indicator of malware propagation or lateral movement.
Note: Some integrations might have these fields labeled differently.
Collecting firewall logs with Elastic Security
Elastic makes it easy to collect network firewall logs. This guide describes how to use Elastic Agent and Fleet for firewall log collection. There are other ways Elastic can allow you to collect network logs, such as using Logstash. In cloud environments, you can also ingest logs directly from object storage (like AWS S3 or Azure Blob). This approach is useful for environments where firewalls log to a centralized store rather than stream data directly.
To effectively collect and analyze network firewall logs using Elastic Security, follow these steps:
- Configure Log Forwarding: Set up your firewall to forward logs to Elastic Agent.
- Syslog Configuration (or similar): Typically, you will direct your firewall to send Syslog data to the host that has the Elastic Agent, specifying the appropriate IP address and port.
- Elastic Agent Setup: Install and configure Elastic Agent on a syslog server, edge server, or similar log collector to receive and process the logs.
- Utilize the relevant Elastic Integrations: Elastic offers integrations tailored for various firewalls, such as:
- Palo Alto Next-Gen firewall
- Fortinet FortiGate firewall
- Check Point
- Cisco ASA
- AWS Network Firewall
- Azure Firewall
- GCP Firewall, among others.
- Ingest Logs into Elastic Security: Ensure that the logs are ingested into Elasticsearch, making them accessible in Elastic Security for analysis and visualization. Elastic also enriches ingested firewall logs with helpful context such as geolocation, IP-to-hostname mapping, threat intelligence matches, and even business metadata making investigations faster and more informed.
By following these steps, you can effectively collect, process, and analyze network firewall logs within Elastic.
Exploring Your Data: The Elastic Security Network Page
Once your firewall logs are flowing into Elastic, you can move from collection to exploration. The Network Page in Elastic Security is your central hub for visualizing and investigating aggregated network data, including firewall data.
Instead of just looking at raw logs, this page provides key network activity metrics in an interactive map and a series of data tables.
Key features of the Network page include:
- Interactive Map: Get an immediate visual overview of your network traffic. You can see source and destination points mapped geographically, helping you instantly spot unusual connections, like an internal server communicating with an IP in a country you don't do business with.
- Drill-down Widgets: Interactive widgets allow you to quickly find baselines and outliers. You can see top talkers for:
- Network Events
- DNS Queries
- TLS Handshakes
- Unique Private IPs
- Focused Data Tabs: The page includes tabs to pivot your investigation into specific data types, such as:
- Flows: See source and destination IP addresses and countries.
- DNS: Analyze all DNS network queries.
- HTTP: Inspect received HTTP requests.
- TLS: Investigate handshake details.
- Timeline Integration: You can drag and drop items of interest—like a suspicious IP address or host name—directly from the Network page into Timeline for deeper investigation and correlation.
Using this page, you can start to answer foundational questions like, "What is normal traffic for my network?" and "Which external IPs are my internal hosts communicating with most?" This visual exploration is the first step before moving into automated detection.
Start Exploring Your Network Data
In this post, we've covered the fundamentals: why firewall logs are critical, what's inside them, how to ingest them using Elastic Agent, and how to begin visually exploring that data on the Network page.
In Part 2, we'll build on this foundation and move from exploration to active threat detection. We will cover how to use Elastic Security’s detection rules to automatically find network-native threats like reconnaissance, C2, and data exfiltration, as well as how to hunt for advanced lateral movement by correlating with other data such as endpoint data and other telemetry.
Ready to turn your own firewall logs into actionable insights?
- New to Elastic? Start your free 14-day trial of Elastic Cloud to see the Network Page in action.
- Already an Elastic user? Head to the Integrations app in Kibana, add your firewall's integration, and start exploring your network data today.
The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.