VPC flow logs are records of the IP traffic metadata that passes through the network interfaces of a virtual private cloud. Each record summarizes a flow, typically identified by source and destination address, ports and protocol, together with packet and byte counts, a time window and whether the traffic was accepted or rejected by security rules.
In cloud networking, VPC flow logs are the closest equivalent to the NetFlow and IPFIX telemetry of physical networks. They answer who talked to whom, when and how much, without capturing packet contents, which makes them a standard source for security monitoring, troubleshooting and cost analysis.
How VPC flow logs capture traffic
Flow logs are produced by the cloud provider’s virtual network layer, not by agents inside virtual machines. The hypervisor or network offload hardware observes packets at each virtual interface, groups them into flows by the five-tuple of source address, destination address, source port, destination port and protocol, and emits one record per flow for each aggregation interval.
Logging can usually be enabled at three scopes: the whole VPC, a single subnet or an individual network interface. Records are delivered in batches to a log service, an object storage bucket or a streaming pipeline, so they arrive minutes after the traffic, not in real time. They are therefore suited to analysis and alerting with some delay, not to inline blocking.
The following table shows the fields found in a typical flow record and what each one tells an analyst.
| Field | Content | Analytical use |
|---|---|---|
| Interface ID | Virtual network interface that saw the flow | Maps traffic to an instance or service |
| Source and destination address | Private or public IP addresses | Identifies peers and external hosts |
| Source and destination port | Transport-layer ports | Reveals the service being accessed |
| Protocol | IANA protocol number such as 6 for TCP | Separates TCP, UDP and ICMP traffic |
| Packets and bytes | Volume within the interval | Finds heavy talkers and data transfer |
| Start and end time | Boundaries of the aggregation window | Builds timelines during investigations |
| Action | ACCEPT or REJECT | Shows effect of security groups and network ACLs |
Many providers offer custom record formats with extra fields such as VPC and subnet identifiers, TCP flags, traffic direction, the original addresses behind NAT and the region or availability zone. Adding fields improves analysis but increases log volume and cost.
Implementations across major cloud platforms
The concept is shared across providers, but sampling, intervals and destinations differ in ways that matter for accuracy and cost.
| Platform | Feature | Notable characteristics |
|---|---|---|
| AWS | VPC Flow Logs | Unsampled per interface; 1-minute or 10-minute aggregation |
| Google Cloud | VPC Flow Logs | Sampled per subnet; configurable interval and sampling rate |
| Microsoft Azure | Virtual network flow logs | Successor to NSG flow logs; stored in a storage account |
| Other clouds | Provider-specific flow logs | Similar five-tuple model with varying fields |
The distinction between sampled and unsampled logging is important. A sampled log is cheaper and adequate for trend analysis, but it can miss short, low-volume flows such as a single scan or a brief beacon. Unsampled logs are better for forensics but grow quickly on busy networks.
What flow logs show and what they miss
VPC flow logs record metadata only. They do not contain payloads, URLs, DNS query names or user identities, so they cannot show what was transferred, only that a transfer happened. Some platform traffic is also excluded, for example traffic to the provider’s own DNS resolver, the instance metadata endpoint or DHCP, depending on the platform.
| Telemetry source | Detail level | Typical purpose |
|---|---|---|
| VPC flow logs | Per-flow metadata | Broad visibility at low cost |
| Traffic mirroring or packet capture | Full packets | Deep inspection and intrusion detection |
| DNS query logs | Domain names requested | Detecting malicious domains |
| Load balancer access logs | HTTP requests and responses | Application-level analysis |
| Cloud firewall logs | Rule matches and decisions | Auditing specific policies |
Another subtlety is stateful filtering. Because security groups are stateful, return traffic for an allowed connection is accepted automatically, while stateless network ACLs evaluate each direction separately. A REJECT in the log therefore needs to be interpreted against which control made the decision, which some providers expose only through additional fields.
Common uses in operations and security
Flow logs serve three main audiences: network engineers, security teams and cost owners. Each reads the same records for a different question.
- Connectivity troubleshooting: a REJECT between two subnets quickly points to a missing security group or route rule.
- Threat detection: unexpected outbound connections, port scans or traffic to known malicious addresses become visible.
- Incident investigation: flow history shows which hosts a compromised instance contacted and how much data left.
- Segmentation design: observed flows reveal real dependencies before tightening security groups.
- Cost analysis: high byte counts through NAT gateways or across zones explain unexpected data transfer charges.
Records are commonly queried with SQL engines over object storage or forwarded to a SIEM, where they are correlated with DNS logs, identity events and endpoint alerts.
Configuration choices and cost control
The cost of VPC flow logs comes mostly from ingestion and storage of log volume, which scales with traffic, record format and retention. Careful configuration keeps visibility while avoiding bills that rival the workloads being monitored.
| Decision | Recommended practice | Reason |
|---|---|---|
| Scope | Enable at VPC level for production | New interfaces are covered automatically |
| Traffic filter | Capture all traffic, not only rejects | Accepted flows matter in investigations |
| Destination | Object storage for retention, stream for alerting | Cheaper long-term storage |
| File format | Columnar format with partitioning | Faster and cheaper queries |
| Retention | Align with incident and compliance needs | Avoids storing logs nobody reads |
A frequent mistake is logging only rejected traffic. It looks efficient, but most attacks that succeed appear as accepted flows, so a rejects-only configuration leaves investigators blind when they need the data most.
Put simply
VPC flow logs are like the call records of a phone company. They do not record what was said. They do record which number called which, when the call started, how long it lasted and whether it was blocked.
For a cloud network, that means a list of which computers talked to which, on what port and how much data moved. This is often enough to spot a problem or trace an intruder.
In practice
A software company receives an alert that its monthly data transfer bill has tripled. The application team insists nothing changed. The cloud operations engineer enables VPC flow logs on the production VPC with a 1-minute aggregation interval and sends them to object storage.
A day of records, queried by bytes per source and destination, shows a single analytics instance sending several hundred gigabytes per day through the NAT gateway to an external storage endpoint. The cause turns out to be a misconfigured backup job that copied data out of the cloud instead of to a bucket in the same region.
After the job is corrected and a gateway endpoint for object storage is added, the transfer volume drops back to normal. The team keeps flow logs enabled and adds an alert on unusual outbound volume, so a similar mistake would be noticed within hours.