Collecting Data

eBPF and BCC provide powerful tools for capturing tracepoint events from the Linux kernel and drivers. Tracepoints are predefined event sources in the kernel, such as function calls or system call entries, and BCC simplifies their collection and analysis through high-level utilities. This section demonstrates how to use BCC to gather and process tracepoint data effectively.

Using BCC Tools to Capture Tracepoint Data¶

BCC includes the trace and tracepoint commands for interacting with kernel tracepoints. These tools abstract the complexity of eBPF program compilation and event handling, allowing users to focus on filtering and analysis.

Basic Tracepoint Collection¶

To capture tracepoint events, use the trace command with the -p option to specify a process ID (PID) or --all to trace all processes:

# Trace all processes for sys_enter_open events
sudo trace -p all -e sys:sys_enter_open

# Trace a specific PID (e.g., PID 1234) for sched_switch events
sudo trace -p 1234 -e sched:sched_switch

The -e flag filters events by name (e.g., sys:sys_enter_open). For a list of available tracepoints, use:

sudo trace -l

Tracing Driver-Specific Events¶

For kernel modules or drivers, use the tracepoint command to target specific events. For example, to trace DMA operations in a network driver:

sudo tracepoint -p all -e net:net_dev_xmit

This command captures events from the net_dev_xmit tracepoint, which is triggered when a network device transmits data.

Filtering and Analyzing Tracepoint Data¶

BCC provides utilities like stat, histogram, and perf to process collected tracepoint data. These tools aggregate events, calculate statistics, and visualize trends.

Aggregating Event Counts¶

Use stat to count occurrences of specific events over time:

# Count sys_open calls per second
sudo trace -p all -e sys:sys_open | sudo stat -c "%t %s %c" sys_open

This outputs timestamps and counts of sys_open events, helping identify patterns or anomalies.

Generating Histograms¶

The histogram tool creates visualizations of event distributions. For example, to analyze latency of sys_open calls:

sudo trace -p all -e sys:sys_open | sudo histogram -c "delta_us" -x "delta_us" -y "count"

This generates a histogram showing the distribution of open() call latencies.

Exporting Data for External Analysis¶

Use perf to export tracepoint data for further analysis with tools like perf report or flamegraph:

sudo trace -p all -e sched:sched_switch --output trace.data
perf report --input trace.data

Key Takeaways¶

Use trace and tracepoint commands to capture kernel and driver events without writing eBPF programs manually.
Filter events with -e and analyze data using stat, histogram, or perf for insights into system behavior.
BCC abstracts eBPF complexity, enabling rapid deployment of tracepoint-based monitoring and troubleshooting workflows.