Filtering Data

Filtering and Aggregating Tracepoint Data

When working with tracepoint data in eBPF, filtering and aggregation are essential to reduce noise and extract meaningful metrics. BPF programs and BCC tools provide flexible mechanisms to process events in kernel space (for low-latency filtering) or user space (for aggregation and visualization). Below are core techniques for handling tracepoint data.

---

Filtering with BPF Programs

BPF programs can filter tracepoint events at the kernel level, avoiding the overhead of collecting all data and processing it in user space. This is achieved using bpf_map structures to store filtered results or applying conditional logic in the BPF code.

Example: Filtering by Function Arguments

A BPF program can filter events based on specific arguments. For instance, to count sys_open calls with a filename containing /tmp/:

// BPF program (pseudo-code)
// Note: This BPF program requires a GPL license and SEC("tracepoint") annotation for the function.
struct event {
    char filename[256];
};
BPF_HASH(counts, char[256], __u64);

int filter(struct pt_regs *ctx) {
    struct event *evt = (struct event *)ctx->di;
    if (strstr(evt->filename, "/tmp/")) {
        __u64 *count = counts.lookup(evt->filename);
        if (count) *count += 1;
        else counts.insert(evt->filename, 1);
    }
    return 0;
}

This program uses a hash map to track counts of filenames matching the filter.

BCC Tool: trace with Filters

BCC abstracts BPF complexity with tools like trace, which allows filtering via regular expressions or keywords:

# Filter sys_open calls with "/tmp/"
sudo bcc trace -t sys_open -p $(pidof your_process) | grep '/tmp/'

This command uses BCC's internal BPF programs to filter events at runtime.

---

Aggregating Data with BPF Maps

BPF maps enable aggregation of tracepoint data by storing statistics in kernel space. Common use cases include counting events, tracking durations, or summarizing metrics.

Example: Counting Function Calls

A BPF program can use a BPF_HASH map to count how often a function is called. Note that this example retrieves the function name from tracepoint arguments rather than using bpf_get_current_comm():

// BPF program (pseudo-code)
// Note: This BPF program requires a GPL license and SEC("tracepoint") annotation for the function.
struct event {
    char func_name[128];
};
BPF_HASH(call_counts, char[128], __u64);

int count_calls(struct pt_regs *ctx) {
    struct event *evt = (struct event *)ctx->di;
    char func_name[128];
    snprintf(func_name, sizeof(func_name), "%s", evt->func_name);
    __u64 *count = call_counts.lookup(func_name);
    if (count) *count += 1;
    else call_counts.insert(func_name, 1);
    return 0;
}

This program increments a counter for each unique function name.

BCC Tool: stat for Aggregation

BCC's stat tool provides high-level aggregation of tracepoint data:

# Aggregate sys_call_count by function name
sudo bcc stat -t sys_call_count -p $(pidof your_process) --percpu

This command outputs statistics like total calls per function, leveraging BPF under the hood.

---

Combining Filtering and Aggregation

For complex analysis, combine kernel-space filtering with user-space aggregation. For example: 1. Use a BPF program to filter and store relevant data in a map. 2. Use BCC tools to export the map data for further processing.

Example: Filtering and Aggregating in BCC

# Filter sys_open calls and aggregate by filename
sudo bcc trace -t sys_open -p $(pidof your_process) | \
    awk '/\/tmp\// {print $2}' | sort | uniq -c

This pipeline filters lines containing /tmp/, counts unique filenames, and outputs the result.

---

Key takeaways

• Kernel-space filtering (via BPF) reduces overhead by processing data before it reaches user space.
• BPF maps enable efficient aggregation of metrics like counts or durations.
• BCC tools like trace, stat, and hist simplify filtering and aggregation for users.
• Combining BPF programs with BCC tools allows precise, scalable analysis of tracepoint data.