perf

Commands
- perf stat: obtain event counts
- perf record: record events for later reporting
- perf report: break down events by process, function, etc.
- perf annotate: annotate assembly or source code with event counts
- perf top: see live event count
- perf bench: run different kernel microbenchmarks
perf record -e <tracepoint> -ag to trace usage of a kernel tracepoint
- list of some tracepoints: https://www.kernel.org/doc/html/latest/core-api/tracepoint.html
- perf list | grep Tracepoint to list all supported tracepoints
- -g enables call graph recording
perf stat -e <event> to count instances of any item from perf list
- perf stat (-d) (-p <pid>) to track CPU counter stats
perf record is like perf stat but also allows recording more context, like call-graphs, but also has more overhead
perf trace is an strace replacement with much lower overhead
perf probe <fn> to add a tracepoint against any kernel function
Compile software to use frame pointers to get full stacktraces (CONFIG_FRAME_POINTER=y for the kernel)
- --call-graph dwarf if this isn’t possible, which uses libunwind
Event types/prefixes:
perf sched for scheduler analysis
perf top shows CPU usage by default, but supports any event with -e <event>