Filtering Trace Data

Intel® Trace Analyzer and Collector User and Reference Guide

Download PDF

ID 767272

Date 3/31/2023

Version 2021.10

Public

Visible to Intel only — GUID: GUID-A3332EF6-3F97-45FB-AB3C-2CA9B38FB52F

View Details

Filtering Trace Data

Filtering in Intel® Trace Collector applies specified filters to the trace collection process. This directly reduces the amount of data collected. The filter rules can be defined in a configuration file, in the environment variables or as command line arguments (see Configuring Intel® Trace Collector for details). Filters are evaluated in the order they are listed, and all matching is case-insensitive. For filtering by collective and point-to-point MPI operations the corresponding mpirun/mpiexec options are also available.

In the following discussion, items within angle brackets (< and >) are placeholders for actual values, optional items are put within square brackets ([ and ]), and alternatives are separated by a vertical bar |.

Filtering by Collective and P2P Operations

Use the following mpirun/mpiexec options at runtime:

-trace-collectives – to collect information only about collective operations
-trace-pt2pt – to collect information only about point-to-point operations

NOTE:

These options are mutually exclusive, use only one option at a time.

An example command line for tracing collective operations may look as follows:

$ mpirun -trace -n 4 -trace-collectives ./myApp

These options are special cases of filtering by specific functions discussed below. They use predefined configuration files to collect the appropriate operation types. The options override the VT_CONFIG environment variable, so you cannot use your own configuration files when using them. If you need to set additional filters, you can set them through individual environment variables.

Basic Function Filtering

Function filtering is accomplished by defining the STATE, SYMBOL and ACTIVITY options. Each option accepts the pattern matching the filtered function, and the filtering rule. The formal definition is as follows:

STATE|SYMBOL|ACTIVITY <PATTERN> <RULE>

The general option is STATE, while SYMBOL and ACTIVITY are its replacers:

SYMBOL filters functions by their names, regardless of the class. The following definitions are considered equal:
```
SYMBOL <PATTERN> <RULE>
STATE **:<PATTERN> <RULE>
```
ACTIVITY filters functions by their class names. The following definitions are considered equal:
```
ACTIVITY <PATTERN> <RULE>
STATE <PATTERN>:* <RULE>
```
STATE can filter functions both by names and their class names.

Pattern should match the name of the function for filtering. You can use the following wildcards:

Wildcard	Description
*	Any number of characters, excluding ":"
**	Any number of characters, including ":"
?	A single character
[ ]	A list of characters

For example the following definition will filter out all functions containing the word send, regardless of the class:

STATE **send* OFF

The basic filter rule should contain one of the following entries:

<RULE> = ON | OFF | <trace level> | <skip level>:<trace level>

The <trace level> value defines how far up the call stack is traced. The <skip level> value defines how many levels to skip while going up the call stack. This is useful if a function is called within a library, and the library stack can be ignored. Specifying ON turns on tracing with a <trace level> of 1 and a <skip level> of 0, and OFF turns off tracing completely (this is not equivalent to 0:0).

Example:

In the example below the following events will be traced: all functions in the class Application, all MPI send functions except MPI_Bsend(), and all receive, test and wait functions. All other MPI functions will not be traced.

# disable all MPI functions
ACTIVITY MPI OFF
# enable all send functions in MPI
STATE MPI:*send ON
# except MPI_Bsend
SYMBOL MPI_bsend OFF
# enable receive functions
SYMBOL MPI_recv ON
# and all test functions
SYMBOL MPI_test* ON
# and all wait functions, tracing four call levels
SYMBOL MPI_wait* 4
# enable all functions within the Application class
ACTIVITY Application 0

Advanced Function Filtering

For function filtering a finer control is also available. Here is a list of additional filter rule entries, which can be used along with the basic rule in any combination:

<ENTRYTRIGGER> | <EXITTRIGGER> | <COUNTERSTATE> | <FOLDING> | <CALLER>

Here is the specification for each filter entry available:

Entry/Exit Trigger:

<ENTRYTRIGGER> = entry <TRIGGER>
<EXITTRIGGER> = exit <TRIGGER>

Activate a trigger on entry/exit for the matching pattern.

<TRIGGER> = [<TRIPLET>] <ACTION> [<ACTION>]

Triggers define a set of actions over a set of processes (triplets, see Filtering by Ranks below for definition).

<ACTION> = traceon | traceoff | restore | none | begin_scope <SCOPE_NAME> | end_scope <SCOPE_NAME>

The action defines what happens to tracing. Using traceon or traceoff turns tracing on or off, respectively. begin_scope and end_scope start or end the named scope.

<SCOPE_NAME> = [<class name as string>:]<scope name as string>

A scope is a user-defined region in the program. See Defining and Recording Scopes.

Counter State:

<COUNTERSTATE> = counteron | counteroff

Counter state turns on or off sampling for the matching pattern. By default, all enabled counters are sampled at every state change. There is no method for controlling which counters are sampled.

Folding:

<FOLDING> = fold | unfold

Enabling folding for a function disables tracing of any functions called by that function. By default, all functions are unfolded.

Caller:

<CALLER> = caller <PATTERN>

Specifying the caller enables tracing only for functions called by the functions matching the pattern.

For details on use of the FOLDING and CALLER keywords, see Tracing Library Calls.

Filtering by Ranks

Besides filtering by functions, you can also filter the trace data by ranks in MPI_COMM_WORLD using the PROCESS configuration option. Its value is a comma separated list of Fortran 90-style triplets. The formal definition is as follows:

PROCESS <TRIPLET>[,<TRIPLET>,...] on | off

Triplet definition is as follows:

<TRIPLET> = <LOWER-BOUND>[:<UPPER-BOUND>[:<INCREMENT>] ]

The default value for <UPPER-BOUND> is the size of MPI_COMM_WORLD (N) and the default value for <INCREMENT> is 1.

For example, to trace only even ranks and rank 1 use the following triplets: 0:N:2,1:1:1, where N is the total number of processes. All processes are enabled by default, so you have to disable all of them first (PROCESS 0:N OFF) before enabling a certain subset again. For SMP clusters, you can also use the CLUSTER option to filter for particular SMP nodes.

Parent topic: Trace Your Applications

Select Your Language

Using Intel.com Search

Quick Links

Recent Searches

Advanced Search

Only search in

Intel® Trace Analyzer and Collector User and Reference Guide

Filtering Trace Data

Filtering by Collective and P2P Operations

Basic Function Filtering

Advanced Function Filtering

Filtering by Ranks