Intel® Trace Analyzer and Collector User and Reference Guide

Download PDF
ID 767272
Date 3/31/2023
Public
Document Table of Contents
Document Table of Contents x
Intel® Trace Analyzer and Collector User and Reference Guide
Intel® Trace Analyzer and Collector User and Reference Guide x
Introduction Install and Set Up Intel® Trace Analyzer and Collector Trace Your Applications Analyze Your Applications Intel® Trace Collector Reference Intel® Trace Analyzer Reference Notices and Disclaimers
Introduction x
Notational Conventions Get Help
Trace Your Applications x
Tracing Conventional MPI Applications Tracing Failing MPI Applications Tracing OpenSHMEM* Applications Tracing MPI File IO Handling of Communicator Names Tracing MPI Load Imbalance Tracing User Defined Events Configuring the Collector Filtering Trace Data Recording OpenMP* Regions Information Tracing System Calls (Linux* OS) Collecting Lightweight Statistics Recording Source Location Information Recording Hardware Performance Information (Linux* OS) Recording Operating System Counters Tracing Library Calls Correctness Checking Tracing Distributed Non-MPI Applications
Correctness Checking x
Correctness Checking of MPI Applications Running with Valgrind* (Linux* OS) Configuring Error Checks Analyzing the Results Debugger Integration
Debugger Integration x
TotalView* Debugger GNU* Symbolic Debugger Allinea* Distributed Debugging Tool* (DDT*)
Analyze Your Applications x
Starting Intel® Trace Analyzer Intel Trace Analyzer Graphical User Interface Navigating Timelines Concepts Viewing Correctness Checking Reports Comparing Two Trace Files Interoperability with Intel® VTune™ Profiler and Intel® Advisor OpenMP* Regions Display Support OTF2 Format Support
Navigating Timelines x
Zoom Stack
Concepts x
Level of Detail Aggregation Advanced Aggregation Tagging and Filtering
Viewing Correctness Checking Reports x
Event Timeline Correctness Checking Reports Qualitative Timeline Correctness Checking Reports Detailed Dialog
Comparing Two Trace Files x
Mappings in Comparison Views Comparison Charts
Mappings in Comparison Views x
Mapping of Processes Mapping of Functions
Comparison Charts x
Comparison Function Profile Comparison Message Profile Comparison Collective Operations Profile
Intel® Trace Collector Reference x
API Reference Configuration Reference Correctness Checking Errors Structured Tracefile Format stftool Utility Time Stamping Secure Loading of Dynamic Link Libraries* on Windows* OS
API Reference x
Initialization, Termination and Control Defining and Recording Source Locations Defining and Recording Functions or Regions Defining and Recording Scopes Defining Groups of Processes Defining and Recording Counters Recording Communication Events Additional API Calls in libVTcs C++ API
Initialization, Termination and Control x
VT_initialize VT_finalize VT_getrank VT_registerthread VT_registernamed VT_registerprefixed VT_getthrank VT_traceon VT_traceoff VT_tracestate VT_symstate VT_flush VT_timestamp VT_timestart VT_setfinalizecallback VT_getdescription VT_countsetcallback
Defining and Recording Functions or Regions x
New Interface Old Interface State Changes VT_begin VT_beginl VT_end VT_endl VT_enter VT_leave VT_enterstate VT_wakeup
C++ API x
VT_FuncDef Class Reference VT_SclDef Class Reference VT_Function Class Reference VT_Region Class Reference
Configuration Reference x
Configuration File Format Protocol File Configuration Options
Configuration Options x
ACTIVITY ALTSTACK AUTOFLUSH CHECK CHECK-LEAK-REPORT-SIZE CHECK-MAX-DATATYPES CHECK-MAX-ERRORS CHECK-MAX-PENDING CHECK-MAX-REPORTS CHECK-MAX-REQUESTS CHECK-SUPPRESSION-LIMIT CHECK-TIMEOUT CHECK-TRACING CLUSTER COMPRESS-RAW-DATA COUNTER CURRENT-DIR DEADLOCK-TIMEOUT DEADLOCK-WARNING DEMANGLE DETAILED-STATES ENTER-USERCODE ENVIRONMENT EXTENDED-VTF FLUSH-PID FLUSH-PREFIX GROUP HANDLE-SIGNALS INTERNAL-MPI KEEP-RAW-EVENTS LOGFILE-FORMAT LOGFILE-NAME LOGFILE-PREFIX LOGFILE-RANK MEM-BLOCKSIZE MEM-FLUSHBLOCKS MEM-INFO MEM-MAXBLOCKS MEM-MINBLOCKS MEM-OVERWRITE NMCMD OS-COUNTER-DELAY PCTRACE PCTRACE-CACHE PCTRACE-FAST PLUGIN PROCESS PROGNAME PROTOFILE-NAME STATISTICS STATE STF-PROCS-PER-FILE STF-USE-HW-STRUCTURE STOPFILE-NAME SYMBOL SYNC-MAX-DURATION SYNC-MAX-MESSAGES SYNC-PERIOD SYNCED-CLUSTER SYNCED-HOST TIME-WINDOWS (Experimental) TIMER TIMER-SKIP UNIFY-COUNTERS UNIFY-GROUPS UNIFY-SCLS UNIFY-SYMBOLS VERBOSE VT_START_PAUSED VT_COMPRESS_TRACE
Correctness Checking Errors x
Supported Errors How the Collection Works
How the Collection Works x
Parameter Checking Premature Exit Overlapping Memory Detecting Illegal Buffer Modifications Buffer Given to MPI Cannot Be Read or Written Distributed Memory Checking Illegal Memory Access Request Handling Datatype Handling Buffered Sends Deadlocks Checking Message Transmission Datatype Mismatches Data Modified during Transmission Checking Collective Operations Freeing Communicators
Structured Tracefile Format x
STF Components Single-File STF Configuring STF
stftool Utility x
stftool Utility Options Expanded ASCII output of STF Files
Time Stamping x
Clock Synchronization Choosing a Timer
Choosing a Timer x
gettimeofday/_ftime QueryPerformanceCounter CPU Cycle Counter Normalized CPU Cycle Counter MPI_Wtime() High Precision Event Timers POSIX* clock_gettime
Intel® Trace Analyzer Reference x
Graphical User Interface Reference Intel® Trace Analyzer Command Line Interface Reference Filter Expression Grammar otf2-to-stf Utility
Graphical User Interface Reference x
Welcome Page Summary Page Main Menu Bar View Menu Bar View Bars Charts Dialogs Settings
Main Menu Bar x
File Menu Options Menu Project Menu  Windows Menu Help Menu
View Menu Bar x
View Charts Navigate Advanced Layout Comparison Menu
Advanced x
Tagging Specific Events Filtering Events Simulating Ideal Communication Checking Application Imbalance Aggregating Results Aggregating Functions Creating Command Line for Intel® VTune™ Profiler and Intel® Advisor
View Bars x
Toolbar Trace Map Status Bar
Charts x
Event Timeline Qualitative Timeline Quantitative Timeline Counter Timeline Function Profile Message Profile Collective Operations Profile Performance Assistant Common Chart Features
Event Timeline x
Context Menu Filtering and Tagging
Qualitative Timeline x
Context Menu Filtering and Tagging
Quantitative Timeline x
Context Menu Filtering and Tagging
Counter Timeline x
Context Menu Filtering and Tagging
Function Profile x
Flat Profile Load Balance Call Tree Call Graph Context Menu Filtering and Tagging Function Profile Settings
Message Profile x
Context Menu Filtering and Tagging Aggregation Message Profile Settings
Collective Operations Profile x
Context Menu Filtering and Tagging Collective Operations Profile Settings
Dialogs x
Process Aggregation Function Aggregation Function Group Color Editor Filtering Dialog Box Tagging Dialog Box Idealization Dialog Box Imbalance Diagram Dialog Box Trace Merge Dialog Box  Details Dialog Box Source View Dialog Time Interval Selection Configuration Dialogs Find Dialog Box Command line for Intel® VTune™ Profiler and Intel® Advisor Dialog Box OTF2 to STF Conversion Dialog Box Configuration Assistant
Process Aggregation x
Comparison Mode
Function Aggregation x
Comparison Mode
Filtering Dialog Box x
Building Filter Expressions Using Graphical Interface Building Filter Expressions Manually Filter Expressions in Comparison Mode
Details Dialog Box x
Detailed Attributes of Function Events Detailed Attributes of Message Events Detailed Attributes of Collective Operation Events
Configuration Dialogs x
Load Configuration File Dialog Edit Configuration File Dialog
Settings x
Preferences Font Settings Number Formatting Settings
Preferences x
General Preferences Tracefile Preferences Event Timeline Settings Qualitative Timeline Settings Quantitative Timeline Settings Counter Timeline Settings
Notices and Disclaimers x
Appendix A Copyright and Licenses
Intel® Trace Analyzer and Collector User and Reference Guide

State Changes

The following functions take a state handle defined with either the new or old interface. Handles defined with the old interface incur a higher overhead in these functions, because they need to be mapped to the real internal handles. Therefore it is better to use the new interface.

Intel®Trace Analyzer and Collector distinguishes between code regions (marked with VT_begin()/VT_end()) and functions (marked with VT_enter()/VT_leave()). The difference is only relevant when passing source code locations.

VT_begin

int VT_begin (int statehandle)

Description

Marks the beginning of a region with the name that was assigned to the symbol.

Regions should be used to subdivide a function into different parts or to mark the location where a function is called.

If automatic tracing of source code locations (PC tracing) is supported, then Intel® Trace Analyzer and Collector will log the location where VT_begin() is called as source code location for this region and the location where VT_end() is called as SCL for the next part of the calling symbol (which may be a function or another larger region).

If an SCL has been set with VT_thisloc(), then this SCL will be used even if PC tracing is supported.

The functions VT_enter() and VT_leave() have been added that can be used to mark the beginning and end of a function call within the function itself. The difference is that a manual source code location which is given to VT_leave() cannot specify where the function call took place, but rather where the function is left.

If PC tracing is enabled, then the VT_leave function stores the SCL where the instrumented function was called as SCL for the next part of the calling symbol. In other words, it skips the location where the function is left, which would be recorded if VT_end() were used instead.

VT_begin() adds an entry to a stack which can be removed only with VT_end().

Fortran

VTBEGIN(statehandle, ierr)

Parameters

statehandle handle defined either with VT_symdef() or VT_funcdef()

Return values

Returns error code

VT_beginl

int VT_beginl (int statehandle, int sclhandle)

Description

Shortcut for VT_thisloc(sclhandle); VT_begin(statehandle).

Fortran

VTBEGINL(statehandle, sclhandle, ierr)

VT_end

int VT_end (int statehandle)

Description

Marks the end of a region.

Has to match a VT_begin(). The parameter was used to check this, but this is no longer done to simplify instrumentation; now it is safe to pass a 0 instead of the original state handle.

Fortran

VTEND(statehandle, ierr)

Parameters

statehandle obsolete, pass anything you want

Return values

Returns error code

VT_endl

int VT_endl (int statehandle, int sclhandle)

Description

Shortcut for VT_thisloc(sclhandle); VT_end(statehandle).

Fortran

VTENDL(statehandle, sclhandle, ierr)

VT_enter

int VT_enter (int statehandle, int sclhandle)

Description

Mark the beginning of a function.

Usage similar to VT_beginl(). See also VT_begin().

Fortran

VTENTER(statehandle, sclhandle, ierr)

Parameters

statehandle handle defined either with VT_symdef() or VT_funcdef()
sclhandle handle, defined by VT_scldef. Use VT_NOSCL if you don't have a specific value.

Return values

Returns error code

VT_leave

int VT_leave (int sclhandle)

Description

Mark the end of a function.

See also VT_begin().

Fortran

VTLEAVE(sclhandle, ierr)

Parameters

sclhandle handle, defined by VT_scldef. Currently ignored, but is meant to specify the location of exactly where the function was left in the future. Use VT_NOSCL if you don't have a specific value.

Return values

Returns error code

VT_enterstate

int VT_enterstate (const char * name, int * statehandle, int * truncated)

Description

Defines a state (when invoked the first time) and enters it.

It relies on the caller to provide persistent storage for state handles.

The corresponding call to leave the state again is the normal VT_leave(). VT_leave() must be called if and only if VT_enterstate() returns a zero return code. 

static int bsend_handle, bsend_truncated;
int bsend_retval;
bsend_retval = VT_enterstate( "MPI:TRANSFER:BSEND", &bsend_handle, &bsend_truncated );
...
if( !bsend_retval) VT_leave( VT_NOSCL );

As demonstrated in this example, one or more colons : may be used to specify parent classes of the state, just as in VT_funcdef() and others.

But in contrast to those, VT_enterstate() also treats a slash / as special and uses it to log states at a varying degree of detail: depending on the value of DETAILED-STATES (0 = OFF, 1 = ON, 2, 3...), only the part of the name before the first slash is used (DETAILED-STATES 0). For higher values of DETAILED-STATES more components of the name are used and the slashes in the part of the name which is used is treated like the class separator (:).

Examples 

  1. "MPI:TRANSFER/SEND/COPY" + DETAILED-STATES 0: "MPI:TRANSFER"
  2. "MPI:TRANSFER/SEND/COPY" + DETAILED-STATES 1: "MPI:TRANSFER:SEND"
  3. "MPI:TRANSFER/SEND/COPY" + DETAILED-STATES >= 2: "MPI:TRANSFER:SEND:COPY"
  4. "/MPI:INTERNAL" + DETAILED-STATES 0: "" = not logged
  5. "/MPI:INTERNAL" + DETAILED-STATES 1: ":MPI:INTERNAL" = "MPI:INTERNAL"

If (and only if) the configuration option DETAILED-STATES causes the truncation of a certain state name, then entering that state is ignored if the process already is in that state.

Example of trace with DETAILED-STATES 0:

  1. enter "MPI:TRANSFER/WAIT" = enter "MPI:TRANSFER"

  2. enter "MPI:TRANSFER/COPY" = "MPI:TRANSFER" = ignored by Intel® Trace Analyzer and Collector, return code != 0

  3. leave "MPI:TRANSFER/COPY" = ignored by application

  4. enter "MPI:TRANSFER/WAIT" = recursive call; ignored too

  5. leave "MPI:TRANSFER/WAIT" = ignored by application

  6. leave "MPI:TRANSFER/WAIT" = leave "MPI:TRANSFER"

The same trace with DETAILED-STATES 1:

  1. enter "MPI:TRANSFER/WAIT" = enter "MPI:TRANSFER:WAIT"

  2. enter "MPI:TRANSFER/COPY" = enter "MPI:TRANSFER:COPY"

  3. leave "MPI:TRANSFER/COPY" = leave "MPI:TRANSFER:COPY"

  4. enter "MPI:TRANSFER/WAIT" = enter "MPI:TRANSFER:WAIT"

  5. leave "MPI:TRANSFER/WAIT" = leave "MPI:TRANSFER:WAIT"

  6. leave "MPI:TRANSFER/WAIT" = leave "MPI:TRANSFER:WAIT"

Fortran

VTENTERSTATE(name, statehandle, truncated, ierr)

Parameters

name the name of the state, with colons and/or slashes as separators as described above

Return values

statehandle must be initialized to zero before calling this function for the first time, then is set inside the function to the state handle which corresponds to the function which is logged
truncated set when calling the function for the first time: zero if the full name is logged

Returns zero if state was entered and VT_leave() needs to be called

VT_wakeup

int VT_wakeup (void)

Description

Triggers the same additional actions as logging a function call, but without actually logging a call.

When Intel® Trace Analyzer and Collector logs a function entry or exit it might also execute other actions, like sampling and logging counter data. If a function runs for a very long time, then Intel® Trace Analyzer and Collector has no chance to execute these actions. To avoid that, the programmer can insert calls to this function into the source code of the long-running function.

Fortran

VTWAKEUP(ierr)

Return values

Returns error code

Parent topic: Defining and Recording Functions or Regions
Level Two Title