Recent versions of MPI have a standardized way of reading out performance variables: the tools interface which improves on the old interface described in section 15.6.2 .
crumb trail: > mpi-tools > Initializing the tools interface
The tools interface requires a different initialization routine MPI_T_init_thread
int MPI_T_init_thread( int required,int *provided );Likewise, there is MPI_T_finalize
int MPI_T_finalize();These matching calls can be made multiple times, after MPI has already been initialized with MPI_Init or MPI_Init_thread .
Verbosity level is an integer parameter.
MPI_T_VERBOSITY_{USER,TUNER,MPIDEV}_{BASIC,DETAIL,ALL}
crumb trail: > mpi-tools > Control variables \index[mpi]{control variable|(} \index[mpi]{cvar|see{control variable}}
Control variables that can be used to inspect and/or control the internal workings of MPI. Accessing control variables requires initializing the tools interface; section 14.1 .
We query how many control variables are available with MPI_T_cvar_get_num .
A description of the control variable can be obtained from MPI_T_cvar_get_info .
// cvar.c
MPI_T_cvar_get_num(&ncvar);
printf("#cvars: %d\n",ncvar);
for (int ivar=0; ivar<ncvar; ivar++) {
char name[100]; int namelen = 100;
char desc[256]; int desclen = 256;
int verbosity,bind,scope;
MPI_Datatype datatype;
MPI_T_enum enumtype;
MPI_T_cvar_get_info
(ivar,
name,&namelen,
&verbosity,&datatype,&enumtype,desc,&desclen,&bind,&scope
);
printf("cvar %3d: %s\n %s\n",ivar,name,desc);
Remark There is no constant indicating a maximum buffer length for these variables. However, you can do the following:
Conversely, given a variable name, its index can be retrieved with MPI_T_cvar_get_index :
int MPI_T_cvar_get_index(const char *name, int *cvar_index)If the name can not be matched, the index is MPI_T_ERR_INVALID_NAME .
Accessing a control variable is done through a control variable handle .
int MPI_T_cvar_handle_alloc
(int cvar_index, void *obj_handle,
MPI_T_cvar_handle *handle, int *count)
The handle is freed with
MPI_T_cvar_handle_free
:
int MPI_T_cvar_handle_free(MPI_T_cvar_handle *handle)
Control variable access is done through MPI_T_cvar_read and MPI_T_cvar_write :
int MPI_T_cvar_read(MPI_T_cvar_handle handle, void* buf); int MPI_T_cvar_write(MPI_T_cvar_handle handle, const void* buf);
\index[mpi]{control variable|)}
crumb trail: > mpi-tools > Performance variables \index[mpi]{performance variable|(} \index[mpi]{pvar|see{performance variable}}
The realization of the tools interface is installation-dependent, you first need to query how much of the tools interface is provided.
// mpitpvar.c MPI_Init_thread(&argc,&argv,MPI_THREAD_SINGLE,&tlevel); MPI_T_init_thread(MPI_THREAD_SINGLE,&tlevel); int npvar; MPI_T_pvar_get_num(&npvar);
int name_len=256,desc_len=256,
verbosity,var_class,binding,isreadonly,iscontiguous,isatomic;
char var_name[256],description[256];
MPI_Datatype datatype; MPI_T_enum enumtype;
for (int pvar=0; pvar<npvar; pvar++) {
name_len = 256; desc_len=256;
MPI_T_pvar_get_info(pvar,var_name,&name_len,
&verbosity,&var_class,
&datatype,&enumtype,
description,&desc_len,
&binding,&isreadonly,&iscontiguous,&isatomic);
if (procid==0)
printf("pvar %d: %d/%s = %s\n",pvar,var_class,var_name,description);
}
\begin{raggedlist} Performance variables come in classes: MPI_T_PVAR_CLASS_STATE MPI_T_PVAR_CLASS_LEVEL MPI_T_PVAR_CLASS_SIZE MPI_T_PVAR_CLASS_PERCENTAGE MPI_T_PVAR_CLASS_HIGHWATERMARK MPI_T_PVAR_CLASS_LOWWATERMARK MPI_T_PVAR_CLASS_COUNTER MPI_T_PVAR_CLASS_AGGREGATE MPI_T_PVAR_CLASS_TIMER MPI_T_PVAR_CLASS_GENERIC \end{raggedlist}
Query the number of performance variables with MPI_T_pvar_get_num :
int MPI_T_pvar_get_num(int *num_pvar);Get information about each variable, by index, with MPI_T_pvar_get_info :
int MPI_T_pvar_get_info
(int pvar_index, char *name, int *name_len,
int *verbosity, int *var_class, MPI_Datatype *datatype,
MPI_T_enum *enumtype, char *desc, int *desc_len, int *bind,
int *readonly, int *continuous, int *atomic)
See general remarks about these in section
14.2
.
Given a name, the index can be retried with MPI_T_pvar_get_index :
int MPI_T_pvar_get_index(const char *name, int var_class, int *pvar_index)Again, see section 14.2 .
crumb trail: > mpi-tools > Performance variables > Performance experiment sessions
To prevent measurements from getting mixed up, they need to be done in performance experiment session s, to be called `sessions' in this chapter. However see section 8.3 .
Create a session with MPI_T_pvar_session_create
int MPI_T_pvar_session_create(MPI_T_pvar_session *session)and release it with MPI_T_pvar_session_free :
int MPI_T_pvar_session_free(MPI_T_pvar_session *session)which sets the session variable to MPI_T_PVAR_SESSION_NULL .
We access a variable through a handle, associated with a certain session. The handle is created with MPI_T_pvar_handle_alloc :
int MPI_T_pvar_handle_alloc
(MPI_T_pvar_session session, int pvar_index,
void *obj_handle, MPI_T_pvar_handle *handle, int *count)
(If a routine takes both a session and handle argument, and
the two are not associated, an error of
MPI_T_ERR_INVALID_HANDLE
is returned.)
Free the handle with MPI_T_pvar_handle_free :
int MPI_T_pvar_handle_free
(MPI_T_pvar_session session,
MPI_T_pvar_handle *handle)
which sets the variable to
MPI_T_PVAR_HANDLE_NULL
.
Continuous variables (see MPI_T_pvar_get_info above, which outputs this) can be started and stopped with MPI_T_pvar_start and MPI_T_pvar_stop :
int MPI_T_pvar_start(MPI_T_pvar_session session, MPI_T_pvar_handle handle); int MPI_T_pvar_stop(MPI_T_pvar_session session, MPI_T_pvar_handle handle)Passing MPI_T_PVAR_ALL_HANDLES to the stop call attempts to stop all variables within the session. Failure to stop a variable returns MPI_T_ERR_PVAR_NO_STARTSTOP .
Variables can be read and written with MPI_T_pvar_read and MPI_T_pvar_write :
int MPI_T_pvar_read
(MPI_T_pvar_session session, MPI_T_pvar_handle handle,
void* buf)
int MPI_T_pvar_write
(MPI_T_pvar_session session, MPI_T_pvar_handle handle,
const void* buf)
If the variable can not be written
(see the
readonly
parameter of
MPI_T_pvar_get_info
),
MPI_T_ERR_PVAR_NO_WRITE
is returned.
A special case of writing the variable is to reset it with
int MPI_T_pvar_reset(MPI_T_pvar_session session, MPI_T_pvar_handle handle)The handle value of MPI_T_PVAR_ALL_HANDLES is allowed.
A call to MPI_T_pvar_readreset is an atomic combination of the read and reset calls:
int MPI_T_pvar_readreset
(MPI_T_pvar_session session,MPI_T_pvar_handle handle,
void* buf)
\index[mpi]{performance variable|)}
crumb trail: > mpi-tools > Categories of variables
Variables, both the control and performance kind, can be grouped into categories by the MPI implementation.
The number of categories is queried with MPI_T_category_get_num :
int MPI_T_category_get_num(int *num_cat)and for each category the information is retrieved with MPI_T_category_get_info :
int MPI_T_category_get_info
(int cat_index,
char *name, int *name_len, char *desc, int *desc_len,
int *num_cvars, int *num_pvars, int *num_categories)
For a given category name the index can be found with
MPI_T_category_get_index
:
int MPI_T_category_get_index(const char *name, int *cat_index)
The contents of a category are retrieved with MPI_T_category_get_cvars , MPI_T_category_get_pvars , MPI_T_category_get_categories :
int MPI_T_category_get_cvars(int cat_index, int len, int indices[]) int MPI_T_category_get_pvars(int cat_index, int len, int indices[]) int MPI_T_category_get_categories(int cat_index, int len, int indices[])
\begin{raggedlist} These indices can subsequently be used in the calls MPI_T_cvar_get_info , MPI_T_pvar_get_info , MPI_T_category_get_info . \end{raggedlist}
If categories change dynamically, this can be detected with MPI_T_category_changed
int MPI_T_category_changed(int *stamp)
crumb trail: > mpi-tools > Events
// mpitevent.c int nsource; MPI_T_source_get_num(&nsource);
int name_len=256,desc_len=256;
char var_name[256],description[256];
MPI_T_source_order ordering;
MPI_Count ticks_per_second,max_ticks;
MPI_Info info;
MPI_Datatype datatype; MPI_T_enum enumtype;
for (int source=0; source<nsource; source++) {
name_len = 256; desc_len=256;
MPI_T_source_get_info(source,var_name,&name_len,
description,&desc_len,
&ordering,&ticks_per_second,&max_ticks,&info);