25.2.21. PMIx_Session_control
PMIx_Session_control — Request a control action be applied to a
session.
25.2.21.1. SYNOPSIS
#include <pmix.h>
pmix_status_t PMIx_Session_control(uint32_t sessionID,
const pmix_info_t directives[], size_t ndirs,
pmix_info_cbfunc_t cbfunc, void *cbdata);
25.2.21.1.1. Python Syntax
from pmix import *
# session control is bound on PMIxServer and inherited by PMIxTool and
# PMIxScheduler
foo = PMIxServer()
# ... after a successful foo.init() ...
# the directives is a list of Python ``pmix_info_t`` dictionaries
pydirs = [{'key': PMIX_SESSION_APP,
'value': "myapp", 'val_type': PMIX_STRING}]
rc = foo.session_control(1, pydirs)
25.2.21.2. INPUT PARAMETERS
sessionID: Auint32_tidentifying the session to which the control action is to be applied. A value ofUINT32_MAXindicates that the action applies to all sessions under the caller’s control.directives: Pointer to an array of pmix_info_t(5) structures describing the control action(s) to be taken (see DIRECTIVES). ANULLvalue is supported when no directives are desired.ndirs: Number of elements in thedirectivesarray.cbfunc: Callback function of type pmix_info_cbfunc_t. IfNULL, the call is treated as a blocking operation; otherwise it is non-blocking and the callback is invoked once the request has been processed.cbdata: Opaque pointer that is passed, unmodified, tocbfunc.
25.2.21.3. DESCRIPTION
Request that a control action be applied to the session identified by
sessionID. The specific action(s) to be taken are conveyed as attributes in
the directives array.
PMIx_Session_control supports both blocking and non-blocking operation
through a single entry point, selected by the cbfunc argument:
If
cbfuncisNULL, the call is blocking: it does not return until the request has been processed, and the return status reflects the overall outcome of the operation.If
cbfuncis non-NULL, the call is non-blocking: it returns immediately, andcbfuncis invoked with the final status and any returned information once the request has been processed. The callback reports whether the request was granted and may supply additional information — for example, the reason for any denial — in its array of pmix_info_t(5) structures.
As with all non-blocking PMIx APIs, when a cbfunc is supplied the caller
must keep the directives array valid until cbfunc is invoked.
25.2.21.4. DIRECTIVES
The requested action is expressed entirely through the directives array.
The PMIx library is not required to interpret these directives itself —
it passes them to the host environment (typically the scheduler) for
processing. Actual support for any given directive depends on the host
environment.
Identifying the request:
PMIX_SESSION_CTRL_ID(char*) — a string identifier for this request, allowing its status to later be queried or the operation to be cancelled.PMIX_REQUESTOR(pmix_proc_t*) — identifier of the process on whose behalf the request is being made, used when one process relays the operation to the PMIx library on behalf of another (the API itself takes no requestor parameter).
Instantiating a session (typically issued by a scheduler):
PMIX_SESSION_INSTANTIATE(bool) — instantiate the specified session.PMIX_ALLOC_UID(uint32_t) — the user id the session belongs to.PMIX_ALLOC_GID(uint32_t) — the group id the session belongs to. A scheduler instantiates a session on behalf of a user, and these say which one.PMIX_USERIDandPMIX_GRPIDcannot: the library sets them to the calling process’s own user and group, which are the scheduler’s.PMIX_SESSION_RESOURCES(pmix_data_array_t*) — array of pmix_info_t(5) structures describing the resources to be included in the session (e.g., a regular expression of node names, an array of resource blocks, etc.).PMIX_SESSION_APP(pmix_data_array_t*) — array ofpmix_app_tstructures to be executed in the assigned session upon instantiation.PMIX_SESSION_JOB(pmix_data_array_t*) — array of pmix_info_t(5) structures describing the job information to be applied in conjunction with the specifiedPMIX_SESSION_APPdefinitions. It is an error to provide this without aPMIX_SESSION_APPdescription.PMIX_SESSION_PROVISION(pmix_data_array_t*) — description of the nodes to be provisioned with a specified image.PMIX_SESSION_PROVISION_NODES(char*) — regular expression identifying nodes that are to be provisioned.PMIX_SESSION_PROVISION_IMAGE(char*) — name of the image that is to be provisioned.
Controlling a session’s operation:
PMIX_SESSION_PAUSE(bool) — pause all jobs in the specified session.PMIX_SESSION_RESUME(bool) — resume (“un-pause”) all jobs in the specified session.PMIX_SESSION_PREEMPT(bool) — preempt the indicated jobs (named via aPMIX_NSPACEattribute in the accompanying array) in the specified session and recover all their resources. If noPMIX_NSPACEis given, all jobs in the session are preempted.PMIX_SESSION_RESTORE(bool) — restore the indicated jobs (named via aPMIX_NSPACEattribute in the accompanying array) in the specified session, including all their resources. If noPMIX_NSPACEis given, all jobs in the session are restored.PMIX_SESSION_SIGNAL(int) — send the given signal to all processes of every job in the session.PMIX_SESSION_EXTEND(bool) — extend the specified session, either in time or in resources, based on additional provided attributes.PMIX_SESSION_SEP(bool) — separate the specified session from its parent — i.e., allow the two to terminate independently.PMIX_SESSION_TERMINATE(bool) — terminate all jobs in the specified session and recover all resources included in the session.PMIX_SESSION_COMPLETE(bool) — indicate that the specified session has completed and all of its resources have been recovered and are available for scheduling. The accompanying array must include pmix_info_t(5) structures reporting the returned status of any jobs executed in the session.
25.2.21.5. RETURN VALUE
For the blocking form (cbfunc == NULL), a successful request returns
PMIX_OPERATION_SUCCEEDED. For the non-blocking form, a return of
PMIX_SUCCESS indicates only that the request was accepted for processing;
the final status and any data are delivered to cbfunc.
PMIX_SUCCESS— (non-blocking form) the request was accepted for processing.PMIX_OPERATION_SUCCEEDED— (blocking form) the request was processed successfully.PMIX_ERR_NOT_SUPPORTED— the operation is not supported in the caller’s role or environment — for example, the caller is a system controller that is not connected to the scheduler.PMIX_ERR_UNREACH— the caller is not connected to a server capable of servicing the request.PMIX_ERR_COMM_FAILURE— the connection to the server was lost before a result could be returned.PMIX_ERR_NOT_AVAILABLE— the operation cannot be serviced because the library’s progress engine has been stopped.PMIX_ERR_INIT— the PMIx library has not been initialized.
Any other negative value indicates an appropriate error condition. PMIx error
constants are defined in pmix_common.h.
25.2.21.6. NOTES
Unlike most PMIx operations, PMIx_Session_control does not provide a
separate _nb entry point; the single API selects blocking versus
non-blocking behavior based on whether cbfunc is NULL.
PMIx_Session_control is a PMIx library extension and is not part of the
published PMIx Standard.
25.2.21.7. PROGRESS THREAD RESTRICTION
A blocking PMIx call must not be made from within the PMIx progress thread. Any code the library itself invokes runs on that thread: an event handler registered through PMIx_Register_event_handler(3), a callback passed to a non-blocking PMIx API, and — in a server or tool — the completion of a host-module up-call. A blocking call waits for work that the progress thread has to perform, so making one from that thread waits for itself and never returns. The PMIx Standard disallows it, and there is no way for an implementation to service such a request.
Where this call has a blocking form — including the blocking
behavior a non-blocking entry point adopts when it is passed a NULL
cbfunc — that form detects the situation and returns
PMIX_ERR_WOULD_BLOCK immediately, accompanied by a diagnostic naming
the call. Nothing is done and no callback is invoked.
PMIX_ERR_WOULD_BLOCK here is not a transient condition to retry: it
reports a call that cannot be serviced from where it was made. Reissue
it as the non-blocking form with a callback, or from a thread of your
own.
25.2.21.8. SESSION CONTROL AND ALLOCATION REQUESTS
PMIx_Session_control issues directives to a session. Most of them
— pause, resume, signal, terminate — act on a session that
already exists, and may come from the user the session belongs to as well as
from the scheduler.
PMIX_SESSION_INSTANTIATE and PMIX_SESSION_EXTEND are different. They
do not ask for resources: a user asks for resources with
PMIx_Allocation_request(3), and the
scheduler decides. These two directives are how the scheduler then tells the
runtime managing the session — a distributed virtual machine (DVM), for
example — what it decided: start a session over these resources for
this user, or give this session more time or more resources. A runtime would
typically accept them only from the scheduler (or, where there is none, from
whoever acts with the scheduler’s authority) — not from the user the
session is for, who asks with an allocation request instead.
The scheduler’s answer to an allocation request goes to the requester, and may carry much the same information as the directive, but the directive is still needed:
The runtime may never see the answer. The host environment that passed the request on is not required to watch for the scheduler’s reply, and the scheduler cannot know whether it will intercept the answer and act on it. A runtime that is told directly needs no such arrangement.
The runtime may not know a request was made. A request can come from a tool connected to the scheduler, or to some other server, rather than from one of the runtime’s own processes. The runtime then has no record of the request at all, and only a directive from the scheduler tells it what to do.
Not every change starts with a request. A scheduler sometimes needs a runtime to change a session for its own reasons — for example, to modify a job’s resources to keep the whole system within an overall energy limit. There is no allocation request to answer in that case; there is only the directive.
25.2.21.9. EXAMPLES
A workflow tool, connected to the scheduler rather than to any runtime, asks for four nodes for two hours on behalf of the user running it:
pmix_info_t info[2];
pmix_info_t *results = NULL;
size_t nresults = 0;
uint64_t nnodes = 4;
pmix_status_t rc;
PMIx_Info_load(&info[0], PMIX_ALLOC_NUM_NODES, &nnodes, PMIX_UINT64);
PMIx_Info_load(&info[1], PMIX_ALLOC_TIME, "2:00:00", PMIX_STRING);
rc = PMIx_Allocation_request(PMIX_ALLOC_NEW, info, 2, &results, &nresults);
/* on success, results include the PMIX_ALLOC_ID of the new allocation */
PMIx_Info_destruct(&info[0]);
PMIx_Info_destruct(&info[1]);
if (NULL != results) {
PMIx_Info_free(results, nresults);
}
The runtime that will host the work saw none of this. The scheduler chooses
the nodes, answers the tool with the allocation’s identifier, and directs the
runtime to instantiate a session over them for the user who asked —
identified by the PMIX_USERID and PMIX_GRPID the library attached to
the tool’s request (here, 1001):
pmix_info_t dirs[4], *res;
pmix_data_array_t resources;
uint32_t uid = 1001, gid = 1001;
bool flag = true;
pmix_status_t rc;
size_t n;
PMIx_Data_array_construct(&resources, 3, PMIX_INFO);
res = (pmix_info_t *) resources.array;
PMIx_Info_load(&res[0], PMIX_ALLOC_NODE_LIST,
"node01,node02,node03,node04", PMIX_STRING);
PMIx_Info_load(&res[1], PMIX_ALLOC_ID, "alloc-57", PMIX_STRING);
PMIx_Info_load(&res[2], PMIX_ALLOC_TIME, "2:00:00", PMIX_STRING);
PMIx_Info_load(&dirs[0], PMIX_SESSION_INSTANTIATE, &flag, PMIX_BOOL);
PMIx_Info_load(&dirs[1], PMIX_ALLOC_UID, &uid, PMIX_UINT32);
PMIx_Info_load(&dirs[2], PMIX_ALLOC_GID, &gid, PMIX_UINT32);
/* the info takes its own copy of the array */
PMIx_Info_load(&dirs[3], PMIX_SESSION_RESOURCES, &resources, PMIX_DATA_ARRAY);
rc = PMIx_Session_control(12, dirs, 4, NULL, NULL);
/* PMIX_OPERATION_SUCCEEDED once the runtime has created session 12 */
for (n = 0; n < 4; n++) {
PMIx_Info_destruct(&dirs[n]);
}
PMIx_Data_array_destruct(&resources);
The tool can now refer to the allocation by its PMIX_ALLOC_ID —
for example, to spawn a job into it. See PMIx_Allocation_request(3) for an extension requested by a process
already running in a session.