10.3.4. Deciding What a Requester May Do
Once a process is connected, every request it makes is either handled by the PMIx library or passed up to the host. Whoever performs the operation decides whether the requester may have it. The library decides for what it does itself. The host decides for everything passed up to it - and the library gives the host what it needs to decide.
The host itself is not restricted. These rules govern requests made by clients and tools.
10.3.4.1. Who is asking
Every up-call made for a client or tool identifies the requester - the process connected to this server - as its connection established it:
The requester’s process ID is the up-call’s
procargument. Where an up-call has no such argument (iof_pull,register_events), it isPMIX_REQUESTORin the info array.PMIX_USERID and PMIX_GRPID are the uid and gid PMIx established when the requester connected. Any the requester put in its request are removed. This is the identity to authorize.
PMIX_REQUESTOR_PID is the pid the kernel reported for the requester’s connection. It is present only where the kernel reports one: on the local transport, on platforms that do. It is never present for a peer connected over TCP, where the only pid on record is the one the peer’s library sent. Any
PMIX_REQUESTOR_PIDthe requester put in its request is removed.PMIX_REQUESTED_GRPID is the group the requester asked to have the work charged to, if it named one - as
PMIX_REQUESTED_GRPID, or asPMIX_GRPID, which earlier releases passed up for it. PMIx resolves a group name to a number but does not check that the requester belongs to the group it names. If the group matters to your decision, check membership yourself, and refuse the request (for example withPMIX_ERR_NO_PERMISSIONS) if it fails. A request that names more than one, or a name that cannot be resolved, is refused before it reaches you.
Each appears at most once. A few up-calls differ:
fence_nb,connect,disconnectandgroupare made for all participants, not for one requester. They carry no identity. PMIx has already checked that each local caller is a participant.publishcarries onlyPMIX_USERIDandPMIX_GRPID: its info array is the data being published, so nothing more is added that a host might store as a published key. Skip those, and anyPMIX_ORIGIN_*, when you store the data.tool_connected2carries only what the tool’s connection established:PMIX_USERID,PMIX_GRPIDand, where there is one,PMIX_REQUESTOR_PID. Any requester pid, requested group,PMIX_ORIGIN_*orPMIX_REQUESTORthe tool sent is removed. A tool that asked for an identity of its own is named byPMIX_NSPACEandPMIX_RANK.direct_modexcarriesPMIX_REQUESTOR,PMIX_USERID,PMIX_GRPIDand, where there is one,PMIX_REQUESTOR_PIDfor the requesters it stands for. It carries no requested group.notify_event, for an event a client raised, carries that client’sPMIX_USERID,PMIX_GRPIDand, where there is one,PMIX_REQUESTOR_PID; thesourceargument is the client. A host passing an event on decides with that identity whether it may. ForPMIX_DEBUGGER_RELEASE, check, before relaying or delivering it, that the user may access every job the release can reach - the jobs named in a custom range (a wildcard namespace meaning every job); the source’s own job forPMIX_RANGE_NAMESPACE; and for any wider range, every job you would deliver it to. PMIx applies that rule itself to the jobs registered with this server; for any other, the decision is yours.The deprecated
aborthas no info array; useabort2, which carries the identity.
When the host itself calls a PMIx API that is serviced by its own module, the up-call names the server as the requester and adds nothing.
The full list, per up-call, is under “The requester’s identity” in pmix_server_module_t(5).
10.3.4.2. Requests that pass through more than one server
A request can travel through several servers. For example, a workflow manager relays a request from its own client to the resource manager, or a launcher relays one to a scheduler. Whether to pass a request on, and whether to believe what a relay says about who first asked for it, is policy - and it belongs to the host at each level. PMIx does not make that decision, and does not relay a peer’s request on its own: a request it cannot serve goes to your up-call, and is refused if you have none.
What you are told is always the relay.
PMIX_USERID,PMIX_GRPIDandPMIX_REQUESTOR_PIDin an up-call describe the process connected to you, as its connection established it. Nothing a relay sends replaces them. A host that knows nothing of relays therefore judges the request as the relay’s own - it fails safe.What the relay claims comes beside it. A relay names the process that first made the request with
PMIX_ORIGIN_USERID,PMIX_ORIGIN_GRPID,PMIX_ORIGIN_PID,PMIX_ORIGIN_IS_TOOLandPMIX_ORIGIN_IS_CLIENT, and withPMIX_REQUESTORfor its process ID. PMIx passes these to you untouched - only you can judge them - and refuses a request that carries any of them more than once.When you accept relays, decide from the verified identity (and pid) whether this peer may speak for others. If it may, attribute the work to the
PMIX_ORIGIN_*claim, and check anyPMIX_REQUESTED_GRPIDagainst whoever you attribute the work to. If it may not, judge the request as the relay’s own, or refuse it.When you relay a request, copy the identity you attributed the work to into the
PMIX_ORIGIN_*attributes. At the first hop that is the up-call’s ownPMIX_USERID,PMIX_GRPIDandPMIX_REQUESTOR_PID. Carry a requested group on asPMIX_REQUESTED_GRPID.Carry the identity between your own daemons when you forward a request from one node to another.
10.3.4.3. The access rule
PMIx has one rule for access to a job: a requester may access a job if any of these holds:
its uid is 0 (root);
its uid is the server’s own effective uid;
its uid is the job owner’s;
its uid is in the job’s access list;
it belongs to a group in the job’s access list.
The job’s owner and access list come from the host, when it registers
the job (PMIX_USERID, PMIX_GRPID,
PMIX_ACCESS_PERMISSIONS). If the host gives no owner, the owner is
the user the job’s clients were registered as, and failing that the
server’s own user. A host running as a service account should
therefore always register each job’s owner.
The rule, what it covers, and how it handles data held on other nodes are described in full in Access control by user and group.
10.3.4.4. What PMIx decides
PMIx applies the rule to requests it answers or performs itself:
reading another job’s data (
PMIx_Get), whether held locally or fetched from another node;queries and resolve requests it can answer itself;
monitoring of processes on its own node;
cleanup directives on
PMIx_Job_control;output that follows a job into the jobs it spawns;
the ownership and mode of output files and shared-memory job data.
PMIx also checks participation: a process calling PMIx_Fence,
PMIx_Connect, PMIx_Disconnect, PMIx_Group_construct or
PMIx_Group_destruct must be one of the participants it names, or
the call fails with PMIX_ERR_NOT_A_MEMBER.
10.3.4.5. What the host decides
PMIx does not decide whether a requester may have an operation it passes up. The host does. In particular, PMIx makes no access check on the targets of these up-calls:
Up-call |
The host decides |
|---|---|
whether the requester may terminate the processes it names. |
|
whether the requester may signal, kill, pause, resume, or otherwise act on the processes it names. |
|
whether the requester may send input to the processes it names. |
|
whether the requester may launch the job, with the resources and options it asks for. |
|
whether the requester may obtain, change, or release the resources or session concerned. |
|
whether the requester may receive output from the named processes. PMIx forwards nothing until the host approves. |
|
whether the requester may monitor processes on other nodes. (PMIx handles processes on its own node.) |
|
what the requester may learn from the queries PMIx could not answer itself. |
|
who may see and remove each published item, honoring any
|
|
whether the requester may log to the channels it names. |
|
what fabric information the requester may receive. |
|
whether to issue or check a credential for the requester. |
A host that is linked with the library can apply exactly the library’s
rule by calling pmix_server_access_check() with its own copies of a
job’s access list and the requester’s groups. See “What a host needs to
do” in Access control by user and group.
10.3.4.6. Summary for host developers
Register each job’s owner and access list on every node that hosts part of the job.
Accept
PMIX_ACCESS_PERMISSIONSin spawn requests and carry it with the job.Authorize every operation PMIx passes up, using
PMIX_USERIDfrom the up-call - never an identity the request itself claims.Check group membership yourself before acting on a
PMIX_REQUESTED_GRPID.Before relaying or delivering a
PMIX_DEBUGGER_RELEASE, check that its raiser may access every job it can reach.Accept a
PMIX_ORIGIN_*claim only from a relay you trust to make it. Carry the identity you attributed the work to inPMIX_ORIGIN_*when you pass a request to another server.Carry the requester’s
PMIX_USERID,PMIX_GRPIDandPMIX_REQUESTORwith PMIx_server_dmodex_request2(3) when you fetch data for it.