9.3. Controlling Authentication

This page covers how PMIx authenticates connections on a system, and the controls a system administrator has over it.

9.3.1. How PMIx authenticates connections

When a process connects to a PMIx server, a security (psec) component establishes the process’s user and group. Two exist:

  • native - asks the kernel. Always built, and the default.

  • munge - uses the MUNGE daemon.

Either serves processes on the server’s own node only: a server takes no connection from another node.

What each one checks is described in Accepting Connections. The host environment then makes its own checks before accepting the connection.

The settings that count are the server’s. A server accepts only the components it has loaded, and passes its list to the processes it starts. A process cannot use a method the server does not offer. So apply the controls below where the host environment’s daemons read them.

You control which methods are available in four ways, from the most permanent to the least.

9.3.1.1. 1. Do not build the component

munge is built only when you ask for it, with --with-munge[=DIR].

To leave out a component that would otherwise build, name it in --enable-mca-no-build:

shell$ ./configure --enable-mca-no-build=psec-munge ...

Keep at least one component. A server with none fails to start; it never accepts connections without authentication.

See CLI Options for optional support libraries and Installation options.

9.3.1.2. 2. Restrict the components in a parameter file

The psec MCA parameter lists the components that may be used, or, with a leading ^, those that may not:

# use only munge
psec = munge

# use anything except munge
psec = ^munge

Put the setting in a parameter file. A parameter file belongs to one PMIx installation: each PMIx library reads the files under its own installation prefix when it initializes, and no other. Put the setting in the installation the host environment’s server runs from - its settings are the ones that count. A system with more than one PMIx installation, or applications that bring their own library (in a container, say), has a set of files for each:

  • $prefix/etc/pmix-mca-params.conf - the installation’s default. A user’s $HOME/.pmix/mca-params.conf and the user’s environment can override it.

  • $prefix/etc/pmix-mca-params-override.conf - settings here take precedence over the environment and over every other parameter file. Use this file for settings that users, and the host environment, should not change.

A process reads these files because its environment does not say otherwise. The files themselves are named by MCA parameters (mca_base_param_files, mca_base_override_param_file), so a process whose environment sets those - or sets mca_base_param_files to none - reads other files, or none. A parameter file is therefore the policy for the processes started without such settings: the host’s servers, when the administrator starts them or controls their environment. To make a component unavailable to every process of an installation, build the installation without it, or remove its plugin (below).

A process running as root reads a parameter file only if root owns it and no other user or group can write to it.

See The Modular Component Architecture (MCA) for the file format and Selecting which PMIx components are used at run time for the include and exclude syntax.

9.3.1.3. 3. Remove the component’s plugin

If PMIx was configured with --enable-mca-dso, each component is a separate plugin file under $prefix/lib/pmix. Deleting a plugin makes that component unavailable - for example, a component that was built by mistake:

shell$ rm $prefix/lib/pmix/pmix_mca_psec_munge.so

Components built into the library itself (the default) cannot be removed this way. Use the parameter file instead.

pmix_info lists the components an installation can load:

shell$ pmix_info | grep psec

9.3.1.4. 4. Tune the connection checks

These MCA parameters adjust the checks PMIx makes on connections:

Parameter

Default

Effect

ptl_base_legacy_clients

false

When true, a server takes clients and tools of PMIx releases before v7.0, which connect over TCP; a host’s PMIX_SERVER_LEGACY_PEERS overrides it, and ptl_base_legacy_peers sets it too in a build with legacy clients. Older clients need it. On a platform whose kernel has no table of TCP connections to confirm a peer with (any but Linux, macOS and FreeBSD 12 or later), native then takes such a peer on the identity it gives, as every earlier release did. Only a PMIx configured --enable-legacy-clients takes them; any other refuses this setting and fails its init, while ptl_base_legacy_peers asks it for nothing. See Processes of releases before v7.0.

ptl_base_if_include / ptl_base_if_exclude

none

Network interfaces the server may or may not listen on. Keep loopback included.

ptl_base_connect_ack_timeout

5

Seconds a server allows for a new connection’s opening message.

ptl_base_max_pending_connections

0

How many connections may still be sending their request, and, separately, how many tools may wait for the host’s decision. 0 means an eighth of the process’s file-descriptor limit, for each; -1 means no limit. A user other than the server’s, root and the host’s PMIX_USERID may have at most a quarter of the waiting tools (at least one).

ptl_base_max_pending_bytes

16777216

Most bytes the server allocates, between them, for connection requests still arriving. Each request’s buffer starts at 4 KB and doubles as its bytes arrive, up to the length it names; a request whose buffer cannot grow within the limit is closed. 0 means no limit.

ptl_base_max_msg_size

32

Largest message accepted, in megabytes. 0 means no limit.

ptl_base_server_access

none

Servers a client or tool will talk to on its host besides one running as its own user or as root, when no server started it: users=<list>,groups=<list>, each list user or group names or numbers separated by :. Over TCP only users can match: the kernel records no group for a TCP connection. A name the system does not know, an empty list, or a directive given twice makes the process fail to initialize.

To see every parameter and its current value:

shell$ pmix_info --param psec all
shell$ pmix_info --param ptl all

Some policies are set by the host environment rather than by MCA parameters, because they depend on how the host is deployed:

  • Whether tools of other users may connect. By default a server refuses a tool whose user is not the server’s own, root, or the user the host named as the owner of its local socket (PMIX_USERID). A host can allow them (PMIX_SERVER_ALLOW_FOREIGN_TOOLS), and must then also widen the server’s local socket to them (PMIX_SOCKET_MODE, with PMIX_GRPID for a group): by default only the server’s user can connect to it.

  • Processes on other nodes never connect. A server takes no TCP connection at all by default, and when it takes clients and tools of earlier releases it listens for them on loopback only. A host that asks for connections from other nodes (PMIX_SERVER_REMOTE_CONNECTIONS set to true) fails its init. A client or tool refuses to dial a server at an address that is not its own host’s, and a server refuses a TCP peer at such an address.

  • Clients are always checked against the user and group the host registered them with. No setting changes that.

Consult your host environment’s documentation for how it exposes these choices.

9.3.2. Temporary files

A PMIx server writes rendezvous files, session directories and its local socket under its temporary directory ($TMPDIR, or /tmp). Tools find servers through those files. The local socket is in a directory of its own, pmix-XXXXXX. A server that is killed can leave its files behind; PMIx reclaims a rendezvous file whose server is gone, but session directories and socket directories accumulate. Clean out old pmix* entries as you would other stale temporary files.

Keep the temporary directory’s path short: the local socket’s path is built under it, and is limited to about 100 characters. A server whose path is too long fails to start. The default $TMPDIR on macOS is too long - start the host environment with a shorter one.

See Session Directories.