Slurmrestd Access

This page describes how-to setup Slurm slurmrestd service and Slurm-web agent with:

Unix socket,
JWT authentication,
auto token management mode.

< back to all slurmrestd setups

With jwt authentication method, the client must provide user and token in HTTP headers. Then, slurmrestd service forwards these credentials in RPCs to slurmctld and slurmdbd which are responsible to check expiration and signature of the token.

In this configuration, slurmrestd service and Slurm-web agent can run with unprivileged system users.

Setup JWT authentication

Generate random Slurm JWT signing key with restrictive permissions:

# dd if=/dev/random of=/var/spool/slurm/jwt_hs256.key bs=32 count=1
# chown slurm:slurm /var/spool/slurm/jwt_hs256.key
# chmod 0600 /var/spool/slurm/jwt_hs256.key

Edit main Slurm and SlurmDBD configuration to enable JWT alternative authentication:

AuthAltTypes=auth/jwt
AuthAltParameters=jwt_key=/var/spool/slurm/jwt_hs256.key

Restart slurmctld and slurmdbd services to update configuration:

# systemctl restart slurmctld slurmdbd

Setup `slurmrestd`

Create /etc/systemd/system/slurmrestd.service.d/slurm-web.conf drop-in configuration override for slurmrestd service:

[Service]
# Unset vendor unit ExecStart and Environment to avoid cumulative definition
ExecStart=
Environment=
Environment="SLURM_JWT=daemon"
ExecStart=/usr/sbin/slurmrestd $SLURMRESTD_OPTIONS -a rest_auth/jwt unix:/run/slurmrestd/slurmrestd.socket
RuntimeDirectory=slurmrestd
RuntimeDirectoryMode=0755
User=slurmrestd
Group=slurmrestd
DynamicUser=yes

With DynamicUser=yes, systemd creates a transient slurmrestd system user during the lifetime of the service and executes the daemon with this unprivileged user.

Reload systemd units and enable the service:

# systemctl daemon-reload && systemctl enable --now slurmrestd.service

Setup Agent

Copy Slurm JWT signing key and restrict permissions to this copy:

# cp /var/spool/slurm/jwt_hs256.key /var/lib/slurm-web/slurmrestd.key

Restrict access to this sensitive file with read permission to slurm-web system user only:

# chown slurm-web:slurm-web /var/lib/slurm-web/slurmrestd.key
# chmod 0400 /var/lib/slurm-web/slurmrestd.key

Test Access

To test Slurm-web agent and slurmrestd service configuration parameters, you can run slurm-web-connect-check utility. It tries to send HTTP request to slurmrestd with Slurm-web agent configuration parameters and reports the status. For example:

# /usr/libexec/slurm-web/slurm-web-connect-check
✅ connection successful (slurm: 24.11.0, cluster: hpc)

Restart agent

Upon succesful test, restart agent service to apply changes:

Native service
Production HTTP server

When using default native service:

# systemctl restart slurm-web-agent.service

When using production HTTP server:

# systemctl restart slurm-web-agent-uwsgi.service