Slurmrestd Access

This page describes how-to setup Slurm slurmrestd service and Slurm-web agent with:

  • Unix socket,

  • JWT authentication,

  • auto token management mode.

< back to all slurmrestd setups

slurmrestd access modes unix jwt auto

With jwt authentication method, the client must provide user and token in HTTP headers. Then, slurmrestd service forwards these credentials in RPCs to slurmctld and slurmdbd which are responsible to check expiration and signature of the token.

In this configuration, slurmrestd service and Slurm-web agent can run with unprivileged system users.

Setup JWT authentication

Generate random Slurm JWT signing key with restrictive permissions:

# dd if=/dev/random of=/var/spool/slurm/jwt_hs256.key bs=32 count=1
# chown slurm:slurm /var/spool/slurm/jwt_hs256.key
# chmod 0600 /var/spool/slurm/jwt_hs256.key

Edit main Slurm and SlurmDBD configuration to enable JWT alternative authentication:

AuthAltTypes=auth/jwt
AuthAltParameters=jwt_key=/var/spool/slurm/jwt_hs256.key

Restart slurmctld and slurmdbd services to update configuration:

# systemctl restart slurmctld slurmdbd
More links

Setup slurmrestd

Create /etc/systemd/system/slurmrestd.service.d/slurm-web.conf drop-in configuration override for slurmrestd service:

[Service]
# Unset vendor unit ExecStart and Environment to avoid cumulative definition
ExecStart=
Environment=
Environment="SLURM_JWT=daemon"
ExecStart=/usr/sbin/slurmrestd $SLURMRESTD_OPTIONS -a rest_auth/jwt unix:/run/slurmrestd/slurmrestd.socket
RuntimeDirectory=slurmrestd
RuntimeDirectoryMode=0755
User=slurmrestd
Group=slurmrestd
DynamicUser=yes
With DynamicUser=yes, systemd creates a transient slurmrestd system user during the lifetime of the service and executes the daemon with this unprivileged user.

Reload systemd units and enable the service:

# systemctl daemon-reload && systemctl enable --now slurmrestd.service

Setup Agent

Copy Slurm JWT signing key and restrict permissions to this copy:

# cp /var/spool/slurm/jwt_hs256.key /var/lib/slurm-web/slurmrestd.key

Restrict access to this sensitive file with read permission to slurm-web system user only:

# chown slurm-web:slurm-web /var/lib/slurm-web/slurmrestd.key
# chmod 0400 /var/lib/slurm-web/slurmrestd.key

Test Access

To test Slurm-web agent and slurmrestd service configuration parameters, you can run slurm-web-connect-check utility. It tries to send HTTP request to slurmrestd with Slurm-web agent configuration parameters and reports the status. For example:

  • System packages

  • Docker

  • Podman

  • Compose

# slurm-web connect-check
$ docker run --rm \
    --user $(id -u slurm-web):$(id -g slurm-web) \
    -v /etc/slurm-web/agent.ini:/etc/slurm-web/agent.ini:ro \
    -v /var/lib/slurm-web:/var/lib/slurm-web \
    ghcr.io/rackslab/slurm-web-agent:latest connect-check
$ podman run --rm \
    --userns=keep-id --user $(id -u slurm-web):$(id -g slurm-web) \
    -v /etc/slurm-web/agent.ini:/etc/slurm-web/agent.ini:ro \
    -v /var/lib/slurm-web:/var/lib/slurm-web \
    ghcr.io/rackslab/slurm-web-agent:latest connect-check
With rootful Podman, omit --userns=keep-id. 
$ LOCAL_UID=$(id -u) LOCAL_GID=$(id -g) docker compose run --rm agent connect-check

With Podman Compose:

$ LOCAL_UID=$(id -u) LOCAL_GID=$(id -g) podman compose run --rm agent connect-check

When successful, the command prints:

✅ connection successful! (cluster: hpc, slurm: 25.11.0, api: 0.0.44)

Restart agent

Upon successful test, restart agent service to apply changes:

  • Native service

  • Production HTTP server

  • Containers

  • Compose

When using default native service:

# systemctl restart slurm-web-agent.service

When using production HTTP server:

# systemctl restart slurm-web-agent-uwsgi.service

When using raw containers:

$ docker restart slurm-web-agent

With Podman:

$ podman restart slurm-web-agent

When using the reference Compose stack:

$ LOCAL_UID=$(id -u) LOCAL_GID=$(id -g) docker compose restart agent

With Podman Compose:

$ LOCAL_UID=$(id -u) LOCAL_GID=$(id -g) podman compose restart agent