Slurmrestd Access

This page describes how-to setup Slurm slurmrestd service and Slurm-web agent with:

Unix socket,
JWT authentication,
static token management mode.

< back to all slurmrestd setups

Setup Slurm

With jwt authentication method, the client must provide user and token in HTTP headers. Then, slurmrestd service forwards these credentials in RPCs to slurmctld and slurmdbd which are responsible to check expiration and signature of the token .

In this configuration, slurmrestd service and Slurm-web agent can run with unprivileged system users.

Create /etc/systemd/system/slurmrestd.service.d/slurm-web.conf drop-in configuration override for slurmrestd service:

[Service]
# Unset vendor unit ExecStart to avoid cumulative definition
ExecStart=
Environment=
ExecStart=/usr/sbin/slurmrestd $SLURMRESTD_OPTIONS -a rest_auth/jwt unix:/run/slurmrestd/slurmrestd.socket
RuntimeDirectory=slurmrestd
RuntimeDirectoryMode=0755
User=slurmrestd
Group=slurmrestd
DynamicUser=yes

With DynamicUser=yes, systemd creates a transient slurmrestd system user during the lifetime of the service and executes the daemon with this unprivileged user.

Reload systemd units and enable the service:

# systemctl daemon-reload && systemctl enable --now slurmrestd.service

Generate random Slurm JWT signing key with restrictive permissions:

# dd if=/dev/random of=/var/spool/slurm/jwt_hs256.key bs=32 count=1
# chown slurm:slurm /var/spool/slurm/jwt_hs256.key
# chmod 0600 /var/spool/slurm/jwt_hs256.key

Edit main Slurm and SlurmDBD configuration to enable JWT alternative authentication:

AuthAltTypes=auth/jwt
AuthAltParameters=jwt_key=/var/spool/slurm/jwt_hs256.key

Restart slurmctld and slurmdbd services to update configuration:

# systemctl restart slurmctld slurmdbd

Setup Agent

Generate a token with Slurm for slurm admin user:

# scontrol token lifespan=infinite username=slurm

In this example, we generate a token with an infinite lifespan to avoid its expiration and the requirement to update the token on a regular basis. You can also choose to generate tokens with a short lifespan and manage these updates.

Copy the token in output and edit Slurm-web agent configuration file /etc/slurm-web/agent.ini:

[slurmrestd]
auth=jwt
jwt_mode=static
jwt_token=<secret_token>

To improve security, edit Slurm-web agent service to run as unprivileged slurm-web system user:

With native services, edit agent service settings:

# systemctl edit slurm-web-agent.service

Add the following lines:

[Service]
User=slurm-web

Restart the service:

# systemctl restart slurm-web-agent.service

With production HTTP server, edit agent uWSGI service settings /etc/systemd/system/slurm-web-agent-uwsgi.service:

--- a/etc/systemd/system/slurm-web-agent-uwsgi.service
+++ b/etc/systemd/system/slurm-web-agent-uwsgi.service
@@ -6,7 +6,7 @@
 # By default, this service runs with slurm admin user for local authentication
 # on slurmrestd. When slurmrestd is setup with JWT authentication, it is
 # recommended to change this value to more restricted slurm-web system user.
-User=slurm
+User=slurm-web
 RuntimeDirectory=slurm-web-agent
 ExecStart=/usr/sbin/uwsgi --ini /usr/share/slurm-web/wsgi/agent/slurm-web-agent.ini

Reload service units:

# systemctl daemon-reload

Restart Slurm-web agent:

# systemctl restart slurm-web-agent.service

Test Access

To test Slurm-web agent and slurmrestd service configuration parameters, you can run slurm-web-connect-check utility. It tries to send HTTP request to slurmrestd with Slurm-web agent configuration parameters and reports the status. For example:

# /usr/libexec/slurm-web/slurm-web-connect-check
✅ connection successful (slurm: 24.11.0, cluster: hpc)