jeffersonlab/adm repository overview

⁠adm

A Jakarta EE 10⁠ web application for app deployment at Jefferson Lab built with the Smoothness⁠ web template.

• Overview⁠
• Quick Start with Compose⁠
• Install⁠
• Configure⁠
• API⁠
• Build⁠
• Develop⁠
• Release⁠
• Deploy⁠
• See Also⁠

⁠Overview

The automated deployment manager app aids programmatic app deployment by providing an HTTP endpoint for executing curated remote deployment commands securely.

⁠Quick Start with Compose

1. Grab project

git clone https://github.com/JeffersonLab/adm
cd adm

1. Launch Compose⁠

docker compose up

1. Navigate to page

http://localhost:8080/adm

Note: Login with demo username "tbrown" and password "password". Use env "local-demo", app "testapp", ver "1.0.0".

⁠Ports

The containers publish these ports on the host, listening only on its local address (127.0.0.1), so other computers can't reach the demo database, accounts, and management consoles. To use them from another computer, forward them over SSH, such as ssh -N -L 8443:localhost:8443 -L 8081:localhost:8081 <host> for the app and Keycloak. If another project already uses some of them, or you run a second copy of adm, set other ports in a .env file next to compose.yaml, which Compose reads, or in the environment:

For example, a .env for a second copy:

ADM_HTTPS_PORT=18443
ADM_HTTP_PORT=18080
ADM_MANAGEMENT_PORT=19990
ADM_KEYCLOAK_PORT=18081
ADM_KEYCLOAK_MANAGEMENT_PORT=19991
ADM_ORACLE_PORT=11521
ADM_ORACLE_EM_PORT=15500
ADM_SSHD_PORT=11234

Compose names the containers, network, and images after the project's directory, such as adm-oracle-1, so a copy in another directory, such as a git worktree, gets its own. Each copy uses about 4 GB of memory.

⁠Install

This application requires a Java 17+ JVM and standard library to run, plus a Jakarta EE 10 application server (developed with Wildfly).

1. Install service dependencies⁠
2. Download Wildfly 37.0.1⁠
3. Configure⁠ Wildfly and start it
4. Download adm.war⁠ and deploy it to Wildfly
5. Navigate your web browser to localhost:8080/adm⁠

⁠Configure

⁠Configtime

Wildfly must be pre-configured before the first deployment of the app. The wildfly bash scripts⁠ can be used to accomplish this. See the Dockerfile⁠ for an example.

⁠Runtime

Uses a subset of the Smoothness Environment Variables⁠ including:

• BACKEND_SERVER_URL
• FRONTEND_SERVER_URL

The deployment commands rely on ssh and therefore the user running Wildlfy needs to have ssh configured such that remote connections can be made non-interactively (programmatically) without password prompt. This generally means creating a public and private key pair for the Wildfly user via ssh-keygen and then add the public key to the authorized_keys file of each remote user account needed for deployments. It's also possible to use Kerberos and keytabs instead.

⁠Database

The application requires an Oracle 19+ database with the following schema⁠ installed. The application server hosting the app must also be configured with a JNDI datasource.

⁠Deploying to several hosts

An app env's Deploy Hostname can list several hosts, separated by commas, such as epicsweb1, epicsweb2, epicsweb3 for an app that runs on each. A deploy then runs the Deploy Command as the Deploy Username on each host in turn, in the order listed, with the same port, and stops at the first host where it fails, so a version that fails on one host doesn't reach the rest. It is one deploy job, which succeeds only if the command succeeded on every host.

⁠Request Username presets

The Deployment Allowlist's Add and Edit dialog offers the CI service accounts that request deploys as presets, so admins needn't type them. Each preset is a setting with that account's username, which admins can change on the Settings page:

A preset is offered only when its setting exists, so an installation with neither offers only a custom account. To add them to an existing database, insert the rows as 04_settings.sql⁠ does, with your installation's usernames, then restart the app, or change any setting on the Settings page, so it reads them.

⁠API

Clients such as CI authenticate with a Keycloak bearer token, usually a service account's from the client credentials grant, as the Deploy to JLab⁠ action does. Only an app env's request service user and users with the adm-admin role may deploy to it and see its jobs.

A request without a token is redirected to the login page (HTTP status 302), and one with an invalid or expired token returns HTTP status 401. Access tokens are short-lived, 5 minutes by default in Keycloak, and a deploy may take longer, so a client that waits for a job should request a new token for each poll.

⁠Deploy

POST /adm/deploy with the form parameters env, app and ver (a semantic version) starts a deploy job and returns its ID at once:

{"jobId": 13}

A refused deploy returns its reason, with HTTP status 200: {"exception": "..."}.

⁠Job status

GET /adm/job?id=<jobId> returns a deploy job's status:

{"jobId": 13, "app": "testapp", "env": "local-demo", "version": "1.2.3",
 "start": "2026-10-05T14:12:55-04:00", "end": "2026-10-05T14:12:56-04:00",
 "exitCode": 0, "out": "1.2.3\n", "err": null, "error": null}

• end and exitCode are null while the job runs; poll until end is set. A deploy command is stopped after 10 minutes; for an env with several hosts⁠, the 10 minutes are for all of them together. The deploy succeeded if exitCode is 0.
• out and err are the deploy command's standard output and standard error. error says why the command could not run or finish, such as an unknown host or a timeout. Each is cut to its last 64 KiB, and is null when empty.
• For an env with several hosts, exitCode is the last host's to run, and error names the host that failed and those not deployed to. Each host's part of out starts with a line naming it, such as == epicsweb1 ==, and so does its part of err if it has one.
• A missing or invalid ID returns HTTP status 400, a caller who may not see the job 403, and an unknown job 404, each with the reason: {"exception": "..."}.

For example, to wait for a job, for a little longer than a deploy command may run, with a new token for each poll, and fail unless it succeeded:

token() {
  curl -sS --fail-with-body -u "${CLIENT_ID}:${SECRET}" -d "grant_type=client_credentials" "https://ace.jlab.org/auth/realms/ace/protocol/openid-connect/token" | jq -er '.access_token'
}

for i in $(seq 1 130); do
  ACCESS_TOKEN=$(token) || { echo "token request failed"; exit 1; }
  STATUS=$(curl -sS --fail-with-body -H "Authorization: Bearer ${ACCESS_TOKEN}" "https://ace.jlab.org/adm/job?id=${JOB_ID}") || { echo "${STATUS}"; exit 1; }
  [ "$(echo "${STATUS}" | jq '.end')" != "null" ] && break
  sleep 5
done
echo "${STATUS}"
echo "${STATUS}" | jq -e '.exitCode == 0'

⁠Build

This project is built with Java 21⁠ (compiled to Java 17 bytecode), and uses the Gradle 9⁠ build tool to automatically download dependencies and build the project from source:

git clone https://github.com/JeffersonLab/adm
cd adm
gradlew build

Note: If you do not already have Gradle installed, it will be installed automatically by the wrapper script included in the source

Note for JLab On-Site Users: Jefferson Lab has an intercepting proxy⁠

⁠Develop

In order to iterate rapidly when making changes it's often useful to run the app directly on the local workstation, perhaps leveraging an IDE. In this scenario run the service dependencies with:

docker compose -f deps.yaml up

Note: The local install of Wildfly should be configured⁠ to proxy connections to services via localhost and therefore the environment variables should contain:

KEYCLOAK_BACKEND_SERVER_URL=http://localhost:8081
FRONTEND_SERVER_URL=https://localhost:8443

Further, the local DataSource must also leverage localhost port forwarding so the standalone.xml connection-url field should be: jdbc:oracle:thin:@//localhost:1521/xepdb1.

The server⁠ and app⁠ setup scripts can be used to setup a local instance of Wildfly.

The user you use to run Wildfly needs to have an SSH public/private key pair (ssh-keygen) in the default location (~/.ssh). The public key needs to be added to the authorized_keys file of user testuser in the container of the sshd service. This can be done by creating a file named .env in the root of the project containing the env name "TEST_USER_AUTHORIZED_KEY" with value being Wildfly user's public key. This env will then be passed in via deps.yaml environment setting.

The unit tests run with gradlew build, or alone with gradlew test. They need no database, Wildfly, or containers: the SSH tests start their own in-process SSH server.

The integration tests run against the app and its services in containers, built from source, and are not part of gradlew build:

docker compose -f build.yaml up -d --build
gradlew integrationTest

They wait up to 5 minutes for the services to start (set ADM_READY_TIMEOUT_SECONDS to change that), then log in with the demo users and the adm client's service account, as CI deploys do. They add their own app envs to deploy to, and remove them afterwards. They use the same ports⁠ as Compose, including those in .env. To test an app elsewhere, set ADM_URL (default https://localhost:8443/adm) and KEYCLOAK_URL (default http://localhost:8081/auth).

⁠Release

1. Bump the version number in the VERSION file (using Semantic Versioning⁠) in a pull request. Give its description a ## Release notes section, with ### Highlights and any ### Upgrade steps such as database changes, and put checks and attribution under later ## headings so they aren't published. See Release notes from a pull request⁠.
2. Merge the pull request.
3. The CD⁠ GitHub Action should run automatically invoking:
   • The Create release⁠ GitHub Action to tag the source and create release notes: the merged pull request's Release notes section, followed by GitHub's generated summary of the pull requests. A war file artifact is attached to the release.
   • The Publish docker image⁠ GitHub Action to create a new demo Docker image.
   • The Deploy to JLab⁠ GitHub Action to deploy to the JLab test environment.

⁠Deploy

The deploy to JLab's acctest is handled automatically via the release workflow.

At JLab this app is found at ace.jlab.org/adm⁠ and internally at acctest.acc.jlab.org/adm⁠. However, those servers are proxies for wildfly1.acc.jlab.org and wildflytest1.acc.jlab.org respectively. A deploy script⁠ is provided on each server to automate wget and deploy. Example:

/opt/wildfly/cd/deploy.sh adm v1.2.3

JLab Internal Docs: RHEL9 Wildfly⁠

⁠See Also

• JLab ACE management-app list⁠