Releases and downloads (forge)

Source: lib/forge.func. It is loaded through lib/tools.func, see how helpers reach a script.

Almost every app script uses two functions from this file: check_for_gh_release in update_script and fetch_and_deploy_gh_release in both the install and the update path. The GitLab and Codeberg variants are the same implementation with a different forge name.

Concepts shared by all functionsCopy Anchor Link

The version stampCopy Anchor Link

Every deploy helper records what it installed in $HOME/.<app>, where <app> is the app name in lower case with spaces removed ("Crafty-Controller" becomes ~/.crafty-controller). Every check_for_* helper compares that file with upstream. Use the same app name in both calls, otherwise the check never matches.

fetch_and_deploy_release additionally writes $HOME/.<app>.pending when it starts. The marker holds the version the update started from and is removed by the engine's exit handler only when the script exits 0. While it exists, check_for_release reports an update again ("the last run did not finish, doing it again") and the deploy runs again even if the stamp already matches.

Variables the helpers setCopy Anchor Link

Tokens and hostsCopy Anchor Link

Rate limitsCopy Anchor Link

The GitHub API allows a limited number of unauthenticated calls per address. When it answers 403 to github_api_call (there also 429) or to a deploy helper:

1. For a release lookup (/releases/latest or /releases/tags/<tag>), the helper reads the same release from github.com without the API and prints GitHub API rate limit reached; read the release from github.com instead. A token avoids this: var_github_token. This fallback is not used with a tag prefix, nor for "latest" with GH_INCLUDE_PRERELEASE=1.
2. Otherwise it waits (2 s, then 4 s) and retries, three attempts in total.
3. Then it calls prompt_for_github_token. With a token the counters start over.
4. Without a token it returns 22.

In the deploy helpers GitLab and Codeberg (status 429) get the retries but neither the fallback nor the prompt.

The check_for_* helpers do none of this. A rate-limited check prints github: rate limit reached. Raise it with: export GITHUB_TOKEN=ghp_your_token and returns 22 at once.

Clean installsCopy Anchor Link

The deploy helpers copy the new files over the target directory. Old files that no longer exist upstream stay unless you ask for a wipe:

Anything else you need to keep (an .env, uploads) has to be saved first, see create_backup and restore_backup in system helpers. From ct/adventurelog.sh:

CLEAN_INSTALL=1 CLEAN_INSTALL_KEEP="backend/server/media" fetch_and_deploy_gh_release "adventurelog" "seanmorley15/adventurelog" "tarball"

Check for an updateCopy Anchor Link

check_for_gh_release, check_for_gl_release, check_for_codeberg_releaseCopy Anchor Link

Report whether upstream has a release that differs from the installed one. The three are wrappers around check_for_release with the forge filled in.

check_for_gh_release <app> <owner/repo> [pinned_version] [pin_reason] [tag_prefix]
check_for_gl_release <app> <namespace/project> [pinned_version] [pin_reason] [tag_prefix]
check_for_codeberg_release <app> <owner/repo> [pinned_version] [pin_reason] [tag_prefix]

Which release is "the latest":

• With a pin, the pinned tag is requested directly.
• Without pin, prefix and pre-releases, the release the maintainer marked as latest on the forge is used.
• Otherwise up to 100 releases are listed, drafts and pre-releases are dropped (on GitLab: upcoming releases), and the highest version wins.

The installed version counts as current when it equals that release, or when both look like dotted versions and the installed one is already ahead.

If ~/.<app> is missing and /opt holds exactly one *_version.txt file, that file is moved to ~/.<app> (migration from old installs).

Scripts use it only as a condition. From ct/listenarr.sh:

if GH_INCLUDE_PRERELEASE=1 check_for_gh_release "listenarr" "Listenarrs/Listenarr"; then

With a pin and a reason, from ct/affine.sh:

if check_for_gh_release "affine_app" "toeverything/AFFiNE" "${RELEASE}" "each release is tested individually before the version is updated. Please do not open issues for this"; then

On another GitLab instance, from ct/tor-snowflake.sh:

if GITLAB_URL="https://gitlab.torproject.org" check_for_gl_release "tor-snowflake" "tpo/anti-censorship/pluggable-transports/snowflake"; then

Codeberg, from ct/forgejo.sh:

if check_for_codeberg_release "forgejo" "forgejo/forgejo"; then

check_for_releaseCopy Anchor Link

The implementation behind the three wrappers, with the forge as first argument.

check_for_release <github|gitlab|codeberg> <app> <repo> [pinned_version] [pin_reason] [tag_prefix]

Arguments, variables and return codes are those of the wrappers above. An unknown forge name returns 22. No script in ProxmoxVE calls it directly; they use the wrappers.

check_for_gh_tagCopy Anchor Link

The same check for GitHub repositories that publish tags but no releases.

check_for_gh_tag <app> <owner/repo> [prefix]

Returns 0 with CHECK_UPDATE_RELEASE set to the tag when it differs from the stamp, 1 when it matches, 22 when the tag lookup failed. From ct/drawdb.sh:

if check_for_gh_tag "drawdb" "drawdb-io/drawdb"; then

check_for_gh_branchCopy Anchor Link

The same check for apps deployed from a branch: compares the short commit SHA in the stamp with the branch tip, using git ls-remote.

check_for_gh_branch <app> <owner/repo> [branch]

Installs git if it is missing. From ct/vane.sh:

if check_for_gh_branch "searxng" "searxng/searxng" "master"; then

Deploy a releaseCopy Anchor Link

fetch_and_deploy_gh_release, fetch_and_deploy_gl_release, fetch_and_deploy_codeberg_releaseCopy Anchor Link

Download a release and put it in place, then write the version stamp. Wrappers around fetch_and_deploy_release.

fetch_and_deploy_gh_release <app> <owner/repo> [mode] [version] [target] [asset_pattern] [tag_prefix]
fetch_and_deploy_gl_release <app> <namespace/project> [mode] [version] [target] [asset_pattern] [tag_prefix]
fetch_and_deploy_codeberg_release <app> <owner/repo> [mode] [version] [target] [asset_pattern] [tag_prefix]

Behaviour worth knowing:

• The call is skipped with <app> <version> already installed when the stamp matches the selected release and the target directory exists (in binary mode the stamp alone is enough), unless FORCE_UPDATE=1 or a .pending marker exists.
• The stamp stores the tag without a leading v when a digit follows it.
• If the selected release has no matching asset (binary, prebuild, singlefile), the last 15 releases are searched for one that has. On a terminal the user is asked Use version <tag> instead? [Y/n] (auto-yes in 60s); without a terminal the older release is used. Codeberg has no such search.

A failed connection to the API returns curl's own exit code.

One example per mode:

# tarball, ct/2fauth.sh
fetch_and_deploy_gh_release "2fauth" "Bubka/2FAuth" "tarball"

# binary, install/authelia-install.sh
fetch_and_deploy_gh_release "authelia" "authelia/authelia" "binary"

# prebuild, ct/listenarr.sh
GH_INCLUDE_PRERELEASE=1 CLEAN_INSTALL=1 fetch_and_deploy_gh_release "listenarr" "Listenarrs/Listenarr" "prebuild" "latest" "/opt/listenarr" "listenarr-*-linux-x64.zip"

# singlefile, install/argus-install.sh
fetch_and_deploy_gh_release "Argus" "release-argus/Argus" "singlefile" "latest" "/opt/argus" "Argus*linux-$(arch_resolve)"

With a tag prefix, from ct/deemix.sh:

CLEAN_INSTALL=1 fetch_and_deploy_gh_release "deemix" "bambanah/deemix" "tarball" "latest" "/opt/deemix" "" "deemix-webui@"

Keeping the user's config files of a .deb, from ct/caddy.sh:

DPKG_FORCE_CONFOLD=1 fetch_and_deploy_gh_release "caddy" "caddyserver/caddy" "binary" "latest" "/opt/caddy" "caddy_*_linux_$(arch_resolve).deb"

Keeping the asset's file name, from ct/komga.sh:

USE_ORIGINAL_FILENAME="true" fetch_and_deploy_gh_release "komga-org" "gotson/komga" "singlefile" "latest" "/opt/komga" "komga*.jar"

GitLab, from ct/storyteller.sh:

CLEAN_INSTALL=1 CLEAN_INSTALL_KEEP="data" fetch_and_deploy_gl_release "storyteller" "storyteller-platform/storyteller" "tarball" "latest" "/opt/storyteller" "" "web-v2"

Codeberg, from ct/forgejo.sh:

fetch_and_deploy_codeberg_release "forgejo" "forgejo/forgejo" "singlefile" "latest" "/opt/forgejo" "forgejo-*-linux-$(arch_resolve)"

fetch_and_deploy_releaseCopy Anchor Link

The implementation behind the three wrappers, with the forge as first argument.

fetch_and_deploy_release <github|gitlab|codeberg> <app> <repo> [mode] [version] [target] [asset_pattern] [tag_prefix]

Everything else is as described above; an unknown forge returns 65. No script in ProxmoxVE calls it directly.

fetch_and_deploy_gh_tagCopy Anchor Link

Deploy the source archive of a GitHub tag, for repositories without releases.

fetch_and_deploy_gh_tag <app> <owner/repo> [tag] [target]

Skipped with is already up-to-date when the stamp equals the tag. The stamp holds the tag exactly as upstream names it.

From install/drawdb-install.sh:

fetch_and_deploy_gh_tag "drawdb" "drawdb-io/drawdb" "latest" "/opt/drawdb"

fetch_and_deploy_gl_tagCopy Anchor Link

The same for a GitLab tag, with an optional glob to choose among tags.

fetch_and_deploy_gl_tag <app> <namespace/project> [tag_pattern] [target]

Return codes are those of fetch_and_deploy_gh_tag. No script in ProxmoxVE calls it at the moment.

fetch_and_deploy_gh_branchCopy Anchor Link

Deploy a GitHub repository from a branch, for projects without releases or tags.

fetch_and_deploy_gh_branch <app> <owner/repo> [branch] [target]

If the target already is a git checkout it is updated in place (git fetch --depth 1, git reset --hard, git clean -fd). Otherwise the repository is cloned next to the target, the target is emptied and the clone copied in, so a failed clone leaves the target untouched. The stamp receives the 7-character commit SHA.

Returns 0 on success, 65 when app or repo is missing, 22 for any git or copy failure. From install/vane-install.sh:

fetch_and_deploy_gh_branch "searxng" "searxng/searxng" "master"

fetch_and_deploy_from_urlCopy Anchor Link

Download an archive or .deb from any URL and extract or install it. The type is detected from the file content with file, not from the URL.

fetch_and_deploy_from_url <url> [directory]

Compressed tarballs, plain tar, zip and 7z are extracted (a single top-level directory is stripped); a Debian package is installed with apt install, falling back to dpkg -i. No version stamp is written.

From ct/fileflows.sh (archive) and ct/foldingathome.sh (.deb):

fetch_and_deploy_from_url "https://fileflows.com/downloads/ff-latest.tar.xz" "/opt/fileflows"
fetch_and_deploy_from_url "https://download.foldingathome.org/releases/public/fah-client/$(arch_resolve "debian-10-64bit" "debian-stable-arm64")/release/latest.deb"

Look up a versionCopy Anchor Link

These print a version on stdout and change nothing. Use them when a script needs the number itself, for example to build a download URL.

get_latest_github_releaseCopy Anchor Link

Print the tag of the release GitHub marks as latest.

get_latest_github_release <owner/repo> [strip_v]

Returns 0, 22 when the API call failed, 250 when the response held no tag. From ct/actualbudget.sh:

RELEASE=$(get_latest_github_release "actualbudget/actual")

get_latest_gh_tagCopy Anchor Link

Print the latest tag of a GitHub repository.

get_latest_gh_tag <owner/repo> [prefix]

The tag is printed as is, including the prefix. Returns 0, 22 when the API call failed, 250 when no tag was found, 100 when jq could not be installed. From install/safebucket-install.sh:

GARAGE_VERSION=$(get_latest_gh_tag "deuxfleurs-org/garage" "v")

get_latest_codeberg_releaseCopy Anchor Link

Print the tag of the first release in Codeberg's list that is neither a draft nor a pre-release.

get_latest_codeberg_release <owner/repo> [strip_v]

strip_v defaults to true. Returns 0, 22, 250 or 100 as above. From ct/autocaliweb.sh:

RELEASE=$(get_latest_codeberg_release "gelbphoenix/autocaliweb")

get_latest_gitlab_releaseCopy Anchor Link

Print the highest-versioned tag among the 100 most recent GitLab releases.

get_latest_gitlab_release <namespace/project> [strip_v]

strip_v defaults to true and removes a leading v only when a digit follows. Reads GITLAB_TOKEN and GITLAB_URL. Returns 0, 22, 250 or 100 as above. No script in ProxmoxVE calls it at the moment.

get_latest_gl_tagCopy Anchor Link

Print the latest GitLab repository tag, optionally limited by a glob.

get_latest_gl_tag <namespace/project> [pattern]

Tags containing alpha, beta or rc are always skipped. Returns 0, 22 when the API did not answer 200, 250 when nothing matched. Used by fetch_and_deploy_gl_tag.

extract_version_from_jsonCopy Anchor Link

Print one field of a release JSON document.

extract_version_from_json <json> [field] [strip_v]

Returns 250 when the field is empty. Used by get_latest_github_release.

Raw API access and tokensCopy Anchor Link

github_api_callCopy Anchor Link

GET a GitHub API URL with authentication, retries and the rate-limit handling described above.

github_api_call <url> [output_file]

Returns 0 on HTTP 200, otherwise 22 after printing the reason (401, rate limit, 404, no response, other status). A 401 offers prompt_for_github_token first. From install/victoriametrics-install.sh:

github_api_call "https://api.github.com/repos/VictoriaMetrics/VictoriaLogs/releases/latest" "$vl_release_json"

codeberg_api_callCopy Anchor Link

GET a Codeberg API URL with three attempts.

codeberg_api_call <url> [output_file]

output_file defaults to /dev/stdout. Returns 0 on HTTP 200, otherwise 22. No token is sent. Used by get_latest_codeberg_release.

prompt_for_github_tokenCopy Anchor Link

Obtain a GitHub token after an authentication or rate-limit failure.

prompt_for_github_token

Without a terminal on stdin it only tries var_github_token. On a terminal it asks Would you like to enter a GitHub Personal Access Token (PAT)? [y/N] and repeats the input until validate_github_token accepts the token. Returns 0 when it set GITHUB_TOKEN (from var_github_token or from the input), 1 otherwise, also when GITHUB_TOKEN was already set and there is no terminal. Scripts do not call it; the GitHub helpers do.

validate_github_tokenCopy Anchor Link

Check a token against https://api.github.com/user.

validate_github_token [token]

token defaults to GITHUB_TOKEN.

Internal functionsCopy Anchor Link

Names starting with _ are not part of the interface for scripts. They can change without notice.