Error handling
What catch_errors installs, how a failure is reported and explained, the recovery menu after a failed install, and where the logs are.
Error handling lives in
core/error_handler.func.
A script enables it with one call and from then on must not install its own
ERR or EXIT traps.
catch_errors
catch_errorsNo arguments. It runs set -Ee -o pipefail and installs five traps:
| Trap | Handler | Result |
|---|---|---|
ERR | error_handler | report, explain, exit with the failing command's code |
EXIT | on_exit | final bookkeeping on every exit, successful or not |
INT | on_interrupt | Interrupted by user (SIGINT), exit 130 |
TERM | on_terminate | Terminated by signal (SIGTERM), exit 143 |
HUP | on_hangup | output moves to the log, the run continues |
set -u is added only when STRICT_UNSET=1.
Who calls it:
-
ui/build-ui.funcright after loadingcore/core.funcand this file, so the engine is covered while it loads. -
Every ct script, after
variablesandcolor(ct/vaultwarden.sh):header_info "$APP" variables color catch_errorsOn a Proxmox VE host this second call matters:
pve/backend.funcsets its ownEXIT,ERR,INTandTERMtraps when it loads, and the script'scatch_errorsputs the handlers described here back in place. -
lxc/install.funcduring the container bootstrap, and every install script again afterverb_ip6(install/vaultwarden-install.sh):source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" color verb_ip6 catch_errors setting_up_container network_check update_os -
Add-on scripts, after sourcing the files themselves.
On Incus hosts incus/core.func redefines catch_errors and error_handler
as its own incus_catch_errors and incus_error_handler, and
incus/build.func installs its own signal traps. The rest of this page
describes the handlers in core/error_handler.func, which are the ones used on
Proxmox VE hosts and inside containers on both platforms.
What "failure" means
With set -e and the ERR trap, any command that returns non-zero ends the
run, unless it is part of a condition (if, while, &&, ||) or you add
|| true. Commands run through $STD behave the same way: silent returns
the command's status and the trap fires on that line.
What error_handler does
In order:
- Works out the exit code, the command and the line. If the failing command
ran through
silent, the real command, its line and its captured output replace what the trap saw (which would only be thesilentwrapper or a helper'sreturn). - Treats the failure as exit 130 when the captured output shows that the
command received a Ctrl-C (
received signal 2orKeyboardInterrupt). - Inside a
$( )or( )subshell it only exits with the code; the calling shell fails on the same error and reports it once. - Stops the spinner and restores the cursor.
- Reports the failure once (see Who reports).
- Prints the error line.
- Prints what the failing command printed.
- Prints hints for known causes.
- On the host: prints
Full log: <path>and offers to remove a half-built container. - Exits with the original exit code.
The error line
✖️ in line 24: exit code 100 (APT: Package manager error (broken packages / dependency problems)): while executing command apt install -y build-essential| Part | Meaning |
|---|---|
in line N | Line of the script that was running. The host passes an install script to bash -c behind one extra line, so there N is one higher than the line in install/<app>-install.sh |
exit code C | The command's exit status |
(explanation) | Text for that status, see below |
while executing command … | The command, with the $STD prefix removed |
The explanation normally comes from explain_exit_code, the table on the
Exit codes page. The handler adjusts it to the command that
actually ran, because many tools reuse the same numbers:
| Situation | Explanation shown |
|---|---|
Exit 1 from (( … )) | Arithmetic expression evaluated to 0 |
Exit 1 from [[ … ]], [ … ] or test | Test condition was false |
VAR=$(…) failed | Command substitution exited with status N |
Exit 1 from grep, egrep, fgrep, zgrep, rg | grep found no match (with the tool's name) |
Exit 5 from systemctl | Systemd: unit not found |
The command is return or exit | Exit status N |
| The table entry belongs to a tool family (curl, APT, DPKG, Node.js, Python, PostgreSQL, MySQL/MariaDB, MongoDB, Systemd, git) and a different program ran | <program> exited with status N |
| Exit 1, 2 or 3 from a known program | <program> exited with status N |
So cargo build failing with 101 is reported as
cargo exited with status 101, not as the APT configuration error that 101
means for apt.
Output of the failing command
One of three blocks follows the error line, written to stderr:
| Heading | Shown when | Content |
|---|---|---|
--- Output of the failing command --- | the command ran through $STD | up to 25 lines of that command's own output |
--- Output of the failing step --- | otherwise, and api/errorlog.func is loaded | the log from the last [INFO]/[OK] line on, with progress noise removed, last 25 lines |
--- Last 20 lines of log --- | otherwise | the tail of the active log |
Hints
| Condition | Hint |
|---|---|
| Exit 127 and a command word that is not installed | <name> is not installed or not on PATH, plus where it normally comes from (setup_nodejs, setup_rust, setup_go, /usr/local/bin). Nothing is installed for you |
Exit 243, or the log mentions a JavaScript heap limit, or exit 134/137 from an npm, pnpm, yarn, node, vite or turbo command | Possible Node.js heap OOM. Try: export NODE_OPTIONS="--max-old-space-size=<MB>". The value is 75 % of the memory found, between 1024 and 12288 |
| Log shows unmet or broken package dependencies | APT dependency conflict, or a PostgreSQL version conflict when a postgresql-<N> package is named |
Log shows ERR_PNPM_OUTDATED_LOCKFILE, ERR_PNPM_LOCKFILE_CONFIG_MISMATCH or ELOCKVERIFY | The release's lockfile does not match its package.json |
Log shows NO_PUBKEY, KEYEXPIRED, is not signed, sqv | APT repository signature error |
Log shows Could not resolve, Failed to fetch, Network is unreachable and similar | Network or DNS failure |
Log shows Could not get lock, E: Unable to lock | APT or dpkg lock conflict |
Log shows No space left on device, ENOSPC, disk quota exceeded | Disk space exhausted |
The log-based hints look at the last 60 lines of the log and only the first matching row is shown.
Who reports: host and container
One run sends one terminal telemetry event, and the host sends it.
| Side | What happens on failure |
|---|---|
| Container | Nothing is sent. The handler writes /root/.install-<SESSION_ID>.failed (the exit code) and /root/.install-<SESSION_ID>.log.errinfo (the capture) |
| Host | Writes <log>.errinfo and calls post_update_to_api "failed" <code> |
The handler decides which side it is on with TELEMETRY_CONTEXT
(lxc/install.func exports container), and otherwise by looking for
pveversion, pct or incus.
The container is not silent on telemetry altogether: lxc/install.func sends
progress pings (post_progress_to_api) while the install runs. Only the final
status comes from the host. See Telemetry.
The .errinfo capture
Written by silent and by the error handler; the format is defined at the top
of api/errorlog.func:
EXIT_CODE=100
LINE=24
COMMAND=apt install -y build-essential
FUNCTION=setup_deb_based
STEP=Installing Dependencies
MESSAGE=
PHASE=container
--- OUTPUT ---
E: Unable to locate package build-essentialSTEP is the text of the msg_info that was open, or after <text> of the
last msg_ok. MESSAGE is the first msg_error of that step. PHASE is
host or container.
build_error_string turns this into the trace telemetry receives:
exit_code=100 | APT: Package manager error (broken packages / dependency problems) | step: Installing Dependencies | in setup_deb_based | at line 24: apt install -y build-essential
---
E: Unable to locate package build-essentialBefore it is sent, the output is reduced to the failing step, package-manager
and download progress lines are removed, IPv4 addresses are shortened to
<first octet>.x.x.x, and the result is limited to 60 lines and 10240 bytes.
Failures on the host
When a host-side step fails after the container exists, the handler asks:
Remove broken container 105? (Y/n) [auto-remove in 60s]:| Answer | Result |
|---|---|
Enter, y | container stopped and destroyed |
n | Container 105 kept for debugging |
| no answer for 60 seconds | container stopped and destroyed |
| the terminal cannot be read | container kept; pct destroy <id> is printed for manual removal |
The question is only asked for a container this run created
(_CS_CREATED_CTID). An add-on that sets CTID to an existing container never
gets it.
The recovery menu after a failed install
A failure of the install script inside the container is handled by
build_container in
pve/backend.func,
not by the ERR trap. After lxc-attach returns, the host reads
/root/.install-<SESSION_ID>.failed from the container. If it exists (or
lxc-attach itself returned non-zero), the host:
- prints
Installation failed in container <CTID> (exit code: N); - builds a combined log,
/tmp/<app>-<CTID>-<SESSION_ID>.log, from the host build log ("PHASE 1") and the install log pulled out of the container ("PHASE 2"), and pulls the.errinfocapture; - reports the failure to telemetry;
- prints
Installation log: <path>andError: <explanation>; - shows the menu.
What would you like to do?
1) Remove container and exit
2) Keep container for debugging
3) Retry with verbose mode (full rebuild)
4) Repair APT/DPKG state and re-run install (in-place)
Select option [1-4] (default: 1, auto-remove in 60s):Options 1 to 3 are always there. The others appear when the failure looks like a case they can fix, and are numbered from 4 in this order:
| Option | Offered when | What it does |
|---|---|---|
Remove container and exit | always | pct stop, pct destroy, exit with the install's code |
Keep container for debugging | always | leaves the container running and exits with the install's code |
Retry with verbose mode (full rebuild) | always | destroys the container, picks a valid container ID again, sets VERBOSE=yes and runs build_container again with the same settings |
Repair APT/DPKG state and re-run install (in-place) | exit 100, 101 or 102 from an apt/dpkg command; exit 255 or 1 with apt/dpkg errors in the output | runs dpkg --configure -a, apt-get -f install -y, apt-get clean, apt-get update in the container, then runs the install script again in the same container, verbose |
Repair APK state and re-run install (in-place) | the same, on Alpine | apk fix, apk cache clean, apk update, then the install script again |
Retry with more resources (RAM: …, CPU: …) | exit 134, 137 or 243; exit 1 with out-of-memory messages | destroys the container and rebuilds with RAM and cores doubled. Offered twice at most |
Retry with more disk space (Disk: …) | exit 228 or 23; ENOSPC or No space left on device in the output | destroys the container and rebuilds with the disk size doubled. Offered twice at most |
Retry with DNS override in LXC (8.8.8.8 / 1.1.1.1) | exit 6, 7, 22, 28, 35, 52, 56, 57, 75 or 78; exit 100 or 128 with network errors in the output; exit 1 with resolver or connection errors | destroys the container and rebuilds; after the base setup /etc/resolv.conf in the container is set to 8.8.8.8 and 1.1.1.1 |
How the prompt resolves:
| Input | Result |
|---|---|
| Enter | option 1 |
| nothing for 60 seconds | container removed |
| a number that is not offered | Invalid option. Container <CTID> kept. |
| the terminal cannot be read | container kept |
| the console was closed during the install | option 2 is taken automatically |
If the in-place repair does not help, the container is kept and the script exits with the new code. A rebuild that succeeds continues as a normal install.
Extra lines above the menu explain some codes: exit 10 (the container probably needs privileged mode or nesting), exit 125/126 (permission problem), a full disk (with the current disk size), and a missing command (its name).
Two dev mode flags skip the menu: keep leaves the container
untouched, breakpoint opens a shell in it.
On Incus, incus/backend.func has its own version of this step.
on_exit
Runs on every exit and keeps the exit code.
| Where | Exit code | Action |
|---|---|---|
| both | any | prints the dev_mode=timing summary; runs apt_conffile_report |
| both | 0 after an update | cleanup_lxc, then update_motd_ip if defined |
| both | 0 after a release was deployed | removes the .pending markers in $HOME |
| container | non-zero | writes the .failed flag and .errinfo if the ERR trap did not |
| container | 0 | creates /root/.install-<SESSION_ID>.done |
| host | non-zero, nothing reported yet | post_update_to_api "failed" <code> |
| host | 0, an install was started but no final status sent | post_update_to_api "aborted" 0 (you backed out of a dialog) |
| host | non-zero | pulls the install log from the container (ensure_log_on_host); stops the container if the install was still running |
| both | any | removes $lockfile if set |
Signals
| Signal | Handler | Behaviour |
|---|---|---|
Ctrl-C (SIGINT) | on_interrupt | Host reports aborted with code 130; container writes its failure flag. A container that is mid-install is stopped. Exit 130 |
SIGTERM | on_terminate | The same with code 143 |
SIGHUP | on_hangup | Not an abort, see below |
Closing the console
Closing the browser console or losing the SSH session sends SIGHUP. The run
is not aborted:
on_hanguppoints stdin at/dev/nulland stdout/stderr at the active log, exports_CS_HANGUP=1, and writesThe terminal closed; continuing, output follows here.to the log.- Programs started through
$STDignoreSIGHUPand finish. - On the host,
lxc-attachdies with the terminal but the install inside the container keeps running. The host then waits for/root/.install-<SESSION_ID>.doneor.failedin the container, checking every 15 seconds for up to six hours. - If the install then fails, nobody can answer the recovery menu, so the container is kept.
To watch an install whose console you closed:
pct exec <CTID> -- tail -f /root/.install-<SESSION_ID>.logCtrl-C does abort.
What is logged where
| File | Side | Content |
|---|---|---|
/tmp/create-lxc-<SESSION_ID>.log | host | BUILD_LOG: host messages and the output of host commands run through $STD |
/tmp/pct_create_<CTID>_<timestamp>_<SESSION_ID>.log | host | output of pct create; appended to the build log |
/root/.install-<SESSION_ID>.log | container | INSTALL_LOG: everything the install script logged |
/root/.install-<SESSION_ID>.log.errinfo | container | capture of the failing command |
/root/.install-<SESSION_ID>.failed | container | exit code of a failed install; read and removed by the host |
/root/.install-<SESSION_ID>.done | container | empty marker of a finished install |
/root/.install-<SESSION_ID>.pid | container | PID of the running install script |
/tmp/<app>-<CTID>-<SESSION_ID>.log | host | combined log after a failed install |
/tmp/.errinfo-<SESSION_ID> | host | the container's capture, pulled for telemetry |
/tmp/build-<date>_<time>.log | either | scripts without a build log, such as add-ons |
DEBUG_LOGFILE | either | when you set this variable, each failure appends a block with timestamp, exit code, line and command |
SESSION_ID is the first eight characters of the run's random UUID.
With dev_mode=logs the host build log is written to
/var/log/community-scripts/ instead of /tmp.
Log lines written by the message functions look like this:
[2026-01-15 10:32:07] [INFO] Installing Dependencies
[2026-01-15 10:32:41] [OK] Installed DependenciesRules for script authors
- Call
catch_errorsonce, where the templates do. Do not addtrap … ERRortrap … EXITof your own. - Do not call the telemetry API from an install script.
- Let commands fail with their real exit status. Use an explicit
exit <code>only with a code from the exit-code table that says what happened. - Print a
msg_errorbefore a deliberateexit; it becomes the failure's message. - Use
|| truefor commands that may fail, notset +e.