Troubleshooting

---

title: “Troubleshooting” description: “Troubleshooting issues” lead: “” date: 2023-06-12T12:20:59-06:00 lastmod: 2023-06-12T12:20:59-06:00 draft: false images: [] menu: docs: parent: “help” weight: 502 toc: true

Appliance Logs

The Trimble Access Management team can monitor appliance health as metrics and logs regarding the appliance will be forwarded to the Trimble Access Management team. You can also determine health by inspecting the health endpoint or logs that are JSON formatted to stdout on your running appliance container.

Server Error Codes

When calling a server API, you may receive an ErrorResponse with a Request ID and error message as part of an error. This can help the Trimble Access Management team triage your issue and provide better guidance when contacting them.

Scope of This Guide

Use this page to triage TAM Appliance incidents across these deployment modes:

• Main appliance runtime
• Appliance extension runtime
• Embedded runtime
• Offline and restricted-network (including gov cloud style) runtime

This guide consolidates common failures and recovery actions so you can diagnose quickly without switching between mode-specific troubleshooting files.

Quick Triage Workflow

1. Confirm deployment mode and runtime state (process/container up, restart history, recent config changes).
2. Check readiness (GET /health/readyz) when available and collect a short log window around the first failure.
3. Match the error text to an issue in the lookup table below.
4. Apply category recovery steps (startup, auth, bundle, or decision).
5. If unresolved, collect the escalation package in Incident and Escalation Checklist.

Health Check by Deployment Mode

Mode	Fast checks
Main appliance	Process/container running; /health/readyz healthy; ENVIRONMENT, CLIENT_ID, CLIENT_SECRET, API_PORT set; bundle endpoint reachable.
Extension runtime	Extension process running; health endpoint reachable where exposed; CLIENT_ID and CLIENT_SECRET valid; bundle source reachable.
Embedded runtime	Host process running; policy engine initialized; authorization header forwarding in place; decision payload shape valid.
Offline / gov cloud	OFFLINE_MODE=true; BUNDLE_PATH readable and points to .tar.gz; readiness healthy; no bundle status errors in logs.

If needed, set LOG_LEVEL=debug, restart, and retest to capture high-value diagnostics.

Unified Error Lookup

Use this section to quickly map log text to root-cause category and affected mode.

Error text to search	Category	Applies to
client credentials are not configured	Startup/config	Main, Offline
invalid_client	Startup/config	Main, Extension, Offline
Token generation failed for provided scopes	Startup/config	Main, Extension, Offline
Cannot launch API server - API listening port is not set via API_PORT env var	Startup/config	Main
policy engine not initialized yet	Startup/config	Main, Embedded
unable to load the bundle in the provided path	Bundle	Main, Extension, Offline
Provided path is not a .tar.gz file	Bundle	Main, Extension, Offline
Found errors in policy engine status update	Bundle	Main, Extension, Offline
Local bundle is outdated	Bundle	Offline
request is missing authorization header	Authentication	Main, Extension, Embedded, Offline
request header is missing bearer token	Authentication	Main, Extension, Embedded, Offline
JSONParseError	Decision/policy	Main, Extension, Embedded, Offline
UndefinedDecisionError	Decision/policy	Main, Offline
RegoBuiltinError	Decision/policy	Main, Extension, Offline
RegoEvaluationError	Decision/policy	Main, Extension, Embedded, Offline
OPA Decision Result is an error	Decision/policy	Main, Extension, Offline
Server Panic	Runtime/fatal	Main, Extension, Offline

Detailed Resolution by Category

Startup and Configuration

Missing or invalid client credentials

Symptoms:

• Startup fails quickly, or token request returns invalid_client.

Recovery:

1. Verify CLIENT_ID and CLIENT_SECRET are present and mapped to the correct tenant/environment.
2. Rotate credentials if they were revoked or expired.
3. Restart the runtime and verify readiness.

Scope configuration failure

Symptoms:

• Startup or registration flow reports token generation failures for provided scopes.

Recovery:

1. Validate scope metadata from package registration or cached registration data.
2. Correct invalid scope values and retry registration/startup.

API port missing (main mode)

Symptoms:

• API server fails to launch with API_PORT not set.

Recovery:

1. Set API_PORT to a valid, free port.
2. Restart and confirm the service binds successfully.

Policy engine not ready

Symptoms:

• Requests fail while engine initialization is still in progress.

Recovery:

1. Wait for initial policy/bundle load to complete.
2. Check logs for bundle activation failures before retrying decisions.

Bundle and Policy Artifact

Invalid or unreadable bundle path

Symptoms:

• Runtime reports it cannot load a bundle from BUNDLE_PATH.

Recovery:

1. Ensure BUNDLE_PATH points to an existing readable file.
2. Validate runtime filesystem permissions.
3. Restart and confirm bundle load success.

Invalid bundle format

Symptoms:

• Startup or activation fails with bundle format errors.

Recovery:

1. Replace the artifact with a valid .tar.gz bundle.
2. Restart and verify policy engine status.

Bundle status update contains errors

Symptoms:

• Periodic status logs report bundle or activation errors.

Recovery:

1. Inspect bundle status message payloads for exact failure reason.
2. Validate endpoint reachability and artifact integrity.
3. Retry until status is healthy and decisions recover.

Local bundle drift (offline mode)

Symptoms:

• Decisions behave unexpectedly with an outdated local artifact.

Recovery:

1. Replace local bundle with the approved current artifact.
2. Restart and re-validate expected decisions.

Authentication

Missing authorization header

Recovery:

1. Ensure callers send the Authorization header for protected endpoints.

Missing bearer token

Recovery:

1. Ensure header format is Authorization: Bearer <token>.

Decision and Policy Evaluation

JSON parse failure

Symptoms:

• JSONParseError during request handling.

Recovery:

1. Validate request JSON syntax and schema.
2. Re-test with a minimal known-good payload.

Undefined decision

Symptoms:

• UndefinedDecisionError for expected rule path.

Recovery:

1. Verify namespace and rule path.
2. Confirm active bundle contains the expected policy package/rule.

Rego builtin or evaluation failures

Symptoms:

• RegoBuiltinError or RegoEvaluationError in decision logs.

Recovery:

1. Validate policy inputs and upstream dependency values.
2. Review recent policy changes for invalid assumptions or type mismatches.
3. Re-run with captured input to isolate policy logic issues.

OPA decision result error

Symptoms:

• OPA Decision Result is an error in runtime logs.

Recovery:

1. Capture decision input, policy version, and OPA log context.
2. Correlate with bundle activation and policy evaluation logs.

Runtime and Fatal Errors

Server panic

Recovery:

1. Capture stack trace, request context, and approximate first-seen UTC time.
2. Escalate immediately with the incident package.

Mode-Specific Notes

• Main mode: API_PORT configuration and API server binding errors are main-runtime specific.
• Extension mode: startup failures often surface around extension initialization and runtime config handoff.
• Embedded mode: focus on host process lifecycle and policy engine readiness before request troubleshooting.
• Offline mode: validate OFFLINE_MODE=true and local bundle quality first, then treat auth/decision errors as normal policy-path triage.

Incident and Escalation Checklist

Collect this information before contacting support or escalating to engineering:

1. Error text and mapped category (from Unified Error Lookup).
2. UTC timestamp and impact scope.
3. Exact reproduction steps.
4. Redacted logs around the failure window.
5. Deployment mode and environment tuple values:

• OFFLINE_MODE, DISABLE_REPORTING, BUNDLE_PATH, ENVIRONMENT, API_PORT.

6. Actions already attempted and outcomes.

Escalate immediately for:

• Any server panic.
• Repeated undefined decisions after policy verification.
• Persistent bundle activation/status failures that block readiness.