Operations and observability

Administrators manage accounts in Users and investigate application behavior in Operations. Select Administration in the sidebar. The examples below use synthetic patients and accounts.

Create a user

1. Open Users → New user.
2. Enter the person's full name and email, then choose Clinician or Administrator. Administrators can also manage accounts and inspect operations.
3. Select Create user. OncoBrain opens the account page, where you can edit its details and access settings.

The Users directory supports search, role and status filters, and enabling or disabling accounts. Open an account to change its role or revoke its sessions. You cannot disable your own account or change your own role.

Find the relevant activity

Activity reads the organization database. Overview, Traces, and Sessions read OpenObserve, the separate service that collects technical telemetry. All four views use your OncoBrain administrator session.

Watch a session replay

Open Operations → Sessions, then select Replay on a recorded session. The monitor opens below the list. Use its play/pause button and timeline to inspect the interaction; the speed controls help move through a longer recording.

Search or filter the session list to find a recording. Browser, device, screen size, and capture time help distinguish sessions; unavailable metadata stays unknown. Refresh telemetry loads the latest available recordings within the time window shown above the tabs. Close the monitor when you have finished.

Replay helps reproduce a navigation or interface problem. It does not change the chart or establish clinical facts. Production recordings are masked and sampled; the readable example above comes from the synthetic demo. OpenObserve's separate explorer provides broader log search, alerting, and ingestion administration.

System status

Administration → System is the current operational snapshot, alongside Operations and Users. It answers whether the app can reach its dependencies and shows the configuration those runtimes are using. Operations remains the place to inspect past activity, traces, and replay.

The screen groups services and AI/search above a searchable LanceDB inventory. The environment label comes from the running deployment. Each service's detail popover shows its configured target; loopback addresses refer to that server's host, including Eve running beside the app on Azure. Hosted site checks require explicit public URLs and never fall back to development ports. Service checks and inventory requests load independently. An unavailable table or runtime does not hide successful checks. Status always includes text, and old results keep their original check time and become visibly stale. On narrow screens, table counts and commit times move beneath the table name.

Loading or refreshing the page runs the existing readiness checks and bounded requests to the configured docs and website URLs. Readiness includes the existing temporary object-store write, readback, and delete probe. Public-site results mean reachable from the application server; they do not attest browser behavior or global uptime. Refreshes briefly share cached results. There is no polling or scheduled monitoring service behind the page.

Run checks is an administrator-only POST. It sends one fixed, small inference request from the application and another from Eve, plus one bounded internal search query. The checks use the configured provider adapters and synthetic input, without clinical context, model-visible tools, sessions, or a patient workspace. They test each process's default model; the displayed Quick, background, and workflow selections are configuration, not additional successful inference claims. Provider usage may apply. Concurrent requests share checks, and completed results are briefly cached in process memory. The protected diagnostics extend Bun's per-request idle timeout beyond their bounded probe deadlines so a slow check can return its status.

Eve owns its model-selection projection through the private /internal/system/status and /internal/system/inference channel routes. Both require the existing service bearer and the active nonempty runtime epoch; the development discovery exception does not apply. The app parses the response against the shared protocol schema before returning it to the administrator. Eve's actual disabled web-search tool is reported as Disabled, with no synthetic web-search success and no additional search integration.

Inventory reads use the application's existing LanceDB connection and configured dataset. The inventory displays the exact database URI, storage type, and Azure account where applicable; an S3-compatible endpoint is available in the metadata popover. These values come from the same configuration passed to LanceDB, even when the connection cannot be read. The table list shows the active dataset's actual table names. Eve searches this same corpus through the application. Inventory reads return table names, mapped source families, schema compatibility, row counts, native versions, and commit times. Counts and dates refer to the same inspected table version. Last commit is a storage timestamp, not source freshness. Listing is capped at 50 tables, with three concurrent metadata reads; timeouts and individual failures remain visible. Native reads keep their occupied slots until they settle, preventing repeated refreshes from multiplying stalled work. Count and schema reads are queued before optional commit-history reads; slow history preserves the counts already read. Duplicate table names are collapsed, and uninspected schema compatibility remains unknown. No source rows or content are returned.

Snapshots are disposable and local to a server process. They create no domain entities, monitoring database, or clinical authority. Only administrators can read connection locations. Responses omit credentials, signed URL parameters, raw storage options, provider error bodies, patient data, and source text.

What stays out of telemetry

The app and Eve send a small allowlist of technical fields to OpenObserve: service, release, environment, operation, duration, status, model, token counts, and sanitized route templates. Before export, the service removes exception text, dynamic route values, query strings, user identity, prompts, model output, source text, and patient data.

Browser replay follows the same route filtering. It can retain coarse browser, device, city, and viewport metadata for diagnosis. Treat those fields as operational personal data and apply the deployment's access and retention policy.

Set ONCOBRAIN_OPENOBSERVE_RUM_REPLAY_SAMPLE_RATE=0 to disable production replay. Telemetry remains separate from the organization database, source files, and immutable clinical artifacts.

Run it locally

The normal launcher starts OpenObserve with the synthetic local stack:

bun run dev

It downloads the pinned package once, stores its generated local credential and disposable data outside Git, and starts the collector on loopback. If OpenObserve cannot start, the web app, the advisory-draft service, and Activity still run. Use bun run dev --no-observe to leave it out.

Use these commands when you need to manage the collector outside the main launcher:

bun run observe up
bun run observe logs
bun run observe down

observe down keeps the local telemetry directory. To send app and Eve traces, configure an OTLP endpoint and an ingestion-only header in server configuration. Keep query, ingestion, and root credentials off the browser; never expose them through a VITE_ variable.

Hosted operations

The application can send OTLP and browser telemetry to an external OpenObserve service. Host that service separately from the application state, mount its own encrypted persistent volume, terminate TLS, allow only exact application origins for browser telemetry, and use distinct read and ingestion credentials.

The checked-in OpenObserve retention is seven days. Set the replay stream to three days after it exists, monitor disk space, and snapshot a stopped telemetry volume before upgrading. A telemetry restore can aid investigation, but it is not part of patient-state recovery.

For hosted deployment and backup requirements, see deployment.

Troubleshooting

Verify a change

bun test apps/oncobrain/tests/operations/openobserve-config.test.ts \
  apps/oncobrain/tests/operations/openobserve-payload.test.ts \
  apps/oncobrain/tests/operations/openobserve-deployment.test.ts
bun test apps/eve/tests/agent/openobserve-instrumentation.test.ts
bun run check

Code anchors

• Policy and defaults: apps/oncobrain/src/config/observability.ts
• Administrator screen: apps/oncobrain/src/routes/admin/observability.tsx
• User creation: apps/oncobrain/src/routes/admin/users/new.tsx
• User management: apps/oncobrain/src/lib/server/users/user-management.server.ts
• Replay monitor: apps/oncobrain/src/features/admin/components/openobserve-replay-player.tsx
• Server query boundary: apps/oncobrain/src/lib/server/observability/openobserve-admin.server.ts
• Browser telemetry: apps/oncobrain/src/app/browser-telemetry.tsx
• App and Eve exporters: apps/oncobrain/src/lib/server/observability/openobserve-telemetry.server.ts and apps/eve/agent/lib/openobserve-exporter.ts
• Single-node service: deploy/openobserve/compose.yml