Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.mortemlabs.com/llms.txt

Use this file to discover all available pages before exploring further.

Most Mortem issues fall into a small set of root causes: a misconfigured API key, a missing verify token on the first run, or agent code that bypasses session.run(). Work through the relevant section below to isolate the problem. If none of these resolve your issue, rotate your API key from the agent settings page and re-run the onboarding wizard.
Traces are missing when the SDK cannot authenticate with the ingest service or cannot reach it at all.Check the following in order:
  1. API key — confirm MORTEM_API_KEY in your agent environment matches exactly what the dashboard shows under agent settings. The key is only displayed once during onboarding; if you lost it, rotate it and use the new one.
  2. Agent ID — confirm MORTEM_AGENT_ID is set and matches the ID shown in the dashboard for this agent. Without a valid agent ID, the ingest service has no agent to attach the trace to.
  3. Verify token on first run — if this is the first time you are running the agent, MORTEM_VERIFY_TOKEN must be present in the environment. The token tells the ingest service to associate your first trace with the agent and mark it as verified. See the verify token section below for details.
  4. Ingest URL — confirm MORTEM_INGEST_URL is set correctly. In local development it should be http://localhost:4001. For the hosted service the SDK defaults to https://ingest.mortem.dev — you only need to set MORTEM_INGEST_URL if you want to override that default.
  5. Flush before exit — always call await mortem.close() at the end of your agent process. Without it, buffered events may not flush before the process exits. 
try {
  await session.run(agentWork)
} finally {
  await mortem.close()
}
A 401 or 403 from the ingest service means the API key did not match any agent on record, or the key was present but the batch was rejected for another auth reason.
  • Wrong key — confirm the key was copied directly from the dashboard without any extra whitespace or line breaks. Keys are case-sensitive.
  • Key not yet saved — if you just created the agent, wait a few seconds and retry. The key is cached in Redis after the first lookup.
  • Missing verify token on first run — if this is the agent’s first batch, MORTEM_VERIFY_TOKEN must be included in the same run. Without it the ingest service cannot confirm agent ownership and rejects the batch.
  • Stale key after rotation — if you rotated the API key in the dashboard, update MORTEM_API_KEY in your agent environment and redeploy before retrying.
Analysis is computed asynchronously after the trace finishes. It does not appear while the agent is still running.Check the following:
  1. Trace status — open the trace detail page and confirm the status shows completed or errored, not running. Analysis does not start until the trace is closed. Call await session.complete() or await session.fail(error) to close the trace explicitly, or use session.run() which closes the session automatically.
  2. Wait a few minutes — analysis is asynchronous and may take 1–3 minutes after the trace completes, depending on trace size and LLM provider latency.
  3. Rerun analysis — if the trace is marked completed but analysis still does not appear after several minutes, use the Rerun analysis button on the trace detail page to re-queue it.
The live stream on the agent detail page only shows traces that are currently running. If the agent has already finished its run, the stream is empty by design.
  • Confirm the agent is actively running — start a new run and watch the stream. Events appear in real time as the SDK flushes its buffer (every 250 ms by default).
  • Check the agent ID — confirm the dashboard is showing the live stream for the correct agent. The agent ID in your environment must match the agent whose detail page you are viewing.
  • Confirm session.run() is in use — child events such as LLM calls and tool calls only appear on the live stream when they are emitted inside a session.run() context. See the missing events section below.
The verify token is a one-time credential that proves you control an agent on the first run. It is only valid once and expires after successful verification.
  • Send it on the first run — include MORTEM_VERIFY_TOKEN in your agent environment the first time you run the agent after creating it in the dashboard. The onboarding wizard polls until it sees the token.
  • Remove it after verification — once the wizard shows the agent as verified, delete MORTEM_VERIFY_TOKEN from your .env file and any deployment secrets. Leaving it in place does not cause errors, but it is unnecessary and exposes a credential that is no longer usable.
  • Re-verify an unverified agent — if the dashboard still shows the agent as unverified after the first run, add MORTEM_VERIFY_TOKEN back temporarily and run the agent once more. The token can be resent if verification did not complete the first time.
Do not share your verify token. It is single-use but treat it with the same care as your API key until verification is complete.
If your trace appears in the dashboard but child events — LLM calls, tool calls, Solana transactions — are missing, the most likely cause is that those calls were made outside of a session.run() context.The SDK uses Node.js async local storage to track which session is active. Calls made outside session.run() do not have access to that context, so auto-captured events from wrappers are silently dropped.Always wrap your agent work in session.run():
const session = await mortem.startSession({
  inputSummary: "Evaluate and execute swap",
})

try {
  const result = await session.run(async () => {
    // LLM calls, tool calls, and Solana transactions
    // made here are captured automatically
    return await runAgentLogic()
  })
} catch (error) {
  // session.run() already calls session.fail() on throw,
  // so re-throw to let the outer try/catch handle cleanup
  throw error
} finally {
  await mortem.close()
}
If you need to create events manually outside of session.run(), call session.beginEvent() directly — those events are always recorded regardless of async context.
To rotate a compromised or lost API key, open the agent’s settings page in the dashboard and click Rotate API key. The new key is shown once — copy it immediately and update your agent environment before restarting.