AI Assistant API

The AI assistant provides a conversational interface with full access to Opbox data via 129 built-in tools. It supports multiple AI model providers, real-time streaming responses, plan mode, saved prompts, @-mention entity references, and live UI refresh when the AI performs write operations.

Overview

• Accessible via Cmd+K (or Ctrl+K) -> AI tab
• Real-time streaming responses, displayed word by word
• 129 tools for search, knowledge retrieval, data operations, user interaction, dashboards, files, and more
• Support for multiple AI model providers
• Conversation persistence with thread management
• Page context awareness - AI knows which page you're viewing and can access relevant data automatically
• Plan Mode: restricts AI to read-only tools; no writes possible even if requested
• Saved Prompts: per-user library of reusable prompts (max 50) with usage tracking
• @ Mentions: type @ in the chat input to search and attach entities (matters, forms, tables, documents, workflows, submissions) as context (max 10 references)
• Live UI refresh: when the AI performs write operations (e.g. adding subtasks, completing steps), the host page automatically refreshes its data - no manual reload needed
• Write limits per session and per day to prevent misuse
• Security scanning on all tool results to detect and block injection attempts

Endpoints

Chat History

Skills

Chat Streaming

Sends a conversation to the AI and streams the response as Server-Sent Events (SSE). The AI can call tools to query and modify Opbox data, then continue generating text. The AI may call multiple tools per request to fulfil your query.

POST /api/ai/chat
Content-Type: application/json

{
  "messages": [
    { "role": "user", "content": "Summarise this matter and suggest next steps" }
  ],
  "provider": "anthropic",
  "pageContext": "/matters",
  "references": [
    { "id": "cm1abc...", "type": "matters", "title": "ACME Ltd Incorporation" }
  ]
}

Body Parameters

Stream Events

The response is an SSE stream (text/event-stream). Each event is a JSON line:

Example Stream

data: {"type":"text_delta","content":"Here"}
data: {"type":"text_delta","content":" are"}
data: {"type":"text_delta","content":" your"}
data: {"type":"tool_start","toolName":"list_matters","toolId":"toolu_01...","input":{"status":"OPEN"}}
data: {"type":"tool_result","toolName":"list_matters","toolId":"toolu_01...","result":{...},"isError":false}
data: {"type":"text_delta","content":"You have 5 open matters..."}
data: {"type":"done"}

Available Tools (129)

The AI assistant has access to 129 tools across all major Opbox modules. Tools are automatically called by the AI based on user requests. Write operations use a confirm-then-act pattern - the AI calls ask_confirmation before mutating data, rendering an Approve/Deny dialog in the UI. Archive operations are intentionally excluded from the AI toolset - they are available in the UI only.

User Interaction

Search

Knowledge Retrieval (RAG)

Matters

Boards

Forms & Submissions

Tables & Data

Documents & Workflows

Notifications & Organisation

Transcripts (Private Per-User)

API Access

Dashboard Tools

SQL Query Library

File Registry

Identity & Access

Ownership & Corporate Structure

Registers & Compliance

Entity File Intelligence

Self-Knowledge

Self-knowledge tools enable the AI to answer questions about Opbox's own architecture, features, and data model. REST equivalents are available at GET /api/docs/search and POST /api/docs/embed. See the Intelligence Architecture page for details.

For a complete per-tool reference with parameter types and descriptions, see the AI Tools Reference page.

Parallel Tool Execution

The AI can call multiple tools in a single response. Independent tool calls execute concurrently on the server for maximum speed. The chat UI groups multiple tool calls into a single collapsible disclosure:

• Single tool call: Shown inline with status indicator and label (expandable for input/result)
• Multiple tool calls: Collapsed into "N tool calls" - click to expand and see each call individually, click a call to see its input/result
• During streaming: Tool group auto-expands to show progress; auto-collapses when all tools complete successfully
• Error handling: Failed tool count shown in the collapsed summary; group stays expanded if any tool errored

Chat Thread Management

Conversations are persisted server-side as threads. Each thread stores its messages with tool call history.

Confirm-Then-Act Pattern

Before performing any write operation, the AI calls ask_confirmation which renders an Approve / Deny dialog in the chat UI. The agentic loop pauses until the user responds. If approved, the AI proceeds with the action in the next turn. If denied, it acknowledges and stops. This replaces the previous apply: prefix requirement.

Similarly, ask_question renders an inline input when the AI needs clarification. The user can type a response or click a suggested option.

All write operations are scoped to the authenticated user's active workspace and logged in the audit trail.

Archive operations (archive_matter, archive_table, etc.) are not available to the AI - they are human-only actions accessible only through the Opbox UI.

Security Model

Plan Mode

Plan Mode restricts the AI to read-only tools. When planMode: true is sent, write tools are not available to the AI and cannot be called. Toggle available in the chat UI as a "Plan" pill below the input box.

Page Context Preloading

When a user sends their first message from a matter or document page, the current resource's data is automatically preloaded for the AI. This eliminates the need for extra lookups before performing operations, significantly reducing latency.

Matter pages preload: matter title, status, priority, assignee, due date, labels, board name, board properties (with current values), step pipeline (with stepId for each step), form titles for FORM steps, and APPROVAL step -> form linkage. The AI can use preloaded step IDs directly for add_subtask, complete_step, assign_step, etc. without calling get_matter first.

Document pages preload: document title, content (first 3000 chars), and custom metadata properties.

Form Awareness in Matters

The AI understands which specific forms are associated with matter steps:

FORM steps: the AI sees the assigned form title (e.g. "KYC Questionnaire") for both pending and completed FORM steps. Multi-form steps with formIds show all available form titles. Completed FORM steps also include collected field data.

APPROVAL steps: the AI sees which FORM step an approval reviews (e.g. "reviews: Collect KYC - form: KYC Questionnaire"), derived from the reviewFormStepIndex in the approval step config.

Live UI Refresh

When the AI performs write operations via the sidebar chat, the host page automatically refreshes its data without requiring a manual page reload. For example, completing a step or adding a subtask via AI chat will immediately update the matter detail view.

Saved Prompts

Per-user reusable prompt library backed by Knowledge Base documents. Prompts are stored as full KB documents inside a "Prompts" system folder (auto-created on first KB visit alongside the Transcripts folder). Users can save frequently-used inputs from the chat UI, browse and edit them as rich documents in the Knowledge Base, and organise them into subfolders. Max 50 prompts per user (counted across the entire Prompts folder tree).

Admin: Security Events (OWNER only)

Query the workspace's security event log. Events are recorded automatically by the security monitoring system. Requires OWNER role.

GET /api/admin/security-events?type=AI_INJECTION_ATTEMPT&limit=50&cursor=cm...

{
  "events": [
    {
      "id": "cm...",
      "type": "AI_INJECTION_ATTEMPT",
      "detail": "Injection pattern detected in tool result",
      "metadata": { "toolName": "get_document", "pattern": "\\[INST\\]" },
      "ipAddress": "1.2.3.4",
      "createdAt": "2026-02-17T10:30:00.000Z",
      "user": { "name": "Jane Doe", "email": "jane@example.com" }
    }
  ],
  "total": 42,
  "nextCursor": "cm..."
}

The admin view at Settings -> Security -> Events surfaces this data with filter tabs and an injection alert banner.

Admin: RAG Embeddings (ADMIN / OWNER)

Monitor and manage the RAG (Retrieval-Augmented Generation) embedding pipeline. Content from documents, comments, notes, transcripts, and compliance exceptions is automatically chunked and embedded for semantic search.

Get Status

GET /api/admin/rag?sourceType=DOCUMENT

{
  "ragEnabled": true,
  "embeddingModel": "text-embedding-3-small",
  "statusBySourceType": {
    "DOCUMENT": { "COMPLETED": 42, "PENDING": 2 },
    "MATTER_COMMENT": { "COMPLETED": 15 }
  },
  "chunksBySourceType": {
    "DOCUMENT": { "chunks": 210, "tokens": 48500 }
  },
  "recentFailures": []
}

Trigger Backfill (OWNER only)

POST /api/admin/rag
Content-Type: application/json

{ "sourceType": "DOCUMENT" }  // Optional - omit to backfill all types

Backfill runs asynchronously. Supported source types: DOCUMENT, MATTER_COMMENT, TRANSCRIPT, SUBMISSION_NOTE, REGULATOR_COMMENT, TABLE_ROW_NOTE, COMPLIANCE_EXCEPTION.

AI Execute (Automations)

One-shot AI prompt execution used by automation AI steps and the workflow builder test button. Supports both Anthropic and OpenAI providers. API keys are configured in user preferences (Settings -> AI).

POST /api/integrations/ai/execute
Content-Type: application/json

{
  "provider": "anthropic",
  "model": "claude-sonnet-4-5-20250929",
  "prompt": "Summarize this form submission: ...",
  "systemPrompt": "You are a helpful assistant.",
  "temperature": 0.7,
  "maxTokens": 1024
}

Response

{
  "success": true,
  "requestId": "ai-1708000000-abc1234",
  "provider": "anthropic",
  "model": "claude-sonnet-4-5-20250929",
  "response": {
    "content": "The form submission contains...",
    "finishReason": "end_turn"
  },
  "usage": {
    "promptTokens": 150,
    "completionTokens": 200,
    "totalTokens": 350
  }
}

Provider Configuration

AI providers are configured in Settings -> AI (requires ADMIN or OWNER role). The system supports two providers:

Keys configured in Settings -> AI take priority over environment variables. For automation AI steps, the system loads the org admin/owner's configured keys.

Passport OCR

AI-powered passport data extraction using vision models. Extracts structured data from passport images including name, nationality, date of birth, document number, expiry date, and MRZ data.

POST /api/ai/passport-ocr
Content-Type: application/json

{
  "imageUrl": "https://example.com/passport-scan.jpg"
}

{
  "success": true,
  "data": {
    "fullName": "JOHN DOE",
    "nationality": "United Kingdom",
    "dateOfBirth": "1985-03-15",
    "documentNumber": "AB1234567",
    "expiryDate": "2030-01-20",
    "sex": "M",
    "mrzLine1": "P<GBRDOE<<JOHN<<<<<<<<<<<<<<<<<<<<<<<<",
    "mrzLine2": "AB12345670GBR8503151M3001207<<<<<<<<<<<04"
  }
}

Voice Transcription

Transcribe audio files (voice memos, recordings) using Whisper (cloud) or WhisperX (local with speaker diarization). The provider is configured in Settings -> AI -> Transcription.

Privacy: Transcripts are private to the uploading user. The list and detail endpoints filter by the authenticated user's ID - users can only see and manage their own transcripts. The AI assistant accesses transcripts via dedicated list_my_transcripts / get_my_transcript tools which enforce the same per-user scoping.

Transcription Providers

Provider selection: auto uses WhisperX if available (enabled + installed), otherwise Whisper. Force a specific provider via transcriptionProvider in user AI preferences.

Upload & Transcribe

POST /api/ai/transcribe
Content-Type: multipart/form-data

file: <audio file (max 1GB)>
title: "Lee & Will"  // optional - user-chosen display name

Supported formats: .m4a, .mp3, .wav, .webm, .ogg, .flac, .mp4, .qta (Apple Voice Memos - auto-converted to m4a). The UI shows a naming dialog on file selection - the title defaults to the filename (minus extension) and is used as the Knowledge Base document name with a date prefix (e.g. "18 February 2026 - Lee & Will").

// Response (WhisperX - with speaker diarization)
{
  "id": "cm...",
  "filename": "meeting.m4a",
  "title": "Lee & Will",
  "text": "Hello everyone, let's get started...",
  "language": "en",
  "durationSecs": 120.5,
  "provider": "whisperx",
  "speakerCount": 3,
  "segments": [
    { "start": 0.5, "end": 3.2, "text": "Hello everyone, let's get started.", "speaker": "SPEAKER_00" },
    { "start": 3.8, "end": 7.1, "text": "Thanks for joining.", "speaker": "SPEAKER_01" }
  ],
  "createdAt": "2026-02-18T10:30:00.000Z"
}

// Response (Whisper - plain text, no segments)
{
  "id": "cm...",
  "filename": "recording.m4a",
  "title": null,
  "text": "Hello, this is a transcription of...",
  "language": "english",
  "durationSecs": 42.5,
  "provider": "openai",
  "speakerCount": null,
  "segments": null,
  "createdAt": "2026-02-18T10:30:00.000Z"
}

List Transcripts

GET /api/ai/transcripts?search=meeting&page=1&limit=20

{
  "transcripts": [
    {
      "id": "cm...",
      "filename": "meeting-notes.m4a",
      "title": "Lee & Will",
      "fileSize": 1048576,
      "mimeType": "audio/mp4",
      "durationSecs": 120.3,
      "text": "...",
      "language": "en",
      "status": "completed",
      "provider": "whisperx",
      "speakerCount": 2,
      "segments": [...],
      "createdAt": "2026-02-17T10:30:00.000Z",
      "user": { "name": "John Doe", "email": "john@example.com" }
    }
  ],
  "total": 1,
  "page": 1,
  "totalPages": 1
}

Response Fields

Get / Delete Transcript

GET /api/ai/transcripts/:id     // Returns full transcript record with segments
DELETE /api/ai/transcripts/:id  // Deletes the transcript

Custom System Prompt

Admins can configure a custom system prompt in Settings -> AI that is prepended to the AI assistant's built-in instructions. This allows customising the assistant's behaviour, tone, and domain knowledge for your workspace. Maximum 5,000 characters.

The custom prompt is applied to the Spotlight AI chat only (not to automation AI steps, which use their own per-step prompt configuration in the workflow builder).

Tool Result Entity Cards

When the AI assistant calls read-only tools that return structured entity data, the results are rendered as clickable inline entity cards below the tool call indicator. Each card shows the entity icon, title, and metadata chips (status, priority, assignee, board name, due date, etc.).

Mapped Tools

The following 12 read-only tools produce entity cards. Write/action tools (e.g. create_matter, approve_submission) display only the tool badge.

Card Behaviour

Cards are clickable - clicking navigates to the entity detail page and closes the Spotlight dialog. List results show a maximum of 5 cards with a "Show N more" toggle to expand. Cards render in both the inline Spotlight AI chat (compact mode) and the standalone AI chat panel.

Chat History

Conversations are persisted to the database with per-user isolation. Each chat session stores its messages, provider, and page context. Sessions are auto-created when sending the first message. The last 25 messages are sent per request (compaction).

// List chat sessions
GET /api/ai/chats

// Response
{
  "chats": [
    {
      "id": "cm...",
      "title": "Summarise ACME matter",
      "createdAt": "2026-02-19T10:00:00.000Z",
      "updatedAt": "2026-02-19T10:05:00.000Z",
      "messageCount": 8
    }
  ]
}

// Get messages for a session
GET /api/ai/chats/:id/messages

// Delete a session
DELETE /api/ai/chats/:id

Skills

Per-user library of reusable skills - instructions, playbooks, and saved prompts (max 50). Skills are stored as Knowledge Base documents in a private "Skills" system folder. They are Document records with plain text content and are also discoverable by the AI via the list_skills and get_skill tools.

// List skills
GET /api/ai/skills

// Response
[
  {
    "id": "cm...",
    "title": "Summarise matter progress",
    "content": "Give me a summary of the current matter...",
    "createdAt": "2026-02-18T09:00:00.000Z"
  }
]

// Create a skill
POST /api/ai/skills
{ "title": "Weekly review", "content": "Summarise all active matters..." }

// Delete a skill
DELETE /api/ai/skills/:id

// Record usage
POST /api/ai/skills/:id
{ "action": "use" }

AI Agent (Autonomous Worker)

Server-side autonomous task processor that executes assigned tasks with configurable autonomy levels and tool access controls.

AI Cost Control

Every LLM call records cost to the AiCostLedger with atomic budget snapshot upserts. Per-workspace monthly budgets with configurable hard cap and soft limit.

Transcript Titling

When uploading audio for transcription, an optional title field can be provided via the upload form. The title is stored on the Transcript model and used when syncing to the Knowledge Base (format: "DD Month YYYY - Title"). Transcripts are auto-labeled with the transcript label in the KB. Searchable by title via GET /api/ai/transcripts?search=....

POST /api/ai/transcribe
Content-Type: multipart/form-data

file: <audio file>
title: "Client Meeting - ACME Ltd"     // optional
provider: "openai"                      // or "whisperx"

// Response includes transcript ID and KB document ID
{
  "id": "cm...",
  "text": "...",
  "title": "Client Meeting - ACME Ltd",
  "documentId": "cm..."               // KB page created
}