Skip to main content

AI Capabilities and Guardrails

This page describes what the Cashlytics AI assistant can do, what it is explicitly not allowed to do, and where human approval is required.

All statements are derived from runtime behavior in:

  • src/app/api/chat/route.ts
  • src/lib/ai/tools.ts
  • src/lib/auth/require-auth.ts
  • src/proxy.ts
  • src/components/molecules/chat-message.tsx
  • src/components/molecules/tool-confirmation-card.tsx

Paths are relative to the cashlytics-ai repository.

Provider and models​

  • Chat: OpenAI gpt-4o through the Vercel AI SDK (streamText, @ai-sdk/openai).
  • Receipt scan (POST /api/receipt-scan): OpenAI gpt-4o-mini with generateObject and a Zod schema.
  • CSV import reconciliation (src/lib/ai/csv-reconciliation.ts): batched AI matching; the free-text JSON answer is validated with Zod.

All AI features require OPENAI_API_KEY. Without it, chat answers 503, receipt scanning answers 400, and the CSV import feature is disabled.

Scope: what the AI is designed for​

The assistant is finance-scoped. It is intended to help with:

  • expenses (one-time and recurring)
  • incomes
  • accounts and balances
  • transfers
  • category management
  • analytics and forecasting
  • subscriptions, insights, and affordability checks

The system prompt in route.ts enforces this scope and instructs the model to refuse non-finance topics.

What the AI can do​

Read operations​

The assistant can read user-owned financial data via tools (src/lib/ai/tools.ts):

  • accounts (getAccounts)
  • categories (getCategories)
  • recurring expenses (getExpenses)
  • one-time expenses (getDailyExpenses)
  • incomes (getIncomes)
  • transfers (getTransfers)
  • monthly overview (getMonthlyOverview)
  • forecast (getForecast)
  • category breakdown (getCategoryBreakdown)
  • normalized monthly expenses (getNormalizedMonthlyExpenses)
  • subscriptions (getSubscriptions)
  • month comparison (compareMonths)
  • affordability check (checkAffordability)
  • active insights (getInsights)
  • spending summary (getSpendingSummary)

Write operations​

The assistant can prepare write actions for:

  • create/update/delete account
  • create/update/delete recurring expense
  • create/update/delete one-time expense
  • create/update/delete income
  • create category
  • create transfer
  • mark an insight as read (markInsightRead)

There are no tools to update or delete categories.

Every write action is gated by explicit user approval before execution.

What the AI is not allowed to do​

Out-of-domain requests​

The prompt instructs the model to handle only finance requests and refuse unrelated requests (for example weather, coding, music, sports).

Document and file content access​

The assistant must not read uploaded user documents (receipts, invoices, statements). This is explicitly stated in the chat system prompt.

In addition, tool outputs use explicit response allowlists for expense retrieval and intentionally exclude document payloads from AI context.

Unauthenticated access​

The assistant does not bypass authentication:

  • route protection in proxy.ts requires login for protected routes, including chat
  • the chat route also checks the session itself and answers 401 without one
  • server actions called by tools enforce requireAuth()
  • unauthenticated access resolves to Unauthorized

Human-in-the-loop approvals​

The tool catalog marks write and destructive actions with needsApproval: true.

At runtime, this means:

  • the assistant proposes a tool action
  • the UI shows a confirmation card
  • the user must accept or deny
  • only accepted actions are executed

Approval UI behavior is implemented in:

  • chat-message.tsx (renders pending approval parts)
  • tool-confirmation-card.tsx (accept/deny controls with operation summary)

Runtime safety limits​

The chat endpoint enforces several limits:

  • 401 without a session; 503 ("KI-Assistent ist nicht konfiguriert.") when OPENAI_API_KEY is empty
  • rate limiting: 20 requests per minute per signed-in user (in-memory, per server process)
  • only the last 100 messages are sent to the model; older ones are dropped, not rejected
  • a request whose messages exceed 400,000 characters is rejected with 413
  • the client may only send user and assistant messages; a system role returns 400
  • tool/reasoning step cap: stepCountIs(10); incomplete tool calls are ignored
  • invalid JSON or invalid message shape returns 400
  • user data placed into the system prompt passes through sanitizeForPrompt

These controls reduce abuse and runaway tool execution.

Behavioral notes for developers​

  • The model is instructed to respond in German in the system prompt.
  • For expense capture, the prompt pushes category-first behavior and asks for category confirmation when needed.
  • The prompt also injects the current date (in the user's time zone, sent by the browser) and context to reduce unnecessary discovery calls: accounts, categories, recurring expenses, incomes, dashboard budget status, the last 10 one-time expenses of the month, active insights, and learned merchant-to-category hints.

Practical expectation for users​

  • The assistant can automate common finance workflows quickly.
  • The assistant cannot act outside finance scope.
  • The assistant cannot read uploaded document content.
  • Sensitive modifications are never silent; write tools always require explicit approval.

Known limitations​

  • Data from your account (accounts, categories, expenses, incomes, and so on) is sent to OpenAI when you use the assistant, receipt scan, or AI-assisted CSV import. Do not enable AI features if that is not acceptable for your data.
  • Prompt injection: sanitizeForPrompt keeps newlines, and text from CSV imports and receipt OCR can end up in the system prompt. Approval for write tools limits the impact, but this content is an injection surface. This is tracked in docs/audit/open-findings.md in the repository.
  • checkAffordability and getSpendingSummary pick the month by server time, not the user's time zone.