From 7e92de7cb7d86521198295356014580a07be8b8b Mon Sep 17 00:00:00 2001 From: Haileyesus <118998054+blackmammoth@users.noreply.github.com> Date: Tue, 12 May 2026 20:18:09 +0300 Subject: [PATCH] feat(providers): add comprehensive guide for provider module setup and usage --- server/modules/providers/README.md | 346 +++++++++++++++++++++++++++++ 1 file changed, 346 insertions(+) create mode 100644 server/modules/providers/README.md diff --git a/server/modules/providers/README.md b/server/modules/providers/README.md new file mode 100644 index 00000000..c0d3fcb4 --- /dev/null +++ b/server/modules/providers/README.md @@ -0,0 +1,346 @@ +# Providers Module Guide + +This file documents the current provider contract in `server/modules/providers`. +Keep it current whenever provider wiring, skill discovery, or session sync +behavior changes. The goal is that a human or AI agent can add a new provider +without guessing which files need to move. + +## Current Provider Shape + +Every provider wrapper exposes five facets: + +- `auth` +- `mcp` +- `skills` +- `sessions` +- `sessionSynchronizer` + +These correspond to the shared interfaces in `server/shared/interfaces.ts`: + +- `IProviderAuth` +- `IProviderMcp` +- `IProviderSkills` +- `IProviderSessions` +- `IProviderSessionSynchronizer` + +The services that consume them are: + +- `providerAuthService` +- `providerMcpService` +- `providerSkillsService` +- `sessionsService` +- `sessionSynchronizerService` + +Current provider ids in this repo are: + +- `claude` +- `codex` +- `cursor` +- `gemini` + +Those ids are mirrored in backend unions and frontend provider constants. If +adding a new provider, update every place that hardcodes this list. + +## Current File Layout + +Each provider lives under its own folder in `server/modules/providers/list/`: + +```text +server/modules/providers/list// + .provider.ts + -auth.provider.ts + -mcp.provider.ts + -skills.provider.ts + -sessions.provider.ts + -session-synchronizer.provider.ts +``` + +The existing provider folders are `claude`, `codex`, `cursor`, and `gemini`. + +## What Each Facet Does + +| Facet | Responsibility | Base / Service | +| --- | --- | --- | +| `auth` | Report install/auth state for the provider runtime | `IProviderAuth` -> `providerAuthService` | +| `mcp` | Read, list, write, and remove provider-native MCP config | `McpProvider` -> `providerMcpService` | +| `skills` | Discover provider-native skill markdown files | `SkillsProvider` -> `providerSkillsService` | +| `sessions` | Normalize live events and fetch session history | `IProviderSessions` -> `sessionsService` | +| `sessionSynchronizer` | Scan transcript artifacts and upsert session metadata | `IProviderSessionSynchronizer` -> `sessionSynchronizerService` | + +`sessions` and `sessionSynchronizer` are separate concerns: + +- `sessions` handles runtime event normalization and history fetches. +- `sessionSynchronizer` handles file-backed session indexing into `sessionsDb`. + +## How To Add A Provider + +1. Add the provider id everywhere it is part of the contract. + +- Update `server/shared/types.ts` `LLMProvider`. +- Update `src/types/app.ts` `LLMProvider` if the frontend should know about it. +- Update `server/modules/providers/provider.routes.ts`. +- Update `server/routes/agent.js` if the provider is launchable from the agent runtime. +- Update `server/index.js` if the provider needs runtime boot or shutdown wiring. +- Update `shared/modelConstants.js` if the provider appears in UI provider pickers. +- Update `src/components/chat/hooks/useChatProviderState.ts` and + `src/components/chat/view/subcomponents/ProviderSelectionEmptyState.tsx` if + the provider should be selectable in chat. +- Update `src/components/provider-auth/view/ProviderLoginModal.tsx` if the + provider has a login/setup flow. + +2. Create the wrapper class. + +- Add `server/modules/providers/list//.provider.ts`. +- Extend `AbstractProvider`. +- Expose readonly `auth`, `mcp`, `skills`, `sessions`, and `sessionSynchronizer`. +- Call `super('')`. + +3. Implement auth. + +- Return a full `ProviderAuthStatus`. +- Treat normal `not installed` / `not authenticated` states as data, not exceptions. +- Keep provider-specific credential discovery inside the auth provider. +- If the provider has no auth step, return a stable unauthenticated or not-installed status instead of omitting the facet. + +4. Implement MCP. + +- Extend `McpProvider`. +- Pass the supported scopes and transports to `super(...)`. +- Implement the four required methods: + - `readScopedServers(...)` + - `writeScopedServers(...)` + - `buildServerConfig(...)` + - `normalizeServerConfig(...)` +- Use the shared validation and normalization behavior from `McpProvider`. +- Keep the provider-specific config format local to the provider implementation. + +Current MCP formats in this repo are: + +| Provider | User / Project Storage | Supported Scopes | Supported Transports | +| --- | --- | --- | --- | +| Claude | `.mcp.json` in user / local / project locations | `user`, `local`, `project` | `stdio`, `http`, `sse` | +| Codex | `.codex/config.toml` | `user`, `project` | `stdio`, `http` | +| Cursor | `.cursor/mcp.json` | `user`, `project` | `stdio`, `http` | +| Gemini | `.gemini/settings.json` | `user`, `project` | `stdio`, `http` | + +5. Implement skills. + +- Extend `SkillsProvider`. +- Implement `getSkillSources(workspacePath)`. +- Return the actual discovery roots for the provider. +- Skills are discovered from `SKILL.md` files. +- `readProviderSkillMarkdownDefinition(...)` reads front matter `name` and `description`. +- If `name` is missing, the parent directory name is used as a fallback. +- Use `recursive: true` only when the provider stores skills in nested trees. +- Keep the emitted `command` string aligned with the provider's real skill syntax. + +Current skill discovery roots are: + +| Provider | User Roots | Project / Repo Roots | Prefix | Notes | +| --- | --- | --- | --- | --- | +| Claude | `~/.claude/skills` | `/.claude/skills` | `/` | Also discovers Claude plugin skills from enabled plugin installs. Command skills live under `commands/`; markdown skills live under `skills/` and are scanned recursively. | +| Codex | `~/.agents/skills`, `~/.codex/skills/.system`, `/etc/codex/skills` | `/.agents/skills`, `path.dirname(workspacePath)/.agents/skills`, topmost git root `.agents/skills` | `$` | Overlapping roots are deduplicated before scanning. | +| Cursor | `~/.cursor/skills` | `/.cursor/skills`, `/.agents/skills` | `/` | Uses slash-style commands. | +| Gemini | `~/.gemini/skills`, `~/.agents/skills` | `/.gemini/skills`, `/.agents/skills` | `/` | Uses slash-style commands. | + +Command forms currently used by the providers are: + +- Claude user/project skills: `/skill-name` +- Claude plugin skills: `/plugin-name:skill-name` +- Codex skills: `$skill-name` +- Cursor skills: `/skill-name` +- Gemini skills: `/skill-name` + +6. Implement sessions. + +- Implement `normalizeMessage(raw, sessionId)` and `fetchHistory(sessionId, options)`. +- Use `createNormalizedMessage(...)` and `generateMessageId(...)` for emitted messages. +- Keep normalized message ids unique. If one raw event produces multiple text + parts, append a discriminator so ids do not collide. +- Keep pagination consistent: + - `limit: null` means unbounded/full history. + - `limit: 0` means an empty page. + - always return `total`, `hasMore`, `offset`, and `limit` when paginating. +- Sanitize any filesystem-derived ids before using them in file or database paths. +- Do not assume a provider's history format matches another provider's format. + +7. Implement session synchronization. + +- Implement `synchronize(since?: Date)` to scan provider artifacts and upsert + sessions into `sessionsDb`. +- Implement `synchronizeFile(filePath)` for single-file watcher updates. +- Use the existing helpers when they fit: + - `buildLookupMap(...)` + - `extractFirstValidJsonlData(...)` + - `findFilesRecursivelyCreatedAfter(...)` + - `normalizeSessionName(...)` + - `readFileTimestamps(...)` +- Make the sync resilient to partial, malformed, or missing provider files. +- The orchestration service runs all provider synchronizers and only advances + `scan_state.last_scanned_at` when every provider succeeds. + +Current session sync roots are: + +| Provider | Scan Roots | Metadata Helpers / Notes | +| --- | --- | --- | +| Claude | `~/.claude/projects/**/*.jsonl` | Uses `~/.claude/history.jsonl` for name lookup and the trailing `ai-title`, `last-prompt`, or `custom-title` entries for title recovery. | +| Codex | `~/.codex/sessions/**/*.jsonl` | Uses `~/.codex/session_index.jsonl` for title lookup and the last `task_complete` message for a fallback title. | +| Cursor | `~/.cursor/projects/**/*.jsonl` | Uses sibling `worker.log` to recover `workspacePath`, then derives the session title from the first user prompt. | +| Gemini | `~/.gemini/tmp/**/*.jsonl` | Current full scans only index temp JSONL chat artifacts. Single-file sync also accepts legacy `.json` files. | + +8. Register the provider. + +- Add the new provider class to `server/modules/providers/provider.registry.ts`. +- Update `server/modules/providers/provider.routes.ts` provider parsing. +- If the provider introduces a new service or lifecycle hook, export it from the module entrypoint that consumes providers. + +9. Wire runtime and UI surfaces outside the providers module when needed. + +If the provider can run live chat sessions, update the runtime entrypoints too: + +- `server/routes/agent.js` +- `server/index.js` + +If the provider is visible in the UI, update: + +- `shared/modelConstants.js` +- `src/components/chat/hooks/useChatProviderState.ts` +- `src/components/chat/view/subcomponents/ProviderSelectionEmptyState.tsx` +- `src/components/provider-auth/view/ProviderLoginModal.tsx` + +## Minimal Wrapper Template + +```ts +import { AbstractProvider } from '@/modules/providers/shared/base/abstract.provider.js'; +import { ProviderAuth } from './-auth.provider.js'; +import { McpProvider } from './-mcp.provider.js'; +import { SkillsProvider } from './-skills.provider.js'; +import { SessionsProvider } from './-sessions.provider.js'; +import { SessionSynchronizer } from './-session-synchronizer.provider.js'; +import type { + IProviderAuth, + IProviderMcp, + IProviderSessionSynchronizer, + IProviderSessions, + IProviderSkills, +} from '@/shared/interfaces.js'; + +export class Provider extends AbstractProvider { + readonly auth: IProviderAuth = new ProviderAuth(); + readonly mcp: IProviderMcp = new McpProvider(); + readonly skills: IProviderSkills = new SkillsProvider(); + readonly sessions: IProviderSessions = new SessionsProvider(); + readonly sessionSynchronizer: IProviderSessionSynchronizer = + new SessionSynchronizer(); + + constructor() { + super(''); + } +} +``` + +## Minimal Skills Template + +```ts +import path from 'node:path'; + +import { SkillsProvider } from '@/modules/providers/shared/skills/skills.provider.js'; +import type { ProviderSkillSource } from '@/shared/types.js'; + +export class SkillsProvider extends SkillsProvider { + constructor() { + super(''); + } + + protected async getSkillSources(workspacePath: string): Promise { + return [ + { + scope: 'project', + rootDir: path.join(workspacePath, '.', 'skills'), + commandPrefix: '/', + }, + ]; + } +} +``` + +## Minimal Session Sync Template + +```ts +import type { IProviderSessionSynchronizer } from '@/shared/interfaces.js'; + +export class SessionSynchronizer implements IProviderSessionSynchronizer { + async synchronize(since?: Date): Promise { + return 0; + } + + async synchronizeFile(filePath: string): Promise { + return null; + } +} +``` + +## AI Prompt Template + +Use this prompt when asking an AI agent to add a provider: + +```text +Add a new provider "" using the current provider module architecture. + +Requirements: +1) Create: + - server/modules/providers/list//.provider.ts + - server/modules/providers/list//-auth.provider.ts + - server/modules/providers/list//-mcp.provider.ts + - server/modules/providers/list//-skills.provider.ts + - server/modules/providers/list//-sessions.provider.ts + - server/modules/providers/list//-session-synchronizer.provider.ts +2) Register in: + - server/modules/providers/provider.registry.ts + - server/modules/providers/provider.routes.ts + - server/shared/types.ts LLMProvider + - src/types/app.ts LLMProvider +3) Mirror the nearest existing provider implementation for file naming, style, + and error handling. +4) Implement skills support with SkillsProvider and the current skill roots. +5) Implement session synchronization if the provider stores transcript files. +6) Ensure sessions use unique ids, safe path handling, and correct pagination. +7) Keep `sessions` and `sessionSynchronizer` separate. +8) Run: + - npx eslint + - npx tsc --noEmit -p server/tsconfig.json +``` + +## Validation + +After adding or changing a provider, run the relevant checks: + +```bash +npx eslint server/modules/providers/**/*.ts server/shared/types.ts server/shared/interfaces.ts +npx tsc --noEmit -p server/tsconfig.json +``` + +Useful tests in this repo: + +- `server/modules/providers/tests/mcp.test.ts` +- `server/modules/providers/tests/skills.test.ts` + +If you touch sessions or session synchronization, add or update focused tests +alongside the implementation. + +## Common Mistakes + +- Adding provider files but forgetting `provider.registry.ts` or + `provider.routes.ts`. +- Updating backend provider ids but not `src/types/app.ts` or the frontend + provider constants. +- Omitting `skills` or `sessionSynchronizer` from the wrapper. +- Returning duplicate normalized message ids for split content. +- Treating `limit === 0` as unbounded history. +- Building file paths from raw session ids without validation. +- Hardcoding a skill root without checking the provider's actual discovery rules. +- Forgetting that Claude plugin skills are discovered differently from normal + user/project skill folders. +- Assuming one provider's MCP config file format works for the others. + +