2 Commits

Author	SHA1	Message	Date
Haile	374e9de719	feat: add opencode support (#762) ... * feat: add opencode support * fix: stabilize opencode session startup * fix: /models * fix: improveUI for commands * fix: format commands.js * feat: load models through provider adapters Provider model selection had outgrown a single hardcoded service. The old service mixed shared caching with provider catalogs and CLI lookup details. That made stale model lists more likely as providers changed on separate schedules. Move model discovery behind each provider so lookup lives next to the integration. The shared service now focuses on provider resolution, caching, persistence, and dedupe. Return cache metadata and add bypassCache because model availability changes outside the app. The UI and /models command can show freshness and let users force a provider refresh. Surface model descriptions while keeping fallback catalogs for unavailable CLIs or SDKs. * feat(models): resolve active session models through provider adapters The model inventory command was showing a mix of catalog defaults and composer-local state instead of the model that is actually active for a real provider session. That made /models, /cost, and /status misleading once a session had already started, especially for providers whose effective runtime model can differ from the optimistic model value held in the UI. Introduce an explicit getCurrentActiveModel() contract on IProviderModels so model resolution lives next to each provider's catalog logic and uses the provider-native source of truth: - Claude reads the init event from a resumed stream-json run - Codex reads model from ~/.codex/config.toml - Cursor reads lastUsedModel from the chat store.db - OpenCode reads the persisted session model from opencode.db - Gemini intentionally returns its default because the CLI does not provide a reliable active-session lookup Keep the returned shape intentionally minimal ({ model }). The goal is to expose only what downstream command consumers need and avoid leaking provider-specific metadata into a shared transport shape that would create extra UI coupling and future cleanup cost. Also make command behavior session-aware: when there is no concrete session id, do not spawn provider processes or inspect provider session storage just to answer /models, /cost, or /status. In a new-session view the correct answer is simply the provider default, and doing more work there adds latency and unnecessary side effects for no user value. As part of this, centralize two supporting concerns: - add a shared helper for building the default current-model result from a provider catalog so fallbacks stay aligned with DEFAULT - move leaf-directory validation into shared utils so Cursor session readers and model lookup code enforce the same path-safety rule Tests were expanded to cover both the new service delegation path and the sessionless command behavior, while keeping cache-sensitive tests isolated from persisted host cache state. Why this change: - command output should reflect the model actually driving a session - new-session views should stay fast and side-effect free - provider-specific active-model lookup should not be scattered across routes or UI code - fallback behavior should be explicit, consistent, and limited to the provider default when no true active model can be resolved * feat: support session-scoped model overrides Model selection was acting like a provider-level preference. That made resumed sessions drift back to a default or request-time model. Users expect /models changes made inside a conversation to affect that session. Store explicit session choices in app-owned ~/.cloudcli state. This avoids editing provider transcripts or native provider config. Resolve the effective model before launching each provider runtime. Claude, Cursor, Codex, Gemini, and OpenCode now honor stored resume choices. Expose a backend active-model change endpoint for existing sessions. The models modal can now distinguish default changes from session overrides. It also shows when a selected model will apply on the next response. For Claude, stop probing active model state by resuming with a dummy prompt. Read the indexed JSONL transcript from the end instead. This preserves provider history while honoring /model stdout or model fields. Add service tests for adapter delegation and resume-model precedence. The tests keep cache state, override state, and requested fallback separate. * feat: make command modal more compact * fix: preserve opencode session creation events OpenCode emits the real session id asynchronously on its first JSON output. The runner registered that id from a helper that could not see the spawned process because the process reference was scoped inside the model-resolution callback. That ReferenceError was swallowed by the generic JSON parse fallback, so the client never received session_created. Without that event, a new OpenCode chat stayed on / and the assistant stream was not attached to the new session view. Keep the process reference in the outer spawn scope so registration can update the active-process map and websocket writer as soon as OpenCode announces the session id. Split JSON parsing from event processing so malformed non-JSON output can still stream as raw text, while registration or adapter failures are surfaced as real errors instead of being hidden as assistant content. Add a fake opencode executable regression test to lock in the expected lifecycle ordering: session_created must be sent before live assistant messages, and the same session id must carry through stream_end and complete. * fix: clarify model refresh and onboarding providers OpenCode is now a supported chat provider, but first-run onboarding still only offered Claude, Cursor, Codex, and Gemini. That made OpenCode harder to discover and forced users to finish setup before finding the provider in settings or chat. Adding it to onboarding keeps first-run setup aligned with the providers the application already supports elsewhere. The model refresh control was also doing too much visual work. In the new chat model picker, the previous Hard Refresh label looked like the dialog heading, which made the primary task unclear. Users open that dialog to choose a model; refreshing catalogs is only a secondary maintenance action for stale cached provider model lists. Rename and reposition the refresh affordance so the model picker reads as a model picker first. The copy now explains why catalogs are cached, when a refresh is useful, and that the refresh checks every provider. The /models modal gets the same clarification so both model-selection surfaces describe the cache behavior consistently. * fix: format opencode model catalog labels OpenCode returns provider-prefixed ids directly from the CLI. Passing those ids through as labels made the model picker hard to scan: users saw values like anthropic/claude-3-5-sonnet-20241022 or lowercased, hyphen-split text instead of readable model names. Keep the exact OpenCode id as the option value because that is what the CLI expects, but derive a presentation label for the frontend. The formatter is intentionally generic rather than a catalog of known providers. It handles common identifier structure such as provider/model, hyphen-delimited words, v-prefixed versions, adjacent numeric version tokens, and 8-digit date suffixes. This keeps OpenCode usable as its model list expands across many upstream providers without requiring code changes for every new provider or model family. The description keeps the raw provider-prefixed id visible so users can still confirm the precise model being selected. * feat: add more fallback models for cursor * docs: move model catalog out of shared The model catalog is no longer a frontend/backend runtime contract. Keeping it under shared made ownership misleading. It implied the catalog was application code shared by runtime consumers, even though it now only supports README links and public API documentation. Move the catalog into public so it lives beside the docs surfaces that need it. This gives the API docs a stable, served module and gives README readers a linkable source without suggesting frontend or backend runtime dependency. Render the API docs model list from the exported provider registry instead of a hardcoded Claude/Cursor/Codex subset. That keeps Gemini and OpenCode visible and makes future provider documentation changes flow through one docs-specific file. Update README links, provider maintenance notes, and package files so published artifacts include the standalone docs page and model catalog without relying on the old shared path. * fix: simplify empty-state model selector Keep the provider empty state focused on the setup action users need there: choosing a model. The refresh control, cache timestamp, and refresh explanation made the dialog feel like a cache-management surface. That extra action is out of place in the empty state, where the goal is to start a chat with the selected provider and model. Remove the refresh-specific UI from ProviderSelectionEmptyState and drop the now-unused refresh/cache props from the ChatMessagesPane pass-through. Refresh behavior remains available in the dedicated command result flow.	2026-05-28 10:50:41 +02:00
Haile	44edf94f3a	Refactor provider/session architecture to be DB-driven, modular, and sessionId-first across backend and frontend (#715) ... * refactor: remove unused exports * refactor: remove unused fields from project and session objects * refactor: rename session_names table and related code to sessions for clarity and consistency * refactor(database): move db into typescript - Implemented githubTokensDb for managing GitHub tokens with CRUD operations. - Created otificationPreferencesDb to handle user notification preferences. - Added projectsDb for project path management and related operations. - Introduced pushSubscriptionsDb for managing browser push subscriptions. - Developed scanStateDb to track the last scanned timestamp. - Established sessionsDb for session management with CRUD functionalities. - Created userDb for user management, including authentication and onboarding. - Implemented apidKeysDb for storing and managing VAPID keys. feat(database): define schema for new database tables - Added SQL schema definitions for users, API keys, user credentials, notification preferences, VAPID keys, push subscriptions, projects, sessions, scan state, and app configuration. - Included necessary indexes for performance optimization. refactor(shared): enhance type definitions and utility functions - Updated shared types and interfaces for improved clarity and consistency. - Added new types for credential management and provider-specific operations. - Refined utility functions for better error handling and message normalization. * feat: added session indexer logic * perf(projects): lazy-load TaskMaster metadata per selected project Why: - /api/projects is a hot path (initial load, sidebar refresh, websocket sync). - Scanning .taskmaster for every project on each call added avoidable fs I/O and payload size. - TaskMaster metadata is only needed after selecting a specific project. - Moving it to a project-scoped endpoint makes loading cost match user intent. - The UI now hydrates TaskMaster state on selection and keeps it across refresh events. - This prevents status flicker/regression while still removing global scan overhead. - Selection fetches are sequence-guarded to block stale async responses on fast switching. - isManuallyAdded was removed from responses to keep the public project contract minimal. - Project dumps now use incrementing snapshot files to preserve history for debugging. What changed: - Added GET /api/projects/:projectName/taskmaster and getProjectTaskMaster(). - Removed TaskMaster detection from bulk getProjects(). - Added api.projectTaskmaster(...) plus selection-time hydration in frontend contexts. - Merged cached taskmaster values into refreshed project lists for continuity. - Removed isManuallyAdded from manual project payloads. * refactor: update import paths for database modules and remove legacy db.js and schema.js files * refactor(projects): identify projects by DB projectId instead of folder-derived name GET /api/projects used to scan ~/.claude/projects/ on every request, derive each project's identity from the encoded folder name, and re-parse JSONL files to build session lists. Using the folder-derived name as the project identifier leaked the Claude CLI's on-disk encoding into every API route, forced every downstream endpoint to re-resolve a real path via JSONL 'cwd' inspection, and made the project list endpoint O(projects x sessions) on disk I/O. This change switches the entire API surface to identify projects by the stable primary key from the 'projects' table and drives the listing straight from the DB: - Add projectsDb.getProjectPathById as the canonical projectId -> path resolver so routes no longer need to touch the filesystem to figure out where a project lives. - Rewrite getProjects so it reads the project list from the 'projects' table and the per-project session list from the 'sessions' table (one SELECT per project). No filesystem scanning happens for this endpoint anymore, which removes the dependency on ~/.claude/projects existing, on Cursor's MD5-hashed chat folders being discoverable, and on Codex's JSONL history being on disk. Per the migration spec each session now exposes 'summary' sourced from sessions.custom_name, 'messageCount' = 0 (message counting is not implemented), and sessionMeta.hasMore is pinned to false since this endpoint doesn't drive session pagination. - Introduce id-based wrappers (getSessionsById, renameProjectById, deleteSessionById, deleteProjectById, getProjectTaskMasterById) so every caller can pass projectId and resolve the real path through the DB. renameProjectById also writes to projects.custom_project_name so the DB-driven getProjects response reflects renames immediately; it keeps project-config.json in sync for any legacy reader that still consults the JSON file. - Migrate every /api/projects/:projectName route in server/index.js, server/routes/taskmaster.js, and server/routes/messages.js to :projectId, and change server/routes/git.js so the 'project' query/body parameter carries a projectId that is resolved through the DB before any git command runs. TaskMaster WebSocket broadcasts emit 'projectId' for the same reason so the frontend can match notifications against its current selection without another lookup. - Delete helpers that existed only to feed the old getProjects path (getCursorSessions, getGeminiCliSessions, getProjectTaskMaster) along with their unused imports (better-sqlite3's Database, applyCustomSessionNames). The legacy folder-name helpers (getSessions, renameProject, deleteSession, deleteProject, extractProjectDirectory) are kept as internal implementation details of the id-based wrappers and of destructive cleanup / conversation search, but they are no longer re-exported. - searchConversations still walks JSONL to produce match snippets (that data doesn't live in the DB), but it now includes the resolved projectId in each result so the sidebar can cross-reference hits with its already loaded project list without a second round-trip. Frontend migration: - Project.name is replaced by Project.projectId in src/types/app.ts, and ProjectSession.__projectName becomes __projectId so session tagging and sidebar state keys stay aligned with the backend identifier. Settings continues to use SettingsProject.name for legacy consumers, but it is populated from projectId by normalizeProjectForSettings. - All places that previously indexed per-project state by project.name (sidebar expanded/starred/loading/deletingProjects sets, additionalSessions map, projectHasMoreOverrides, starredProjects localStorage, command history and draft-input localStorage, TaskMaster caches) now key on projectId so state survives display-name edits and is consistent across the app. - src/utils/api.js renames every endpoint parameter to projectId, the unified messages endpoint takes projectId in its query string, and useSessionStore forwards projectId on fetchFromServer / fetchMore / refreshFromServer. Git panel, file tree, code editor, PRD editor, plugins context, MCP server flows and TaskMaster hooks are all updated to pass projectId. - DEFAULT_PROJECT_FOR_EMPTY_SHELL is updated to carry a 'default' projectId sentinel so the empty-shell placeholder still satisfies the Project contract. Bug fix bundled in: - sessionsDb.setName no longer bumps updated_at when a row already exists. Renaming is a label change, not activity, so there is no reason for it to reset 'last activity' in the sidebar. It also no longer relies on SQLite's CURRENT_TIMESTAMP, which stores a naive 'YYYY-MM-DD HH:MM:SS' value that JavaScript parses as local time and caused renamed sessions to appear shifted backwards by the client's UTC offset. When an INSERT actually happens it now writes ISO-8601 UTC with a 'Z' suffix. - buildSessionsByProviderFromDb normalizes any legacy naive timestamps in the sessions table to ISO-8601 UTC on the way out so rows written before this change also render correctly on the client. Other cleanup: - Removed the filesystem-first project-discovery comment block at the top of server/projects.js and replaced it with a short note that describes the new DB-driven flow and lists the few remaining filesystem-dependent helpers (message reads, search, destructive delete, manual project registration). - server/modules/providers/index.ts is added as a small barrel so the providers module exposes a stable public surface. Made-with: Cursor * refactor(projects): reorganize project-related logic into dedicated modules * refactor(projects): rename getProjects with getProjectsWithSessions * refactor: update import path for getProjectsWithSessions to include file extension * refactor: use updated session watcher In addition, for projects_updated websocket response, send the sessionId instead * refactor(websocket): move websocket logic to its own module * refactor(sessions-watcher): remove redundant logging after session sync completion * refactor(index.js): reorganize code structure * refactor(index.js): fix import order * refactor: remove unnecessary GitHub cloning logic from create-workspace endpoint * refactor: modularize project services, and wizard create/clone flow Restructure project creation, listing, GitHub clone progress, and TaskMaster details behind a dedicated TypeScript module under server/modules/projects/, and align the client wizard with a single path-based flow. Server / routing - Remove server/routes/projects.js and mount server/modules/projects/ projects.routes.ts at /api/projects (still behind authenticateToken). - Drop duplicate handlers from server/index.js for GET /api/projects and GET /api/projects/:projectId/taskmaster; those live on the new router. - Import WORKSPACES_ROOT and validateWorkspacePath from shared utils in index.js instead of the deleted projects route module. Projects router (projects.routes.ts) - GET /: list projects with sessions (existing snapshot behavior). - POST /create-project: validate body, reject legacy workspaceType and mixed clone fields, delegate to createProject service, return distinct success copy when an archived path is reactivated. - GET /clone-progress: Server-Sent Events for clone progress/complete/error; requires authenticated user id for token resolution; wires startCloneProject. - GET /:projectId/taskmaster: delegates to getProjectTaskMaster. Services (new) - project-management.service.ts: path validation, workspace directory creation, persistence via projectsDb.createProjectPath, mapping to API project shape; surfaces AppError for validation, conflict, and not-found cases; optional dependency injection for tests. - project-clone.service.ts: validates workspace, resolves GitHub auth (stored token or inline token), runs git clone with progress callbacks, registers project via createProject on success; sanitizes errors and supports cancellation; injectable dependencies for tests. - projects-has-taskmaster.service.ts: moves TaskMaster detection and normalization out of server/projects.js; resolve-by-id and public getProjectTaskMaster with structured AppError responses. Persistence and shared types - projectsDb.createProjectPath now returns CreateProjectPathResult (created | reactivated_archived | active_conflict) using INSERT … ON CONFLICT with selective update when the row is archived; normalizes display name from path or custom name; repository row typing moves to shared ProjectRepositoryRow. - getProjectPaths() returns only non-archived rows (isArchived = 0). - shared/types.ts: ProjectRepositoryRow, CreateProjectPathResult/outcome, WorkspacePathValidationResult. - shared/utils.ts: WORKSPACES_ROOT, forbidden path lists, validateWorkspacePath, asyncHandler for Express async routes. Legacy cleanup - server/projects.js: remove detectTaskMasterFolder, normalizeTaskMasterInfo, and getProjectTaskMasterById (logic lives in the new service). - server/routes/agent.js: register external API project paths with projectsDb.createProjectPath instead of addProjectManually try/catch; treat active_conflict as an existing registration and continue. Tests - Add Node test suites for project-management, project-clone, and projects-has-taskmaster services; update projects.service test import for renamed projects-with-sessions-fetch.service.ts. Rename - projects.service.ts → projects-with-sessions-fetch.service.ts; re-export from modules/projects/index.ts. Client (project creation wizard) - Remove StepTypeSelection and workspaceType from form state and types; wizard is two steps (configure path/GitHub auth, then review). - createWorkspaceRequest → createProjectRequest; clone vs create-only inferred from githubUrl (pathUtils / isCloneWorkflow). - Adjust step indices, WizardProgress, StepConfiguration/Review, WorkspacePathField, and src/utils/api.js as needed for the new API. Docs - Minor websocket README touch-up. Net: ~1.6k insertions / ~0.9k deletions across 29 files; behavior is centralized in typed services with explicit HTTP errors and test seams. * refactor: remove loading sessions logic from sidebar * refactor: move project rename to module * refactor: move project deletion to module * refactor: move project star state from localStorage to backend * refactor: implement optimistic UI for project star state management * feat: optimistic update for session watcher * fix(projects-state): stop websocket message reprocessing loop The websocket projects effect in useProjectsState could re-handle the same latestMessage after local state writes triggered re-renders. Under bursty websocket traffic, this created an update feedback cycle that surfaced as 'Maximum update depth exceeded', often from Sidebar. What changed: - Added lastHandledMessageRef so each latestMessage object is handled once. - Added an early return guard when the current message was already handled. - Made projects updates idempotent by comparing previous and merged payloads before calling setProjects. Result: - Breaks the effect -> state update -> effect re-entry cycle. - Reduces redundant renders during rapid projects_updated traffic while preserving normal project/session synchronization. * refactor: optimize project auto-expand logic * refactor: move projects provider specific logic into respective session providers * refactor: move rename and delete sessions to modules * refactor: move fetching messages to module * fix: remove unused var Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * Potential fix for pull request finding 'Useless assignment to local variable' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * refactor(projects/sidebar): remove temp snapshot side-effects and simplify session metadata UX Why this change was needed: - Project listing had an implicit side effect: every fetch wrote a debug snapshot under `.tmp/project-dumps`. That added unnecessary disk I/O to a hot path, introduced hidden runtime behavior, and created maintenance overhead for code that was not part of product functionality. - Keeping snapshot-specific exports/tests around made the projects module API broader than needed and coupled tests to temporary/debug behavior instead of user-visible behavior. - Codex sessions could remain stuck with a placeholder name (`Untitled Codex Session`) even after a real title became available from newer sync data, which degraded session discoverability in the UI. - Sidebar session rows showed duplicated provider branding and long-form relative times, which added visual noise and reduced scan speed when many sessions are listed. What changed: - Removed temporary projects snapshot dumping from `projects-with-sessions-fetch.service.ts`: - deleted snapshot types/helpers and file-write flow - removed the write call from `getProjectsWithSessions` - Removed snapshot-related surface area from `projects/index.ts`. - Removed the snapshot-focused test `projects.service.test.ts` that only validated removed debug behavior. - Updated `codex-session-synchronizer.provider.ts` to upgrade session names when an existing session still has the placeholder title but a real parsed name is now available. - Updated `SidebarSessionItem.tsx`: - removed duplicate provider logo rendering in each session row - moved age indicator to the right side - made age indicator fade on hover to prioritize action controls - switched to compact relative time format (`<1m`, `Xm`, `Xhr`, `Xd`) for faster list scanning Outcome: - Lower overhead and fewer hidden side effects in project fetches. - Cleaner module boundaries in projects. - Better Codex session naming consistency after sync. - Cleaner sidebar density and clearer hover/action behavior. * refactor: implement pagination for project sessions loading * refactor: move search to module * fix: search performance * refactor: add handling for internal Codex metadata in conversation search * fix(migrations,projects,clone): normalize legacy schema before writes and harden conflict detection Why - Legacy installs can have a sessions table shape that predates provider/custom_name columns. Running migrateLegacySessionNames first caused its INSERT OR REPLACE INTO sessions (...) to target columns that may not exist and fail during startup migration. - Some upgraded databases had projects.project_id as plain TEXT instead of a real PRIMARY KEY. That breaks assumptions used by id-based lookups and can allow invalid/duplicate identity semantics over time. - projectsDb.createProjectPath inferred outcomes from ow.isArchived, but the upsert path always returns the post-update row with isArchived=0, so archived-reactivation and fresh-create could be misclassified. - git clone accepted user-controlled URLs directly in argv position, so inputs beginning with - could be interpreted as options instead of a repository argument. What - Added ebuildProjectsTableWithPrimaryKeySchema in migrations: detect table shape via getTableInfo('projects'), verify project_id has pk=1, and rebuild when missing. - Rebuild flow now creates a canonical projects__new table (project_id TEXT PRIMARY KEY), copies rows with transformation, backfills empty ids via SQLITE_UUID_SQL, deduplicates conflicting ids/paths, then swaps tables inside a transaction. - Replaced the prior ddColumnToTableIfNotExists(...) + UPDATE project_id sequence with PK-aware detection/rebuild logic so legacy DBs converge to the required schema. - Reordered migration sequence to run ebuildSessionsTableWithProjectSchema before migrateLegacySessionNames, ensuring sessions is normalized before legacy session_names merge writes execute. - Updated projectsDb.createProjectPath to generate an ttemptedId before insert, pass it into the prepared statement, and classify outcomes by comparing returned ow.project_id to ttemptedId (created vs eactivated_archived), with no-row remaining ctive_conflict. - Hardened clone execution by inserting -- before clone URL in git argv and rejecting normalized GitHub URLs that start with - in startCloneProject. Tests - Added integration coverage for projectsDb.createProjectPath branches: fresh insert, archived reactivation, and active conflict. - Added clone service test for option-prefixed githubUrl rejection (INVALID_GITHUB_URL). * refactor(session-synchronizer): update last scanned timestamp based on synchronization results * refactor: improve session limit and offset validation in provider routes * refactor: normalize project paths across database and service modules * refactor(database): make session id the primary key in sessions table * fix(codex): preserve reasoning entries as thinking blocks Codex history normalization was downgrading reasoning into plain assistant text because of branch ordering, not because the raw data was missing. Why this mattered: - Codex reasoning JSONL entries are intentionally mapped to history items with type thinking, but they also carry message.role assistant. - normalizeHistoryEntry evaluated the assistant-role branch before the thinking branch. - As a result, reasoning content matched the assistant-text path first and was emitted as kind text instead of kind thinking. - This collapses semantic intent, so UI and downstream features that rely on thinking blocks (separate rendering, filtering, and interpretation of model thought process vs final answer) receive the wrong message kind. What changed: - Prioritized thinking detection (raw.type === thinking or raw.isReasoning) before role-based assistant normalization. - Kept a non-empty content guard for thinking payloads to avoid emitting empty artifacts. Impact: - Reasoning entries from persisted Codex JSONL now remain thinking blocks end-to-end. - Regular assistant text normalization behavior remains unchanged. * refactor: remove dead code * refactor: directly use getProjectPathById from projectsDb * refactor: add gemini jsonl session support	2026-04-30 08:19:26 +02:00