Compare commits

..

81 Commits

Author SHA1 Message Date
Simos Mikelatos
9bcb9436a1 ci: drop redundant desktop-macos-release workflow (consolidated on main)
Main consolidated per-OS desktop release workflows into desktop-release.yml
(d882f80). This standalone macOS workflow survived on the branch via a
modify/delete merge conflict and duplicates the build-macos job now in
desktop-release.yml.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 16:03:39 +00:00
Simos Mikelatos
19e4bf9d80 chore: remove computer use (re-applied after main merge)
The merge of main into this branch (0907d87) hit a modify/delete conflict
for the computer-use module — main removed it in 6761f31 while this branch
had extended it — and the conflict was resolved by keeping the branch's
copies, silently resurrecting the whole feature.

This re-applies 6761f31's removal on top of the browser-use work:
- deletes the computer-use server module, client panel, settings tab,
  desktop-agent websocket service, electron computerAgent, and the
  computer-semantics build scripts
- strips computer-use wiring from shared files (server/index.js, electron
  launcher/main, MainContent, settings, i18n) while preserving browser-use
- removes src/constants/featureFlags.ts (existed only for the CU menu flag)

browser-use (camoufox/noVNC) is untouched. typecheck passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 15:54:22 +00:00
Haile
a34249497e Merge branch 'main' into camoufox-novnc-browser-use 2026-07-01 16:57:16 +03:00
Simos Mikelatos
0907d873f6 Merge remote-tracking branch 'origin/main' into camoufox-novnc-browser-use
# Conflicts:
#	.github/workflows/release.yml
#	electron/desktopWindow.js
#	electron/launcher/launcher.js
#	electron/main.js
#	electron/preload.cjs
#	package.json
#	scripts/release/prepare-desktop-app.js
#	server/modules/websocket/services/websocket-server.service.ts
#	src/components/main-content/view/MainContent.tsx
#	src/components/main-content/view/subcomponents/MainContentTitle.tsx
2026-06-29 12:34:48 +00:00
Simos Mikelatos
ec437072ad Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	server/index.js
2026-06-29 07:43:33 +00:00
Haile
54f4d8aa36 Chat & sidebar UX improvements (#929) 2026-06-29 09:27:04 +02:00
Simos Mikelatos
261690935f Merge pull request #891 from siteboon/electron-app 2026-06-29 09:07:57 +02:00
Simos Mikelatos
46ba8e56b4 Potential fix for pull request finding 'Useless assignment to local variable'
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
2026-06-26 16:14:44 +02:00
Simos Mikelatos
a0899a252e Merge branch 'main' into electron-app 2026-06-26 16:09:19 +02:00
Simos Mikelatos
fff89e6132 Merge branch 'main' into camoufox-novnc-browser-use 2026-06-26 16:07:22 +02:00
Simos Mikelatos
3bc2c777a3 fix: await desktop auth token lookup 2026-06-26 10:33:10 +00:00
Simos Mikelatos
63f3c3941d feat: add desktop notifications and skills updates 2026-06-26 10:25:47 +00:00
Simos Mikelatos
e6c6f89dda Merge branch 'main' into electron-app 2026-06-26 10:02:48 +02:00
Simos Mikelatos
8adcdaa0e5 Merge branch 'main' into camoufox-novnc-browser-use 2026-06-24 23:16:13 +02:00
Simos Mikelatos
6f712269e8 fix: remove invalid windows build options 2026-06-24 21:05:59 +00:00
Simos Mikelatos
52244404a3 chore: remove windows icon generator 2026-06-24 20:50:45 +00:00
Simos Mikelatos
8ad18f8587 fix: improve desktop chat performance 2026-06-24 20:49:24 +00:00
Simos Mikelatos
fe116a7138 ci: restore notarized macOS branch builds 2026-06-24 20:25:53 +00:00
Simos Mikelatos
490e66ebdb fix: stabilize desktop environment auth navigation 2026-06-24 20:09:41 +00:00
Simos Mikelatos
81eb966904 ci: skip notarization for macOS branch builds 2026-06-24 20:05:52 +00:00
Simos Mikelatos
0d68dc2cd0 fix: add Electron tab diagnostics 2026-06-24 20:00:45 +00:00
Simos Mikelatos
0610cc8333 fix: browser use set profile root folder 2026-06-24 19:21:52 +00:00
Simos Mikelatos
9457651fdd fix(browser-use): harden browser settings state 2026-06-24 15:36:25 +00:00
Simos Mikelatos
8c31ebcc63 feat(browser-use): add Camoufox noVNC session viewer 2026-06-24 14:39:41 +00:00
Simos Mikelatos
bb630ef739 fix: hide computer use menus 2026-06-20 01:50:02 +00:00
Simos Mikelatos
1c05fe0905 fix: stabilize cloud computer use mcp 2026-06-19 20:47:53 +00:00
Simos Mikelatos
077baee5f2 fix: authenticate desktop agent websocket 2026-06-19 15:52:49 +00:00
coderabbitai[bot]
f150fa6b09 fix: apply CodeRabbit auto-fixes
Fixed 1 file(s) based on 1 unresolved review comment.

Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
2026-06-19 15:22:43 +00:00
Simos Mikelatos
9f8cee8919 fix: restore macos semantic helper cast 2026-06-19 15:05:47 +00:00
Simos Mikelatos
bb323fc566 fix: respect cloud computer use setting 2026-06-19 15:02:07 +00:00
Simos Mikelatos
5ef40be2d3 fix: macos release 2026-06-19 14:46:58 +00:00
Simos Mikelatos
cf4b28273e fix: compile macos semantic helper 2026-06-19 14:22:47 +00:00
Simos Mikelatos
f4c68942a5 fix: repair desktop launcher local view 2026-06-19 14:20:23 +00:00
Simos Mikelatos
4d70a2588c feat: improve Computer Use linking status 2026-06-19 13:47:16 +00:00
Simos Mikelatos
218e8e2e38 chore: update Codex SDK to latest 2026-06-19 13:12:53 +00:00
Simos Mikelatos
53c3c4c27a Fix long-running desktop resource leaks 2026-06-19 13:07:08 +00:00
Simos Mikelatos
901c6fc956 chore: simplify desktop release artifacts 2026-06-19 13:04:53 +00:00
Simos Mikelatos
278fe4f7b1 Fix semantic review issues and release action runtime 2026-06-19 12:46:40 +00:00
Simos Mikelatos
d7f4d4c342 Fix desktop release review findings 2026-06-19 12:29:46 +00:00
Simos Mikelatos
d1930fecdb fix: build semantic helpers on macos and windows 2026-06-19 12:17:32 +00:00
Simos Mikelatos
1726705459 feat: add CloudCLI computer use semantics, desktop helper packaging, and permission onboarding 2026-06-19 12:09:55 +00:00
Simos Mikelatos
a35200f340 Harden computer use MCP handling 2026-06-19 08:06:26 +00:00
Simos Mikelatos
06c9745489 Update src/i18n/locales/zh-CN/settings.json
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-06-19 09:56:21 +02:00
Simos Mikelatos
0dd22db2bb Update src/i18n/locales/zh-TW/settings.json
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-06-19 09:56:01 +02:00
Simos Mikelatos
e7aa72c41e Update src/i18n/locales/tr/settings.json
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-06-19 09:55:45 +02:00
Simos Mikelatos
9f24f80f33 Fix computer use session error status 2026-06-19 07:47:56 +00:00
Simos Mikelatos
25ab273b05 Publish branch server bundles for thin desktop builds 2026-06-19 07:08:19 +00:00
Simos Mikelatos
5be100ea1b Keep branch desktop artifacts thin 2026-06-19 06:49:28 +00:00
Simos Mikelatos
2af3d38afe Harden desktop workflows and computer use handling 2026-06-19 06:21:13 +00:00
Simos Mikelatos
531833bc87 Merge branch 'main' into electron-app 2026-06-19 08:19:36 +02:00
Simos Mikelatos
b2333e7d93 Fix launcher CodeQL unused helpers 2026-06-18 21:17:09 +00:00
Simos Mikelatos
f75ae385dd Add on-demand desktop server bundle 2026-06-18 21:08:29 +00:00
Simos Mikelatos
7786763dd1 Fix desktop settings modal behavior 2026-06-18 06:15:17 +00:00
Simos Mikelatos
1dbf545fd9 Authenticate ripgrep install in desktop workflows 2026-06-17 22:29:55 +00:00
Simos Mikelatos
ac37213269 Run desktop branch builds on electron app pushes 2026-06-17 22:26:47 +00:00
Simos Mikelatos
65fdc38f2e Add desktop app packaging and settings updates 2026-06-17 22:15:36 +00:00
Simos Mikelatos
6c2652aee6 Merge remote-tracking branch 'origin/main' into electron-app
# Conflicts:
#	package-lock.json
#	package.json
#	server/index.js
#	src/components/main-content/types/types.ts
#	src/components/main-content/view/MainContent.tsx
#	src/components/main-content/view/subcomponents/MainContentHeader.tsx
#	src/components/main-content/view/subcomponents/MainContentTabSwitcher.tsx
#	src/components/main-content/view/subcomponents/MainContentTitle.tsx
#	src/components/settings/hooks/useSettingsController.ts
#	src/components/settings/types/types.ts
#	src/components/settings/view/Settings.tsx
#	src/components/settings/view/SettingsSidebar.tsx
#	src/hooks/useProjectsState.ts
#	src/i18n/locales/de/common.json
#	src/i18n/locales/en/common.json
#	src/i18n/locales/en/settings.json
#	src/i18n/locales/it/common.json
#	src/i18n/locales/ja/common.json
#	src/i18n/locales/ko/common.json
#	src/i18n/locales/ru/common.json
#	src/i18n/locales/tr/common.json
#	src/i18n/locales/zh-CN/common.json
#	src/i18n/locales/zh-TW/common.json
#	src/types/app.ts
2026-06-17 20:18:09 +00:00
Simos Mikelatos
bf50d29c20 Merge remote-tracking branch 'origin/browser-use' into electron-app
# Conflicts:
#	src/i18n/locales/en/settings.json
2026-06-17 20:17:38 +00:00
Simos Mikelatos
ffc0cd7501 Improve Browser settings load and managed MCP display 2026-06-17 20:04:44 +00:00
Simos Mikelatos
59194d1502 Refine Browser naming and managed MCP UX
- Rename Browser Use surfaces to Browser
- Register Browser MCP under the new server name
- Mark CloudCLI-managed MCP servers read-only
- Adjust MCP stdio framing and sidebar footer sizing
2026-06-17 19:18:23 +00:00
Simos Mikelatos
7e6028b113 feat: add desktop computer use runtime 2026-06-17 19:01:15 +00:00
Simos Mikelatos
9881e5e366 feat(browser-use): improve mobile monitoring ux 2026-06-17 18:19:12 +00:00
Simos Mikelatos
496a895e8a feat(browser-use): refine monitoring panel ux 2026-06-17 17:39:55 +00:00
Simos Mikelatos
086df034b4 feat(browser-use): simplify agent session monitoring 2026-06-17 17:04:11 +00:00
Simos Mikelatos
fc71fc7d2b Merge branch 'pr889-fixes' into electron-app
# Conflicts:
#	server/index.js
2026-06-17 15:45:07 +00:00
Simos Mikelatos
a0d56429a7 fix browser use 2026-06-17 15:43:21 +00:00
Simos Mikelatos
6af4afe6f2 Merge branch 'main' into browser-use 2026-06-16 19:02:36 +02:00
Simos Mikelatos
7aeca52669 Merge branch 'browser-use' into electron-app
# Conflicts:
#	src/components/main-content/view/subcomponents/MainContentTabSwitcher.tsx
2026-06-16 06:51:35 +00:00
Simos Mikelatos
56532af33a feat: add browser use guide links 2026-06-15 21:22:49 +00:00
Simos Mikelatos
9438a365f2 feat: improve browser use session controls 2026-06-15 21:14:10 +00:00
Simos Mikelatos
e5c6e5e596 fix: hide browser use runtime mode 2026-06-15 20:20:44 +00:00
Simos Mikelatos
0426522406 feat: expose browser use to agents via MCP 2026-06-15 19:47:58 +00:00
Simos Mikelatos
6e7e2ff4c1 feat: make browser use opt-in 2026-06-15 18:12:27 +00:00
Simos Mikelatos
e6263dbd1f refactor: store browser use settings in database 2026-06-15 17:57:00 +00:00
Simos Mikelatos
260070bae0 feat: add browser use runtime setup settings 2026-06-15 17:52:27 +00:00
Simos Mikelatos
daac6e3fd3 ci: add macos desktop release workflow 2026-06-15 17:26:53 +00:00
Simos Mikelatos
861cfecbaa feat: add electron app support 2026-06-15 16:21:05 +00:00
Simos Mikelatos
a182765e10 Merge branch 'browser-use' into electron-app 2026-06-15 16:15:03 +00:00
Simos Mikelatos
828d1a2302 Merge remote-tracking branch 'origin/feat/unify-websocket-2' into browser-use-independent 2026-06-15 16:12:10 +00:00
Simos Mikelatos
d427004bd7 Merge browser use branch 2026-06-14 20:34:36 +00:00
Simos Mikelatos
243e6cecd5 Add browser use workspace panel 2026-06-14 20:34:16 +00:00
30 changed files with 1390 additions and 701 deletions

View File

@@ -489,7 +489,7 @@
<span class="endpoint-path"><span class="api-url">http://localhost:3001</span>/api/agent</span>
</div>
<p>Trigger an AI agent (Claude, Cursor, Codex, Gemini, or OpenCode) to work on a project.</p>
<p>Trigger an AI agent (Claude, Cursor, or Codex) to work on a project.</p>
<h4>Request Body Parameters</h4>
<table>
@@ -524,7 +524,7 @@
<td><code>provider</code></td>
<td>string</td>
<td><span class="badge badge-optional">Optional</span></td>
<td><code>claude</code>, <code>cursor</code>, <code>codex</code>, <code>gemini</code>, or <code>opencode</code> (default: <code>claude</code>)</td>
<td><code>claude</code>, <code>cursor</code>, or <code>codex</code> (default: <code>claude</code>)</td>
</tr>
<tr>
<td><code>stream</code></td>
@@ -540,12 +540,6 @@
Model identifier for the AI provider (loading from constants...)
</td>
</tr>
<tr>
<td><code>effort</code></td>
<td>string</td>
<td><span class="badge badge-optional">Optional</span></td>
<td>Reasoning effort for Claude and Codex models that expose effort metadata. Use <code>default</code> or omit it to let the provider decide.</td>
</tr>
<tr>
<td><code>cleanup</code></td>
<td>boolean</td>

View File

@@ -69,7 +69,7 @@ const sessionIdSchema = {
const tools: ToolDefinition[] = [
{
name: 'browser_create_session',
description: 'Create a temporary Browser session that the agent can control. Optionally provide a background profileName to reuse cookies and storage.',
description: 'Create a Browser session that the agent can control. Provide profileName to use a specific persistent profile; when omitted, the configured persistent profile is used only if session persistence is enabled, otherwise a temporary session is created.',
inputSchema: {
type: 'object',
properties: {

View File

@@ -12,13 +12,11 @@
* - WebSocket message streaming
*/
import { query } from '@anthropic-ai/claude-agent-sdk';
import crypto from 'crypto';
import { promises as fs } from 'fs';
import os from 'os';
import path from 'path';
import { query } from '@anthropic-ai/claude-agent-sdk';
import os from 'os';
import { CLAUDE_FALLBACK_MODELS } from './modules/providers/list/claude/claude-models.provider.js';
import { providerModelsService } from './modules/providers/services/provider-models.service.js';
import { resolveClaudeCodeExecutablePath } from './shared/claude-cli-path.js';
@@ -43,15 +41,6 @@ const TOOL_APPROVAL_TIMEOUT_MS = parseInt(process.env.CLAUDE_TOOL_APPROVAL_TIMEO
const TOOLS_REQUIRING_INTERACTION = new Set(['AskUserQuestion', 'ExitPlanMode']);
function resolveClaudeEffort(model, effort, modelsDefinition = CLAUDE_FALLBACK_MODELS) {
const selectedModel = modelsDefinition?.OPTIONS?.find((option) => option.value === model) || null;
const allowedEfforts = selectedModel?.effort?.values
?.map((value) => value.value) || [];
return typeof effort === 'string' && effort !== 'default' && allowedEfforts.includes(effort)
? effort
: undefined;
}
function createRequestId() {
if (typeof crypto.randomUUID === 'function') {
return crypto.randomUUID();
@@ -156,8 +145,13 @@ function matchesToolPermission(entry, toolName, input) {
return false;
}
/**
* Maps CLI options to SDK-compatible options format
* @param {Object} options - CLI options
* @returns {Object} SDK-compatible options
*/
function mapCliOptionsToSDK(options = {}) {
const { sessionId, cwd, toolsSettings, permissionMode, effort } = options;
const { sessionId, cwd, toolsSettings, permissionMode } = options;
const sdkOptions = {};
@@ -169,26 +163,32 @@ function mapCliOptionsToSDK(options = {}) {
// which does not reliably follow npm's shell wrappers like cross-spawn does.
sdkOptions.pathToClaudeCodeExecutable = resolveClaudeCodeExecutablePath(process.env.CLAUDE_CLI_PATH);
// Map working directory
if (cwd) {
sdkOptions.cwd = cwd;
}
// Map permission mode
if (permissionMode && permissionMode !== 'default') {
sdkOptions.permissionMode = permissionMode;
}
// Map tool settings
const settings = toolsSettings || {
allowedTools: [],
disallowedTools: [],
skipPermissions: false
};
// Handle tool permissions
if (settings.skipPermissions && permissionMode !== 'plan') {
// When skipping permissions, use bypassPermissions mode
sdkOptions.permissionMode = 'bypassPermissions';
}
let allowedTools = [...(settings.allowedTools || [])];
// Add plan mode default tools
if (permissionMode === 'plan') {
const planModeTools = ['Read', 'Task', 'exit_plan_mode', 'TodoRead', 'TodoWrite', 'WebFetch', 'WebSearch'];
for (const tool of planModeTools) {
@@ -207,24 +207,22 @@ function mapCliOptionsToSDK(options = {}) {
sdkOptions.disallowedTools = settings.disallowedTools || [];
// Map model (default to sonnet)
// Valid models: sonnet, opus, haiku, opusplan, sonnet[1m], fable
sdkOptions.model = options.model || CLAUDE_FALLBACK_MODELS.DEFAULT;
// Model logged at query start below
const resolvedEffort = resolveClaudeEffort(
sdkOptions.model,
effort,
options.effortModels || CLAUDE_FALLBACK_MODELS,
);
if (resolvedEffort) {
sdkOptions.effort = resolvedEffort;
}
// Map system prompt configuration
sdkOptions.systemPrompt = {
type: 'preset',
preset: 'claude_code'
preset: 'claude_code' // Required to use CLAUDE.md
};
// Map setting sources for CLAUDE.md loading
// This loads CLAUDE.md from project, user (~/.config/claude/CLAUDE.md), and local directories
sdkOptions.settingSources = ['project', 'user', 'local'];
// Map resume session
if (sessionId) {
sdkOptions.resume = sessionId;
}
@@ -535,24 +533,20 @@ async function queryClaudeSDK(command, options = {}, ws) {
sessionId,
options.model,
);
let effortModels = CLAUDE_FALLBACK_MODELS;
try {
effortModels = (await providerModelsService.getProviderModels('claude')).models;
} catch (error) {
console.warn('[Claude SDK] Unable to load provider models for effort validation:', error);
}
// Map CLI options to SDK format
const sdkOptions = mapCliOptionsToSDK({
...options,
model: resolvedModel || options.model,
effortModels,
});
// Load MCP configuration
const mcpServers = await loadMcpConfig(options.cwd);
if (mcpServers) {
sdkOptions.mcpServers = mcpServers;
}
// Handle images - save to temp files and modify prompt
const imageResult = await handleImages(command, options.images, options.cwd);
const finalCommand = imageResult.modifiedCommand;
tempImagePaths = imageResult.tempImagePaths;
@@ -656,7 +650,7 @@ async function queryClaudeSDK(command, options = {}, ws) {
return { behavior: 'deny', message: decision.message ?? 'User denied tool use' };
};
// Query constructor reads this synchronously.
// Set stream-close timeout for interactive tools (Query constructor reads it synchronously). Claude Agent SDK has a default of 5s and this overrides it
const prevStreamTimeout = process.env.CLAUDE_CODE_STREAM_CLOSE_TIMEOUT;
process.env.CLAUDE_CODE_STREAM_CLOSE_TIMEOUT = '300000';

View File

@@ -65,7 +65,7 @@ import providerRoutes from './modules/providers/provider.routes.js';
import voiceRoutes from './voice-proxy.js';
import browserUseRoutes from './modules/browser-use/browser-use.routes.js';
import browserUseMcpRoutes from './modules/browser-use/browser-use-mcp.routes.js';
import { browserUseService } from './modules/browser-use/browser-use.service.js';
import { browserUseService, VIEWER_COOKIE_NAME } from './modules/browser-use/index.js';
import { startEnabledPluginServers, stopAllPlugins, getPluginPort } from './utils/plugin-process-manager.js';
import { initializeDatabase, projectsDb, sessionsDb } from './modules/database/index.js';
import { configureWebPush } from './services/vapid-keys.js';
@@ -147,6 +147,8 @@ const wss = createWebSocketServer(server, {
shouldAutoOpenUrlFromOutput,
},
getPluginPort,
browserUseViewer: (ws, pathname) => browserUseService.handleViewerWebSocket(ws, pathname),
authenticateBrowserUseViewer: authenticateBrowserUseViewerPath,
});
// Make WebSocket server available to routes
@@ -214,11 +216,42 @@ app.use('/api/gemini', authenticateToken, geminiRoutes);
// Plugins API Routes (protected)
app.use('/api/plugins', authenticateToken, pluginsRoutes);
function readCookieValue(header, name) {
if (!header) return null;
const prefix = `${name}=`;
const cookie = String(header).split(';').map((part) => part.trim()).find((part) => part.startsWith(prefix));
return cookie ? decodeURIComponent(cookie.slice(prefix.length)) : null;
}
function authenticateBrowserUseViewerPath(pathname, token) {
const parts = String(pathname || '').split('/');
const sessionId = parts[4];
if (parts[1] !== 'api' || parts[2] !== 'browser-use' || parts[3] !== 'sessions' || parts[5] !== 'viewer' || parts[6] !== 'websockify') {
return false;
}
return browserUseService.validateViewerToken(decodeURIComponent(sessionId), token);
}
function authenticateBrowserUse(req, res, next) {
const match = /^\/sessions\/([^/]+)\/viewer(?:\/|$)/.exec(req.path || '');
if (match) {
const sessionId = decodeURIComponent(match[1]);
const token = typeof req.query.viewerToken === 'string'
? req.query.viewerToken
: readCookieValue(req.headers.cookie, VIEWER_COOKIE_NAME);
if (browserUseService.validateViewerToken(sessionId, token)) {
return next();
}
return res.status(401).json({ error: 'Browser viewer access requires a valid session token.' });
}
return authenticateToken(req, res, next);
}
// Browser MCP bridge API (local token protected)
app.use('/api/browser-use-mcp', browserUseMcpRoutes);
// Browser API Routes (protected)
app.use('/api/browser-use', authenticateToken, browserUseRoutes);
app.use('/api/browser-use', authenticateBrowserUse, browserUseRoutes);
// Unified provider MCP routes (protected)
app.use('/api/providers', authenticateToken, providerRoutes);

View File

@@ -1,6 +1,7 @@
import express from 'express';
import { browserUseService } from '@/modules/browser-use/browser-use.service.js';
import { VIEWER_COOKIE_NAME, VIEWER_TOKEN_TTL_MS } from '@/modules/browser-use/browser-use.viewer.js';
const router = express.Router();
@@ -8,6 +9,45 @@ function readParam(value: string | string[] | undefined): string {
return Array.isArray(value) ? value[0] || '' : value || '';
}
const SAFE_VIEWER_ROOT_FILES = new Set(['vnc.html', 'favicon.ico', 'manifest.json']);
const SAFE_VIEWER_ROOT_DIRS = new Set(['app', 'core', 'vendor', 'assets', 'images', 'utils']);
function isSafeViewerPath(viewerPath: string): boolean {
if (!viewerPath || viewerPath.startsWith('/') || viewerPath.includes('..') || viewerPath.includes('\\')) {
return false;
}
if (!/^[A-Za-z0-9][A-Za-z0-9._~/-]*$/.test(viewerPath)) {
return false;
}
if (SAFE_VIEWER_ROOT_FILES.has(viewerPath)) {
return true;
}
const [rootDir] = viewerPath.split('/');
return Boolean(rootDir && SAFE_VIEWER_ROOT_DIRS.has(rootDir));
}
function isSecureRequest(req: express.Request): boolean {
const forwardedProto = String(req.headers['x-forwarded-proto'] || '')
.split(',')[0]
.trim()
.toLowerCase();
return req.secure || forwardedProto === 'https';
}
function readQueryString(originalUrl: string): string {
const queryIndex = originalUrl.indexOf('?');
if (queryIndex < 0) {
return '';
}
const params = new URLSearchParams(originalUrl.slice(queryIndex + 1));
params.delete('viewerToken');
const nextQuery = params.toString();
return nextQuery ? `?${nextQuery}` : '';
}
router.get('/status', async (_req, res) => {
try {
res.json({ success: true, data: await browserUseService.getStatus() });
@@ -62,13 +102,60 @@ router.get('/sessions', async (_req, res) => {
try {
res.json({ success: true, data: { sessions: await browserUseService.listSessions() } });
} catch (error) {
res.status(401).json({
res.status(500).json({
success: false,
error: error instanceof Error ? error.message : 'Failed to list browser sessions.',
});
}
});
router.get('/sessions/:sessionId/viewer/*', async (req, res) => {
try {
const sessionId = readParam(req.params.sessionId);
const originalPath = req.originalUrl.split('?')[0] || '';
const viewerMarker = `/sessions/${sessionId}/viewer/`;
const markerIndex = originalPath.indexOf(viewerMarker);
const rawViewerPath = markerIndex >= 0 ? originalPath.slice(markerIndex + viewerMarker.length) : 'vnc.html';
const viewerPath = decodeURIComponent(rawViewerPath).replace(/^\/+/, '') || 'vnc.html';
if (!isSafeViewerPath(viewerPath)) {
res.status(400).json({ success: false, error: 'Invalid Browser viewer path.' });
return;
}
const viewerToken = readParam(req.query.viewerToken as string | string[] | undefined);
if (viewerPath === 'vnc.html' && browserUseService.validateViewerToken(sessionId, viewerToken)) {
res.cookie(VIEWER_COOKIE_NAME, viewerToken, {
httpOnly: true,
sameSite: 'lax',
secure: isSecureRequest(req),
maxAge: VIEWER_TOKEN_TTL_MS,
path: '/api/browser-use/sessions/' + encodeURIComponent(sessionId) + '/viewer',
});
}
const target = browserUseService.getViewerProxyTarget(sessionId);
const query = readQueryString(req.originalUrl);
const upstream = await fetch(`http://127.0.0.1:${target.websockifyPort}/${viewerPath}${query}`, {
headers: {
accept: String(req.headers.accept || '*/*'),
},
});
const contentType = upstream.headers.get('content-type');
if (contentType) {
res.setHeader('content-type', contentType);
}
const cacheControl = viewerPath === 'vnc.html' ? 'no-store' : 'public, max-age=3600';
res.setHeader('cache-control', cacheControl);
res.status(upstream.status);
const body = Buffer.from(await upstream.arrayBuffer());
res.send(body);
} catch (error) {
res.status(404).json({
success: false,
error: error instanceof Error ? error.message : 'Browser viewer is not available.',
});
}
});
router.post('/sessions/:sessionId/stop', async (req, res) => {
try {
const result = await browserUseService.stopSession(readParam(req.params.sessionId));

View File

@@ -1,128 +1,86 @@
import { createRequire } from 'node:module';
import { randomBytes, randomUUID } from 'node:crypto';
import { spawn } from 'node:child_process';
import { randomUUID } from 'node:crypto';
import { execFileSync, spawn } from 'node:child_process';
import fs from 'node:fs';
import net from 'node:net';
import os from 'node:os';
import path from 'node:path';
import { appConfigDb } from '@/modules/database/index.js';
import { WebSocket } from 'ws';
import { providerMcpService } from '@/modules/providers/index.js';
import { getModuleDir } from '@/utils/runtime-paths.js';
import {
getOrCreateMcpToken,
getProfilePath,
normalizeBrowserBackend,
PROFILE_ROOT,
readSettings,
resolveSessionProfileName,
useVisibleCamoufoxBackend,
writeSettings,
} from './browser-use.settings.js';
import type {
BrowserUseSession,
BrowserUseSettings,
PublicBrowserUseSession,
RuntimeHandle,
RuntimeProbe,
RuntimeReadiness,
} from './browser-use.types.js';
import { getViewerUrl, handleViewerWebSocket, VIEWER_TOKEN_TTL_MS } from './browser-use.viewer.js';
const require = createRequire(import.meta.url);
const __dirname = getModuleDir(import.meta.url);
const IS_PLATFORM = process.env.VITE_IS_PLATFORM === 'true';
const MAX_SESSIONS_PER_OWNER = Number.parseInt(process.env.CLOUDCLI_BROWSER_USE_MAX_SESSIONS_PER_OWNER || '3', 10);
const SESSION_TTL_MS = Number.parseInt(process.env.CLOUDCLI_BROWSER_USE_SESSION_TTL_MS || String(30 * 60 * 1000), 10);
const BROWSER_USE_SETTINGS_KEY = 'browser_use_settings';
const BROWSER_USE_MCP_TOKEN_KEY = 'browser_use_mcp_token';
type BrowserUseRuntime = 'cloud' | 'local';
type BrowserUseSessionStatus = 'ready' | 'stopped' | 'unavailable';
type BrowserUseSession = {
id: string;
ownerId: string;
createdBy: 'agent';
runtime: BrowserUseRuntime;
status: BrowserUseSessionStatus;
url: string | null;
title: string | null;
screenshotDataUrl: string | null;
createdAt: string;
updatedAt: string;
lastAction: string | null;
message: string | null;
profileName: string | null;
viewport: {
width: number;
height: number;
} | null;
cursor: {
x: number;
y: number;
actor: 'agent';
} | null;
};
type PublicBrowserUseSession = Omit<BrowserUseSession, 'ownerId'>;
type RuntimeHandle = {
browser?: any;
context?: any;
page?: any;
};
type BrowserUseSettings = {
enabled: boolean;
};
type RuntimeReadiness = {
playwright: any | null;
playwrightInstalled: boolean;
chromiumInstalled: boolean;
chromiumExecutablePath: string | null;
installInProgress: boolean;
installMessage: string | null;
};
type RuntimeProbe = Omit<RuntimeReadiness, 'installInProgress' | 'installMessage'>;
const sessions = new Map<string, BrowserUseSession>();
const handles = new Map<string, RuntimeHandle>();
const reservedDisplays = new Set<string>();
const viewerTokens = new Map<string, { token: string; expiresAt: number }>();
let installPromise: Promise<{ success: boolean; message: string }> | null = null;
let lastInstallMessage: string | null = null;
let runtimeProbeCache: { value: RuntimeProbe; updatedAt: number } | null = null;
const DEFAULT_SETTINGS: BrowserUseSettings = {
enabled: false,
};
const AGENT_OWNER_ID = 'agent';
const PROFILE_ROOT = path.join(os.homedir(), '.cloudcli', 'browser-use', 'profiles');
const MCP_SERVER_NAME = 'cloudcli-browser';
const LEGACY_MCP_SERVER_NAMES = ['cloudcli-browser-use'];
const RUNTIME_READINESS_CACHE_TTL_MS = 30_000;
const VISIBLE_BROWSER_ENABLED = process.env.CLOUDCLI_BROWSER_USE_VISIBLE !== 'false';
const RUNTIME_ROOT = process.env.CLOUDCLI_BROWSER_USE_RUNTIME_ROOT || '/opt/claudecodeui/.runtime-browser';
const NOVNC_ROOT = process.env.CLOUDCLI_BROWSER_USE_NOVNC_ROOT || path.join(RUNTIME_ROOT, 'novnc');
const X11VNC_BIN = process.env.CLOUDCLI_BROWSER_USE_X11VNC_BIN || path.join(RUNTIME_ROOT, 'rootfs/usr/bin/x11vnc');
const X11VNC_LIB_DIR = process.env.CLOUDCLI_BROWSER_USE_X11VNC_LIB_DIR || path.join(RUNTIME_ROOT, 'rootfs/usr/lib/x86_64-linux-gnu');
const X11VNC_EXTRA_LIB_DIR = process.env.CLOUDCLI_BROWSER_USE_X11VNC_EXTRA_LIB_DIR || path.join(RUNTIME_ROOT, 'rootfs/lib/x86_64-linux-gnu');
const LOG_RUNTIME_PROCESS_OUTPUT = process.env.CLOUDCLI_BROWSER_USE_RUNTIME_LOGS === 'true';
function getRuntime(): BrowserUseRuntime {
function getRuntime(): 'cloud' | 'local' {
return IS_PLATFORM ? 'cloud' : 'local';
}
function readSettings(): BrowserUseSettings {
function getCamoufoxExecutablePath(): string | null {
const configured = process.env.CLOUDCLI_BROWSER_USE_CAMOUFOX_EXECUTABLE;
if (configured && fs.existsSync(configured)) {
return configured;
}
try {
const raw = appConfigDb.get(BROWSER_USE_SETTINGS_KEY);
if (!raw) {
return DEFAULT_SETTINGS;
}
const parsed = JSON.parse(raw) as Partial<BrowserUseSettings>;
return {
enabled: parsed.enabled === true,
};
} catch (error: any) {
console.warn('[Browser] Failed to read settings:', error?.message || error);
return DEFAULT_SETTINGS;
const output = execFileSync(path.join(os.homedir(), '.local/bin/camoufox'), ['path'], {
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'ignore'],
}).trim();
const executablePath = fs.statSync(output).isDirectory()
? path.join(output, 'camoufox')
: output;
return fs.existsSync(executablePath) ? executablePath : null;
} catch {
return null;
}
}
function writeSettings(settings: BrowserUseSettings): BrowserUseSettings {
const normalized = {
enabled: settings.enabled === true,
};
appConfigDb.set(BROWSER_USE_SETTINGS_KEY, JSON.stringify(normalized));
return normalized;
}
function getOrCreateMcpToken(): string {
const existing = appConfigDb.get(BROWSER_USE_MCP_TOKEN_KEY);
if (existing) {
return existing;
}
const token = randomBytes(32).toString('hex');
appConfigDb.set(BROWSER_USE_MCP_TOKEN_KEY, token);
return token;
}
function getSetupMessage(settings: BrowserUseSettings, readiness: RuntimeReadiness): string {
if (!settings.enabled) {
return 'Browser is disabled in settings.';
@@ -132,6 +90,26 @@ function getSetupMessage(settings: BrowserUseSettings, readiness: RuntimeReadine
return 'Install Playwright and Chromium to use browser sessions.';
}
if (settings.browserBackend === 'camoufox-vnc' && !getCamoufoxExecutablePath()) {
return 'Camoufox is selected, but Camoufox is not installed.';
}
if (useVisibleCamoufoxBackend(settings)) {
if (!VISIBLE_BROWSER_ENABLED) {
return 'Camoufox is selected, but visible browser sessions are disabled.';
}
if (!getCamoufoxExecutablePath()) {
return 'Camoufox is selected, but Camoufox is not installed.';
}
if (!fs.existsSync(X11VNC_BIN)) {
return 'Camoufox is selected, but x11vnc is missing.';
}
if (!fs.existsSync(path.join(NOVNC_ROOT, 'vnc.html'))) {
return 'Camoufox is selected, but noVNC is missing.';
}
return readiness.installMessage || 'Camoufox runtime is not ready.';
}
if (!readiness.chromiumInstalled) {
return 'Playwright is installed, but Chromium is missing. Install the Chromium runtime to continue.';
}
@@ -176,24 +154,6 @@ async function removeMcpServerFromAllProviders(name: string) {
return results.map((result) => ({ ...result, name }));
}
function normalizeProfileName(profileName?: string | null): string | null {
const normalized = String(profileName || '').trim();
if (!normalized) {
return null;
}
return normalized.slice(0, 80);
}
function getProfilePath(profileName: string): string {
const safeName = profileName
.toLowerCase()
.replace(/[^a-z0-9._-]+/g, '-')
.replace(/^-+|-+$/g, '')
.slice(0, 80) || 'default';
return path.join(PROFILE_ROOT, safeName);
}
function probeRuntime(): RuntimeProbe {
const playwright = getPlaywright();
const readiness: RuntimeProbe = {
@@ -238,6 +198,175 @@ function getRuntimeReadiness(options: { force?: boolean } = {}): RuntimeReadines
};
}
function findAvailablePort(): Promise<number> {
return new Promise((resolve, reject) => {
const server = net.createServer();
server.on('error', reject);
server.listen(0, '127.0.0.1', () => {
const address = server.address();
server.close(() => {
if (typeof address === 'object' && address?.port) {
resolve(address.port);
} else {
reject(new Error('Failed to reserve a browser runtime port.'));
}
});
});
});
}
function delay(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function isRuntimeProcessAlive(child: ReturnType<typeof spawn>): boolean {
return child.exitCode === null && child.signalCode === null && !child.killed;
}
function assertRuntimeProcessesAlive(processes: Array<ReturnType<typeof spawn>>, label: string) {
const exited = processes.find((child) => !isRuntimeProcessAlive(child));
if (exited) {
throw new Error(`${label} exited before the Browser viewer runtime was ready.`);
}
}
async function isPortListening(port: number): Promise<boolean> {
return new Promise((resolve) => {
const socket = net.createConnection({ host: '127.0.0.1', port });
let settled = false;
const finish = (listening: boolean) => {
if (settled) {
return;
}
settled = true;
socket.destroy();
resolve(listening);
};
socket.setTimeout(250);
socket.once('connect', () => finish(true));
socket.once('timeout', () => finish(false));
socket.once('error', () => finish(false));
});
}
async function waitForRuntimePort(
port: number,
label: string,
processes: Array<ReturnType<typeof spawn>>,
timeoutMs = 5_000,
) {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
assertRuntimeProcessesAlive(processes, label);
if (await isPortListening(port)) {
return;
}
await delay(100);
}
assertRuntimeProcessesAlive(processes, label);
throw new Error(`${label} did not start listening on 127.0.0.1:${port}.`);
}
function killRuntimeProcesses(processes?: Array<ReturnType<typeof spawn>>) {
processes?.forEach((child) => child.kill('SIGTERM'));
}
function reserveDisplay(): string {
for (let index = 90; index < 140; index += 1) {
const display = `:${index}`;
if (!reservedDisplays.has(display)) {
reservedDisplays.add(display);
return display;
}
}
throw new Error('No browser display slots are available.');
}
function spawnRuntimeProcess(command: string, args: string[], options: { env?: NodeJS.ProcessEnv } = {}) {
const child = spawn(command, args, {
env: { ...process.env, ...options.env },
stdio: ['ignore', 'ignore', 'pipe'],
});
child.stderr?.on('data', (chunk) => {
if (!LOG_RUNTIME_PROCESS_OUTPUT) {
return;
}
const text = String(chunk).trim();
if (text) {
console.warn(`[Browser runtime] ${path.basename(command)}: ${text}`);
}
});
child.on('error', (error) => {
console.warn(`[Browser runtime] ${path.basename(command)} failed:`, error.message);
});
return child;
}
async function startVisibleRuntime(): Promise<NonNullable<RuntimeHandle['viewer']> & { processes: Array<ReturnType<typeof spawn>> }> {
const display = reserveDisplay();
const vncPort = await findAvailablePort();
const websockifyPort = await findAvailablePort();
const processes: Array<ReturnType<typeof spawn>> = [];
try {
processes.push(spawnRuntimeProcess('Xvfb', [
display,
'-screen',
'0',
'1440x900x24',
'-ac',
'-nolisten',
'tcp',
]));
await delay(700);
assertRuntimeProcessesAlive(processes, 'Xvfb');
if (!fs.existsSync(X11VNC_BIN)) {
throw new Error(`x11vnc is missing at ${X11VNC_BIN}.`);
}
processes.push(spawnRuntimeProcess(X11VNC_BIN, [
'-display',
display,
'-localhost',
'-forever',
'-shared',
'-rfbport',
String(vncPort),
'-nopw',
'-quiet',
], {
env: {
LD_LIBRARY_PATH: `${X11VNC_LIB_DIR}:${X11VNC_EXTRA_LIB_DIR}:${process.env.LD_LIBRARY_PATH || ''}`,
},
}));
await waitForRuntimePort(vncPort, 'x11vnc', processes);
if (!fs.existsSync(path.join(NOVNC_ROOT, 'vnc.html'))) {
throw new Error(`noVNC is missing at ${NOVNC_ROOT}.`);
}
processes.push(spawnRuntimeProcess(path.join(os.homedir(), '.local/bin/websockify'), [
'--web',
NOVNC_ROOT,
`127.0.0.1:${websockifyPort}`,
`127.0.0.1:${vncPort}`,
]));
await waitForRuntimePort(websockifyPort, 'websockify', processes);
return {
display,
vncPort,
websockifyPort,
noVncRoot: NOVNC_ROOT,
processes,
};
} catch (error) {
killRuntimeProcesses(processes);
reservedDisplays.delete(display);
throw error;
}
}
const INSTALL_COMMAND_TIMEOUT_MS = Number.parseInt(
process.env.CLOUDCLI_BROWSER_USE_INSTALL_TIMEOUT_MS || String(10 * 60 * 1000),
10,
@@ -350,6 +479,45 @@ function publicSession(session: BrowserUseSession): PublicBrowserUseSession {
return publicFields;
}
function getSessionViewer(sessionId: string): RuntimeHandle['viewer'] | null {
const session = sessions.get(sessionId);
if (!session || session.ownerId !== AGENT_OWNER_ID || session.status !== 'ready') {
return null;
}
return handles.get(sessionId)?.viewer || null;
}
function createViewerToken(sessionId: string): string {
const token = randomUUID();
viewerTokens.set(sessionId, {
token,
expiresAt: Date.now() + VIEWER_TOKEN_TTL_MS,
});
return token;
}
function deleteViewerToken(sessionId: string) {
viewerTokens.delete(sessionId);
}
function validateViewerTokenForSession(sessionId: string, token: string | null | undefined): boolean {
if (!token) {
return false;
}
const session = sessions.get(sessionId);
const viewer = session?.ownerId === AGENT_OWNER_ID && session.status === 'ready'
? handles.get(sessionId)?.viewer || null
: null;
const stored = viewerTokens.get(sessionId);
if (!viewer || !stored || stored.token !== token || stored.expiresAt < Date.now()) {
if (stored?.expiresAt && stored.expiresAt < Date.now()) {
viewerTokens.delete(sessionId);
}
return false;
}
return true;
}
function ownerSessions(ownerId: string): BrowserUseSession[] {
return [...sessions.values()].filter((session) => session.ownerId === ownerId);
}
@@ -357,8 +525,13 @@ function ownerSessions(ownerId: string): BrowserUseSession[] {
async function closeHandle(sessionId: string): Promise<void> {
const handle = handles.get(sessionId);
handles.delete(sessionId);
deleteViewerToken(sessionId);
await handle?.context?.close?.().catch(() => undefined);
await handle?.browser?.close().catch(() => undefined);
killRuntimeProcesses(handle?.processes);
if (handle?.viewer?.display) {
reservedDisplays.delete(handle.viewer.display);
}
}
async function expireStaleSessions(now = Date.now()): Promise<void> {
@@ -424,6 +597,11 @@ export const browserUseService = {
const current = readSettings();
const nextSettings = {
enabled: typeof settings.enabled === 'boolean' ? settings.enabled : current.enabled,
persistSessions: typeof settings.persistSessions === 'boolean' ? settings.persistSessions : current.persistSessions,
defaultProfileName: typeof settings.defaultProfileName === 'string'
? settings.defaultProfileName
: current.defaultProfileName,
browserBackend: settings.browserBackend ? normalizeBrowserBackend(settings.browserBackend) : current.browserBackend,
};
const next = writeSettings(nextSettings);
@@ -439,14 +617,28 @@ export const browserUseService = {
async getStatus() {
const settings = readSettings();
const readiness = getRuntimeReadiness();
const available = settings.enabled && readiness.playwrightInstalled && readiness.chromiumInstalled;
const useVisibleBackend = useVisibleCamoufoxBackend(settings);
const visibleCamoufoxReady = useVisibleBackend
&& VISIBLE_BROWSER_ENABLED
&& readiness.playwrightInstalled
&& Boolean(getCamoufoxExecutablePath())
&& fs.existsSync(X11VNC_BIN)
&& fs.existsSync(path.join(NOVNC_ROOT, 'vnc.html'));
const available = settings.enabled
&& readiness.playwrightInstalled
&& (useVisibleBackend ? visibleCamoufoxReady : readiness.chromiumInstalled);
return {
enabled: settings.enabled,
runtime: getRuntime(),
backend: useVisibleBackend ? 'camoufox-vnc' : 'playwright',
browserBackend: settings.browserBackend,
available,
playwrightInstalled: readiness.playwrightInstalled,
chromiumInstalled: readiness.chromiumInstalled,
camoufoxInstalled: Boolean(getCamoufoxExecutablePath()),
noVncInstalled: fs.existsSync(path.join(NOVNC_ROOT, 'vnc.html')),
x11vncInstalled: fs.existsSync(X11VNC_BIN),
installInProgress: readiness.installInProgress,
sessionCount: sessions.size,
message: available
@@ -505,7 +697,7 @@ export const browserUseService = {
}
await expireStaleSessions();
const profileName = normalizeProfileName(options?.profileName);
const profileName = resolveSessionProfileName(settings, options?.profileName);
const now = new Date().toISOString();
const session: BrowserUseSession = {
@@ -521,6 +713,9 @@ export const browserUseService = {
updatedAt: now,
lastAction: 'create',
message: null,
backend: useVisibleCamoufoxBackend(settings) ? 'camoufox-vnc' : 'playwright',
viewerUrl: null,
viewerEmbedUrl: null,
profileName,
viewport: { width: 1440, height: 900 },
cursor: null,
@@ -532,7 +727,13 @@ export const browserUseService = {
}
const readiness = getRuntimeReadiness();
if (!settings.enabled || !readiness.playwrightInstalled || !readiness.chromiumInstalled || !readiness.playwright) {
const useVisibleBackend = useVisibleCamoufoxBackend(settings);
const visibleCamoufoxReady = useVisibleBackend
&& VISIBLE_BROWSER_ENABLED
&& Boolean(getCamoufoxExecutablePath())
&& fs.existsSync(X11VNC_BIN)
&& fs.existsSync(path.join(NOVNC_ROOT, 'vnc.html'));
if (!settings.enabled || !readiness.playwrightInstalled || !readiness.playwright || (useVisibleBackend ? !visibleCamoufoxReady : !readiness.chromiumInstalled)) {
session.message = getSetupMessage(settings, readiness);
sessions.set(session.id, session);
return publicSession(session);
@@ -541,31 +742,73 @@ export const browserUseService = {
let browser: any | undefined;
let context: any | undefined;
let page: any;
const launchOptions = {
headless: true,
let viewer: RuntimeHandle['viewer'];
let processes: RuntimeHandle['processes'];
const launchOptions: Record<string, unknown> = {
headless: !useVisibleBackend,
args: ['--disable-dev-shm-usage'],
};
const contextOptions = {
viewport: { width: 1440, height: 900 },
serviceWorkers: 'block',
};
const contextOptions = useVisibleBackend
? { viewport: null }
: {
viewport: { width: 1440, height: 900 },
serviceWorkers: 'block',
};
if (profileName) {
fs.mkdirSync(PROFILE_ROOT, { recursive: true });
context = await readiness.playwright.chromium.launchPersistentContext(getProfilePath(profileName), {
...launchOptions,
...contextOptions,
});
page = context.pages()[0] || await context.newPage();
} else {
browser = await readiness.playwright.chromium.launch(launchOptions);
context = await browser.newContext(contextOptions);
page = await context.newPage();
try {
if (useVisibleBackend) {
const camoufoxExecutable = getCamoufoxExecutablePath();
if (!camoufoxExecutable) {
throw new Error('Camoufox is not installed.');
}
const runtime = await startVisibleRuntime();
viewer = {
display: runtime.display,
vncPort: runtime.vncPort,
websockifyPort: runtime.websockifyPort,
noVncRoot: runtime.noVncRoot,
};
processes = runtime.processes;
launchOptions.executablePath = camoufoxExecutable;
launchOptions.env = {
...process.env,
DISPLAY: runtime.display,
LD_LIBRARY_PATH: `${X11VNC_LIB_DIR}:${X11VNC_EXTRA_LIB_DIR}:${process.env.LD_LIBRARY_PATH || ''}`,
};
launchOptions.args = [];
session.backend = 'camoufox-vnc';
const viewerToken = createViewerToken(session.id);
session.viewerUrl = getViewerUrl(session.id, viewerToken);
session.viewerEmbedUrl = session.viewerUrl;
}
if (profileName) {
fs.mkdirSync(PROFILE_ROOT, { recursive: true });
const browserType = useVisibleBackend ? readiness.playwright.firefox : readiness.playwright.chromium;
context = await browserType.launchPersistentContext(getProfilePath(profileName), {
...launchOptions,
...contextOptions,
});
page = context.pages()[0] || await context.newPage();
} else {
const browserType = useVisibleBackend ? readiness.playwright.firefox : readiness.playwright.chromium;
browser = await browserType.launch(launchOptions);
context = await browser.newContext(contextOptions);
page = await context.newPage();
}
} catch (error) {
await context?.close?.().catch(() => undefined);
await browser?.close?.().catch(() => undefined);
killRuntimeProcesses(processes);
if (viewer?.display) {
reservedDisplays.delete(viewer.display);
}
throw error;
}
session.status = 'ready';
session.message = 'Browser session is ready.';
sessions.set(session.id, session);
handles.set(session.id, { browser, context, page });
handles.set(session.id, { browser, context, page, processes, viewer });
await captureSession(session, page);
return publicSession(session);
},
@@ -812,6 +1055,25 @@ export const browserUseService = {
return { deleted: true, sessionId };
},
getViewerProxyTarget(sessionId: string) {
const viewer = getSessionViewer(sessionId);
if (!viewer) {
throw new Error('Browser viewer is not available for this session.');
}
return {
websockifyPort: viewer.websockifyPort,
noVncRoot: viewer.noVncRoot,
};
},
validateViewerToken(sessionId: string, token: string | null | undefined) {
return validateViewerTokenForSession(sessionId, token);
},
handleViewerWebSocket(clientWs: WebSocket, pathname: string) {
handleViewerWebSocket(clientWs, pathname, getSessionViewer);
},
async agentStopSession(sessionId: string) {
await this.getAgentSession(sessionId);
return this.stopSession(sessionId);

View File

@@ -0,0 +1,147 @@
import { randomBytes } from 'node:crypto';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { appConfigDb } from '@/modules/database/index.js';
import type { BrowserUseBackend, BrowserUseSettings } from './browser-use.types.js';
const IS_PLATFORM = process.env.VITE_IS_PLATFORM === 'true';
const BROWSER_USE_SETTINGS_KEY = 'browser_use_settings';
const BROWSER_USE_MCP_TOKEN_KEY = 'browser_use_mcp_token';
const MAX_PROFILE_NAME_LENGTH = 80;
export const DEFAULT_BROWSER_USE_SETTINGS: BrowserUseSettings = {
enabled: false,
persistSessions: false,
defaultProfileName: 'default',
browserBackend: IS_PLATFORM ? 'camoufox-vnc' : 'playwright',
};
export const PROFILE_ROOT = process.env.CLOUDCLI_BROWSER_USE_PROFILE_ROOT
|| path.join(os.homedir(), '.cloudcli', 'browser-use', 'profiles');
export function normalizeBrowserBackend(value: unknown): BrowserUseBackend {
return value === 'playwright' || value === 'camoufox-vnc'
? value
: DEFAULT_BROWSER_USE_SETTINGS.browserBackend;
}
function trimEdgeDashes(value: string): string {
let start = 0;
let end = value.length;
while (start < end && value[start] === '-') {
start += 1;
}
while (end > start && value[end - 1] === '-') {
end -= 1;
}
return value.slice(start, end);
}
export function normalizeProfileName(profileName?: string | null): string | null {
const sanitized = trimEdgeDashes(String(profileName || '')
.trim()
.toLowerCase()
.replace(/[^a-z0-9._-]+/g, '-'));
const normalized = sanitized
.slice(0, MAX_PROFILE_NAME_LENGTH)
.replace(/^-+|-+$/g, '');
if (!normalized) {
return null;
}
return /[a-z0-9]/.test(normalized) ? normalized : null;
}
export function normalizeDefaultProfileName(profileName?: string | null): string {
return normalizeProfileName(profileName) || DEFAULT_BROWSER_USE_SETTINGS.defaultProfileName;
}
export function resolveSessionProfileName(settings: BrowserUseSettings, profileName?: string | null): string | null {
const requestedProfileName = normalizeProfileName(profileName);
if (String(profileName || '').trim() && !requestedProfileName) {
throw new Error('Browser profile name must include at least one letter or number.');
}
if (requestedProfileName) {
validateRequestedProfileName(profileName, requestedProfileName);
return requestedProfileName;
}
return settings.persistSessions ? normalizeDefaultProfileName(settings.defaultProfileName) : null;
}
export function getProfilePath(profileName: string): string {
return path.join(PROFILE_ROOT, normalizeDefaultProfileName(profileName));
}
function validateRequestedProfileName(profileName: string | null | undefined, normalizedProfileName: string): void {
const requestedProfileName = String(profileName || '').trim();
const existingProfileName = findExistingProfileName(normalizedProfileName);
if (
existingProfileName
&& (requestedProfileName !== normalizedProfileName || existingProfileName !== normalizedProfileName)
) {
throw new Error(`Browser profile "${requestedProfileName}" resolves to existing profile "${existingProfileName}". Use "${normalizedProfileName}" instead.`);
}
}
function findExistingProfileName(normalizedProfileName: string): string | null {
try {
if (!fs.existsSync(PROFILE_ROOT)) {
return null;
}
const entries = fs.readdirSync(PROFILE_ROOT, { withFileTypes: true });
const match = entries.find((entry) => entry.isDirectory() && normalizeProfileName(entry.name) === normalizedProfileName);
return match?.name || null;
} catch {
return null;
}
}
export function useVisibleCamoufoxBackend(settings: BrowserUseSettings): boolean {
return settings.browserBackend === 'camoufox-vnc';
}
export function readSettings(): BrowserUseSettings {
try {
const raw = appConfigDb.get(BROWSER_USE_SETTINGS_KEY);
if (!raw) {
return DEFAULT_BROWSER_USE_SETTINGS;
}
const parsed = JSON.parse(raw) as Partial<BrowserUseSettings>;
return {
enabled: parsed.enabled === true,
persistSessions: parsed.persistSessions === true,
defaultProfileName: normalizeDefaultProfileName(parsed.defaultProfileName),
browserBackend: normalizeBrowserBackend(parsed.browserBackend),
};
} catch (error: any) {
console.warn('[Browser] Failed to read settings:', error?.message || error);
return DEFAULT_BROWSER_USE_SETTINGS;
}
}
export function writeSettings(settings: BrowserUseSettings): BrowserUseSettings {
const normalized = {
enabled: settings.enabled === true,
persistSessions: settings.persistSessions === true,
defaultProfileName: normalizeDefaultProfileName(settings.defaultProfileName),
browserBackend: normalizeBrowserBackend(settings.browserBackend),
};
appConfigDb.set(BROWSER_USE_SETTINGS_KEY, JSON.stringify(normalized));
return normalized;
}
export function getOrCreateMcpToken(): string {
const existing = appConfigDb.get(BROWSER_USE_MCP_TOKEN_KEY);
if (existing) {
return existing;
}
const token = randomBytes(32).toString('hex');
appConfigDb.set(BROWSER_USE_MCP_TOKEN_KEY, token);
return token;
}

View File

@@ -0,0 +1,66 @@
import type { spawn } from 'node:child_process';
export type BrowserUseRuntime = 'cloud' | 'local';
export type BrowserUseBackend = 'playwright' | 'camoufox-vnc';
export type BrowserUseSessionStatus = 'ready' | 'stopped' | 'unavailable';
export type BrowserUseSession = {
id: string;
ownerId: string;
createdBy: 'agent';
runtime: BrowserUseRuntime;
status: BrowserUseSessionStatus;
url: string | null;
title: string | null;
screenshotDataUrl: string | null;
createdAt: string;
updatedAt: string;
lastAction: string | null;
message: string | null;
backend: BrowserUseBackend;
viewerUrl: string | null;
viewerEmbedUrl: string | null;
profileName: string | null;
viewport: {
width: number;
height: number;
} | null;
cursor: {
x: number;
y: number;
actor: 'agent';
} | null;
};
export type PublicBrowserUseSession = Omit<BrowserUseSession, 'ownerId'>;
export type RuntimeHandle = {
browser?: any;
context?: any;
page?: any;
processes?: Array<ReturnType<typeof spawn>>;
viewer?: {
display: string;
vncPort: number;
websockifyPort: number;
noVncRoot: string;
};
};
export type BrowserUseSettings = {
enabled: boolean;
persistSessions: boolean;
defaultProfileName: string;
browserBackend: BrowserUseBackend;
};
export type RuntimeReadiness = {
playwright: any | null;
playwrightInstalled: boolean;
chromiumInstalled: boolean;
chromiumExecutablePath: string | null;
installInProgress: boolean;
installMessage: string | null;
};
export type RuntimeProbe = Omit<RuntimeReadiness, 'installInProgress' | 'installMessage'>;

View File

@@ -0,0 +1,76 @@
import { WebSocket } from 'ws';
import type { RuntimeHandle } from './browser-use.types.js';
type BrowserUseViewer = NonNullable<RuntimeHandle['viewer']>;
export const VIEWER_COOKIE_NAME = 'browser_use_viewer_token';
const DEFAULT_VIEWER_TOKEN_TTL_MS = 30 * 60 * 1000;
const parsedViewerTokenTtlMs = Number.parseInt(
process.env.CLOUDCLI_BROWSER_USE_VIEWER_TOKEN_TTL_MS || String(DEFAULT_VIEWER_TOKEN_TTL_MS),
10,
);
export const VIEWER_TOKEN_TTL_MS =
Number.isFinite(parsedViewerTokenTtlMs) && parsedViewerTokenTtlMs > 0
? parsedViewerTokenTtlMs
: DEFAULT_VIEWER_TOKEN_TTL_MS;
export function getViewerUrl(sessionId: string, viewerToken?: string): string {
const basePath = `/api/browser-use/sessions/${encodeURIComponent(sessionId)}/viewer`;
const websockifyPath = viewerToken
? `${basePath}/websockify?viewerToken=${encodeURIComponent(viewerToken)}`
: `${basePath}/websockify`;
const params = new URLSearchParams({
autoconnect: '1',
resize: 'scale',
reconnect: '1',
path: websockifyPath,
});
if (viewerToken) {
params.set('viewerToken', viewerToken);
}
return `${basePath}/vnc.html?${params.toString()}`;
}
export function handleViewerWebSocket(
clientWs: WebSocket,
pathname: string,
getSessionViewer: (sessionId: string) => BrowserUseViewer | null | undefined,
) {
const match = /^\/api\/browser-use\/sessions\/([^/]+)\/viewer\/websockify\/?$/.exec(pathname);
const sessionId = match ? decodeURIComponent(match[1]) : '';
const viewer = sessionId ? getSessionViewer(sessionId) : null;
if (!viewer) {
clientWs.close(4404, 'Browser viewer not found');
return;
}
const upstream = new WebSocket(`ws://127.0.0.1:${viewer.websockifyPort}`);
upstream.on('open', () => {
clientWs.on('message', (data) => {
if (upstream.readyState === WebSocket.OPEN) {
upstream.send(data);
}
});
upstream.on('message', (data) => {
if (clientWs.readyState === WebSocket.OPEN) {
clientWs.send(data);
}
});
});
upstream.on('close', (code, reason) => {
if (clientWs.readyState === WebSocket.OPEN) {
clientWs.close(code, reason);
}
});
upstream.on('error', () => {
if (clientWs.readyState === WebSocket.OPEN) {
clientWs.close(4502, 'Browser viewer upstream error');
}
});
clientWs.on('close', () => {
if (upstream.readyState === WebSocket.OPEN) {
upstream.close();
}
});
}

View File

@@ -0,0 +1,2 @@
export { browserUseService } from './browser-use.service.js';
export { VIEWER_COOKIE_NAME } from './browser-use.viewer.js';

View File

@@ -0,0 +1,73 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import test from 'node:test';
const originalProfileRoot = process.env.CLOUDCLI_BROWSER_USE_PROFILE_ROOT;
const testProfileRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'browser-use-profiles-'));
process.env.CLOUDCLI_BROWSER_USE_PROFILE_ROOT = testProfileRoot;
const {
getProfilePath,
normalizeDefaultProfileName,
normalizeProfileName,
PROFILE_ROOT,
resolveSessionProfileName,
} = await import('@/modules/browser-use/browser-use.settings.js');
test.after(() => {
if (originalProfileRoot === undefined) {
delete process.env.CLOUDCLI_BROWSER_USE_PROFILE_ROOT;
} else {
process.env.CLOUDCLI_BROWSER_USE_PROFILE_ROOT = originalProfileRoot;
}
fs.rmSync(testProfileRoot, { recursive: true, force: true });
});
test('browser profile names are canonicalized before storage and path resolution', () => {
assert.equal(normalizeProfileName(' Work Profile!! '), 'work-profile');
assert.equal(normalizeProfileName(`${'-'.repeat(100)}Work Profile`), 'work-profile');
assert.equal(normalizeDefaultProfileName(' Work Profile!! '), 'work-profile');
assert.equal(
getProfilePath(' Work Profile!! '),
`${PROFILE_ROOT}/work-profile`,
);
assert.equal(
resolveSessionProfileName({
enabled: true,
persistSessions: true,
defaultProfileName: ' Work Profile!! ',
browserBackend: 'playwright',
}),
'work-profile',
);
});
test('browser profile aliases are rejected when the normalized profile already exists', () => {
const profileName = `alias-test-${Date.now()}`;
fs.mkdirSync(getProfilePath(profileName), { recursive: true });
try {
assert.throws(
() => resolveSessionProfileName({
enabled: true,
persistSessions: false,
defaultProfileName: 'default',
browserBackend: 'playwright',
}, profileName.toUpperCase()),
/resolves to existing profile/,
);
assert.equal(
resolveSessionProfileName({
enabled: true,
persistSessions: false,
defaultProfileName: 'default',
browserBackend: 'playwright',
}, profileName),
profileName,
);
} finally {
fs.rmSync(getProfilePath(profileName), { recursive: true, force: true });
}
});

View File

@@ -5,7 +5,6 @@ import type { IProviderModels } from '@/shared/interfaces.js';
import type {
ProviderChangeActiveModelInput,
ProviderCurrentActiveModel,
ProviderModelOption,
ProviderModelsDefinition,
ProviderSessionActiveModelChange,
} from '@/shared/types.js';
@@ -19,89 +18,27 @@ export const CLAUDE_FALLBACK_MODELS: ProviderModelsDefinition = {
{
value: 'default',
label: 'Default (recommended)',
description: 'Use the Claude Code default model (currently Sonnet 4.6)',
effort: {
default: 'high',
values: [
{ value: 'low' },
{ value: 'medium' },
{ value: 'high' },
{ value: 'max' },
],
},
description: 'Use the default model (currently Opus 4.8 (1M context)) · $5/$25 per Mtok',
},
{
value: 'fable',
label: 'Fable',
description: 'Fable 5 · Most capable for your hardest and longest-running tasks · Uses your limits ~2× faster than Opus',
effort: {
default: 'high',
values: [
{ value: 'low' },
{ value: 'medium' },
{ value: 'high' },
{ value: 'xhigh' },
{ value: 'max' },
],
},
},
{
value: "sonnet",
label: "Sonnet",
description: "Sonnet 4.6 · Best for everyday tasks · $3/$15 per Mtok",
effort: {
default: 'high',
values: [
{ value: 'low' },
{ value: 'medium' },
{ value: 'high' },
{ value: 'max' },
],
},
},
{
value: 'sonnet[1m]',
label: 'Sonnet (1M context)',
description: 'Sonnet 4.6 for long sessions · $3/$15 per Mtok',
effort: {
default: 'high',
values: [
{ value: 'low' },
{ value: 'medium' },
{ value: 'high' },
{ value: 'max' },
],
},
},
{
value: 'opus',
label: 'Opus',
description: 'Opus 4.8 · Best for everyday, complex tasks · ~2× usage vs Sonnet',
effort: {
default: 'high',
values: [
{ value: 'low' },
{ value: 'medium' },
{ value: 'high' },
{ value: 'xhigh' },
{ value: 'max' },
],
},
},
{
value: 'opus[1m]',
label: 'Opus 4.8 (1M context)',
description: 'Opus 4.8 with 1M context · Most capable for complex work · $5/$25 per Mtok',
effort: {
default: 'high',
values: [
{ value: 'low' },
{ value: 'medium' },
{ value: 'high' },
{ value: 'xhigh' },
{ value: 'max' },
],
},
},
{
value: 'haiku',
@@ -111,15 +48,6 @@ export const CLAUDE_FALLBACK_MODELS: ProviderModelsDefinition = {
],
DEFAULT: 'default',
};
export const findClaudeModelOption = (model: string | undefined | null): ProviderModelOption | null => {
const normalizedModel = typeof model === 'string' ? model.trim() : '';
if (!normalizedModel) {
return null;
}
return CLAUDE_FALLBACK_MODELS.OPTIONS.find((option) => option.value === normalizedModel) ?? null;
};
type ClaudeInitEvent = {
sessionId?: string;
session_id?: string;

View File

@@ -21,46 +21,11 @@ import {
export const CODEX_FALLBACK_MODELS: ProviderModelsDefinition = {
OPTIONS: [
{
value: 'gpt-5.5',
label: 'gpt-5.5',
effort: {
default: 'medium',
values: [{ value: 'low' }, { value: 'medium' }, { value: 'high' }, { value: 'xhigh' }],
},
},
{
value: 'gpt-5.4',
label: 'gpt-5.4',
effort: {
default: 'medium',
values: [{ value: 'low' }, { value: 'medium' }, { value: 'high' }, { value: 'xhigh' }],
},
},
{
value: 'gpt-5.4-mini',
label: 'gpt-5.4-mini',
effort: {
default: 'medium',
values: [{ value: 'low' }, { value: 'medium' }, { value: 'high' }, { value: 'xhigh' }],
},
},
{
value: 'gpt-5.3-codex',
label: 'gpt-5.3-codex',
effort: {
default: 'medium',
values: [{ value: 'low' }, { value: 'medium' }, { value: 'high' }, { value: 'xhigh' }],
},
},
{
value: 'gpt-5.2',
label: 'gpt-5.2',
effort: {
default: 'medium',
values: [{ value: 'low' }, { value: 'medium' }, { value: 'high' }, { value: 'xhigh' }],
},
},
{ value: 'gpt-5.5', label: 'gpt-5.5' },
{ value: 'gpt-5.4', label: 'gpt-5.4' },
{ value: 'gpt-5.4-mini', label: 'gpt-5.4-mini' },
{ value: 'gpt-5.3-codex', label: 'gpt-5.3-codex' },
{ value: 'gpt-5.2', label: 'gpt-5.2' },
],
DEFAULT: 'gpt-5.4',
};
@@ -72,11 +37,6 @@ type CodexCachedModel = {
priority?: number;
visibility?: string;
supported_in_api?: boolean;
default_reasoning_level?: string;
supported_reasoning_levels?: Array<{
effort?: string;
description?: string;
}>;
};
const CODEX_MODELS_CACHE_PATH = path.join(os.homedir(), '.codex', 'models_cache.json');
@@ -91,39 +51,15 @@ const readCodexPriority = (value: unknown): number => (
typeof value === 'number' && Number.isFinite(value) ? value : Number.MAX_SAFE_INTEGER
);
const mapCodexModel = (model: CodexCachedModel): ProviderModelOption => {
const effortValues = Array.isArray(model.supported_reasoning_levels)
? model.supported_reasoning_levels
.map((level) => {
const value = readOptionalString(level?.effort);
if (!value) {
return null;
}
return {
value,
description: readOptionalString(level?.description),
};
})
.filter((level): level is NonNullable<typeof level> => Boolean(level))
: [];
return {
value: model.slug as string,
label: readOptionalString(model.display_name) ?? (model.slug as string),
description: readOptionalString(model.description),
effort: effortValues.length > 0
? {
default: readOptionalString(model.default_reasoning_level) ?? undefined,
values: effortValues,
}
: undefined,
};
};
const mapCodexModel = (model: CodexCachedModel): ProviderModelOption => ({
value: model.slug as string,
label: readOptionalString(model.display_name) ?? (model.slug as string),
description: readOptionalString(model.description),
});
const buildCodexModelsDefinition = (models: CodexCachedModel[]): ProviderModelsDefinition => {
const sortedModels = [...models]
.filter((model) => model.visibility === 'list' && model.supported_in_api !== false)
.filter((model) => model.visibility !== 'hidden' && model.supported_in_api !== false)
.sort((left, right) => readCodexPriority(left.priority) - readCodexPriority(right.priority));
const options: ProviderModelOption[] = [];

View File

@@ -21,8 +21,6 @@ type ProviderCapabilities = {
supportsPermissionRequests: boolean;
/** Whether the token-usage endpoint has data for this provider. */
supportsTokenUsage: boolean;
/** Whether the provider runtime can accept model-level reasoning effort. */
supportsEffort: boolean;
};
/**
@@ -40,7 +38,6 @@ const PROVIDER_CAPABILITIES: Record<LLMProvider, ProviderCapabilities> = {
supportsAbort: true,
supportsPermissionRequests: true,
supportsTokenUsage: true,
supportsEffort: true,
},
cursor: {
provider: 'cursor',
@@ -50,7 +47,6 @@ const PROVIDER_CAPABILITIES: Record<LLMProvider, ProviderCapabilities> = {
supportsAbort: true,
supportsPermissionRequests: false,
supportsTokenUsage: false,
supportsEffort: false,
},
codex: {
provider: 'codex',
@@ -60,7 +56,6 @@ const PROVIDER_CAPABILITIES: Record<LLMProvider, ProviderCapabilities> = {
supportsAbort: true,
supportsPermissionRequests: false,
supportsTokenUsage: true,
supportsEffort: true,
},
gemini: {
provider: 'gemini',
@@ -70,7 +65,6 @@ const PROVIDER_CAPABILITIES: Record<LLMProvider, ProviderCapabilities> = {
supportsAbort: true,
supportsPermissionRequests: false,
supportsTokenUsage: true,
supportsEffort: false,
},
opencode: {
provider: 'opencode',
@@ -80,7 +74,6 @@ const PROVIDER_CAPABILITIES: Record<LLMProvider, ProviderCapabilities> = {
supportsAbort: true,
supportsPermissionRequests: false,
supportsTokenUsage: true,
supportsEffort: false,
},
};

View File

@@ -16,7 +16,7 @@ import type {
import { readProviderSessionActiveModelChange } from '@/shared/utils.js';
export const PROVIDER_MODELS_CACHE_TTL_MS = 3 * 24 * 60 * 60 * 1000;
const PROVIDER_MODELS_CACHE_VERSION = 2;
const PROVIDER_MODELS_CACHE_VERSION = 1;
const UNCACHED_PROVIDERS = new Set<LLMProvider>(['claude', 'gemini']);
type ProviderModelsServiceDependencies = {

View File

@@ -1,8 +1,9 @@
import type { Server as HttpServer } from 'node:http';
import { WebSocketServer, type VerifyClientCallbackSync } from 'ws';
import { WebSocket, WebSocketServer, type VerifyClientCallbackSync } from 'ws';
import { handleChatConnection } from '@/modules/websocket/services/chat-websocket.service.js';
import { VIEWER_COOKIE_NAME } from '@/modules/browser-use/index.js';
import { verifyWebSocketClient } from '@/modules/websocket/services/websocket-auth.service.js';
import { handlePluginWsProxy } from '@/modules/websocket/services/plugin-websocket-proxy.service.js';
import { handleShellConnection } from '@/modules/websocket/services/shell-websocket.service.js';
@@ -14,8 +15,21 @@ type WebSocketServerDependencies = {
chat: Parameters<typeof handleChatConnection>[2];
shell: Parameters<typeof handleShellConnection>[1];
getPluginPort: Parameters<typeof handlePluginWsProxy>[2];
browserUseViewer?: (ws: WebSocket, pathname: string) => void;
authenticateBrowserUseViewer?: (pathname: string, token: string | null) => boolean;
};
function readCookieValue(header: unknown, name: string): string | null {
if (!header) return null;
const prefix = `${name}=`;
const cookie = String(header).split(';').map((part) => part.trim()).find((part) => part.startsWith(prefix));
return cookie ? decodeURIComponent(cookie.slice(prefix.length)) : null;
}
function getBrowserUseViewerToken(url: URL, headers: Record<string, unknown>): string | null {
return url.searchParams.get('viewerToken') || readCookieValue(headers.cookie, VIEWER_COOKIE_NAME);
}
/**
* Creates and wires the server-wide websocket gateway used for chat, shell, and
* plugin proxy routes.
@@ -28,7 +42,17 @@ export function createWebSocketServer(
server,
verifyClient: ((
info: Parameters<VerifyClientCallbackSync<AuthenticatedWebSocketRequest>>[0]
) => verifyWebSocketClient(info, dependencies.verifyClient)),
) => {
const requestUrl = new URL(info.req.url ?? '/', 'http://localhost');
if (
requestUrl.pathname.startsWith('/api/browser-use/sessions/')
&& requestUrl.pathname.endsWith('/viewer/websockify')
) {
const token = getBrowserUseViewerToken(requestUrl, info.req.headers as Record<string, unknown>);
return Boolean(dependencies.authenticateBrowserUseViewer?.(requestUrl.pathname, token));
}
return verifyWebSocketClient(info, dependencies.verifyClient);
}),
});
wss.on('connection', (ws, request) => {
@@ -74,6 +98,11 @@ export function createWebSocketServer(
return;
}
if (pathname.startsWith('/api/browser-use/sessions/') && pathname.endsWith('/viewer/websockify')) {
dependencies.browserUseViewer?.(ws, pathname);
return;
}
console.log('[WARN] Unknown WebSocket path:', pathname);
ws.close();
});

View File

@@ -20,6 +20,7 @@ import { providerAuthService } from './modules/providers/services/provider-auth.
import { providerModelsService } from './modules/providers/services/provider-models.service.js';
import { createCompleteMessage, createNormalizedMessage } from './shared/utils.js';
// Track active sessions
const activeCodexSessions = new Map();
function readUsageNumber(value) {
@@ -227,7 +228,6 @@ export async function queryCodex(command, options = {}, ws) {
cwd,
projectPath,
model,
effort,
permissionMode = 'default'
} = options;
@@ -239,12 +239,6 @@ export async function queryCodex(command, options = {}, ws) {
const workingDirectory = cwd || projectPath || process.cwd();
const { sandboxMode, approvalPolicy } = mapPermissionModeToCodexOptions(permissionMode);
const catalog = (await providerModelsService.getProviderModels('codex')).models;
const selectedModel = catalog.OPTIONS.find((option) => option.value === resolvedModel) || null;
const allowedEfforts = selectedModel?.effort?.values?.map((value) => value.value) || [];
const resolvedEffort = typeof effort === 'string' && effort !== 'default' && allowedEfforts.includes(effort)
? effort
: undefined;
let codex;
let thread;
@@ -254,17 +248,19 @@ export async function queryCodex(command, options = {}, ws) {
const abortController = new AbortController();
try {
// Initialize Codex SDK
codex = new Codex();
// Thread options with sandbox and approval settings
const threadOptions = {
workingDirectory,
skipGitRepoCheck: true,
sandboxMode,
approvalPolicy,
model: resolvedModel,
modelReasoningEffort: resolvedEffort,
model: resolvedModel
};
// Start or resume thread
if (sessionId) {
thread = codex.resumeThread(sessionId, threadOptions);
} else {
@@ -284,10 +280,12 @@ export async function queryCodex(command, options = {}, ws) {
});
};
// Existing sessions can be tracked immediately; new sessions are tracked after thread.started.
if (capturedSessionId) {
registerSession(capturedSessionId);
}
// Execute with streaming
const streamedTurn = await thread.runStreamed(command, {
signal: abortController.signal
});

View File

@@ -646,17 +646,12 @@ class ResponseCollector {
*
* @param {string} model - (Optional) Model identifier for providers.
*
* Claude models: 'default', 'sonnet', 'opus', 'haiku', 'sonnet[1m]', 'opus[1m]', 'fable'
* Claude models: 'sonnet' (default), 'opus', 'haiku', 'opusplan', 'sonnet[1m]', 'fable'
* Cursor models: 'gpt-5' (default), 'gpt-5.2', 'gpt-5.2-high', 'sonnet-4.5', 'opus-4.5',
* 'gemini-3-pro', 'composer-1', 'auto', 'gpt-5.1', 'gpt-5.1-high',
* 'gpt-5.1-codex', 'gpt-5.1-codex-high', 'gpt-5.1-codex-max',
* 'gpt-5.1-codex-max-high', 'opus-4.1', 'grok', and thinking variants
* Codex models: 'gpt-5.4' (default), 'gpt-5.5', 'gpt-5.4-mini', 'gpt-5.3-codex', 'gpt-5.2'
*
* @param {string} effort - (Optional) Reasoning effort for providers/models that support it.
* Claude supports: 'low', 'medium', 'high', 'xhigh', 'max' depending on model.
* Codex supports: 'low', 'medium', 'high', 'xhigh'.
* 'default' or omission lets the provider decide.
* Codex models: 'gpt-5.2' (default), 'gpt-5.1-codex-max', 'o3', 'o4-mini'
*
* @param {boolean} cleanup - (Optional) Auto-cleanup project directory after completion.
* Default: true
@@ -849,9 +844,6 @@ class ResponseCollector {
*/
router.post('/', validateExternalApiKey, async (req, res) => {
const { githubUrl, projectPath, message, provider = 'claude', model, githubToken, branchName, sessionId } = req.body;
const effort = typeof req.body.effort === 'string' && req.body.effort.trim()
? req.body.effort.trim()
: undefined;
// Parse stream and cleanup as booleans (handle string "true"/"false" from curl)
const stream = req.body.stream === undefined ? true : (req.body.stream === true || req.body.stream === 'true');
@@ -962,7 +954,6 @@ router.post('/', validateExternalApiKey, async (req, res) => {
cwd: finalProjectPath,
sessionId: sessionId || null,
model: model,
effort,
permissionMode: 'bypassPermissions' // Bypass all permissions for API calls
}, writer);
@@ -984,7 +975,6 @@ router.post('/', validateExternalApiKey, async (req, res) => {
cwd: finalProjectPath,
sessionId: sessionId || null,
model: model || codexModels.DEFAULT,
effort,
permissionMode: 'bypassPermissions'
}, writer);
} else if (provider === 'gemini') {

View File

@@ -74,13 +74,6 @@ export type ProviderModelOption = {
value: string;
label: string;
description?: string;
effort?: {
default?: string;
values: {
value: string;
description?: string;
}[];
};
};
/**

View File

@@ -1,4 +1,4 @@
import { useCallback, useEffect, useMemo, useState } from 'react';
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import {
Bot,
Clock3,
@@ -7,6 +7,7 @@ import {
ExternalLink,
Loader2,
MonitorPlay,
MousePointer2,
RefreshCw,
Settings,
Square,
@@ -19,9 +20,14 @@ import { Badge, Button } from '../../../shared/view/ui';
import { authenticatedFetch } from '../../../utils/api';
import type { SettingsMainTab } from '../../settings/types/types';
const BROWSER_USE_GUIDE_URL = 'https://cloudcli.ai/docs/browser-use';
const BROWSER_USE_CACHE_TTL_MS = 30_000;
type BrowserUseStatus = {
enabled: boolean;
available: boolean;
backend: 'playwright' | 'camoufox-vnc';
browserBackend: 'playwright' | 'camoufox-vnc';
playwrightInstalled: boolean;
chromiumInstalled: boolean;
installInProgress: boolean;
@@ -39,6 +45,9 @@ type BrowserUseSession = {
updatedAt: string;
lastAction: string | null;
message: string | null;
backend?: 'playwright' | 'camoufox-vnc';
viewerUrl?: string | null;
viewerEmbedUrl?: string | null;
createdBy: 'agent';
profileName: string | null;
viewport: {
@@ -54,17 +63,48 @@ type BrowserUseSession = {
type BrowserUsePanelProps = {
isVisible: boolean;
projectId?: string | null;
onShowSettings?: (tab?: SettingsMainTab) => void;
};
type BrowserUsePanelCacheEntry = {
status: BrowserUseStatus | null;
sessions: BrowserUseSession[];
selectedSessionId: string | null;
updatedAt: number;
};
const browserUsePanelCache = new Map<string, BrowserUsePanelCacheEntry>();
async function readJson<T>(response: Response): Promise<T> {
const data = await response.json();
const text = await response.text();
let data: any = {};
if (text) {
try {
data = JSON.parse(text);
} catch {
throw new Error(response.ok ? 'Received an invalid Browser response.' : `Browser request failed (${response.status}).`);
}
}
if (!response.ok || data.success === false) {
throw new Error(data.error || data.details || `Request failed (${response.status})`);
}
return data as T;
}
async function fetchBrowserPanelData() {
const [statusResponse, sessionsResponse] = await Promise.all([
authenticatedFetch('/api/browser-use/status'),
authenticatedFetch('/api/browser-use/sessions'),
]);
const statusData = await readJson<{ data: BrowserUseStatus }>(statusResponse);
const sessionsData = await readJson<{ data: { sessions: BrowserUseSession[] } }>(sessionsResponse);
return {
status: statusData.data,
sessions: [...sessionsData.data.sessions].sort((a, b) => Date.parse(b.createdAt) - Date.parse(a.createdAt)),
};
}
function formatRelativeTime(value: string | null): string {
if (!value) return 'Never';
@@ -119,20 +159,42 @@ function getStatusDot(status: BrowserUseSession['status']): string {
return 'bg-border';
}
function getEngineLabel(backend?: BrowserUseStatus['backend'] | BrowserUseSession['backend']): string {
return backend === 'camoufox-vnc' ? 'Visible browser' : 'Playwright';
}
const PROMPTS = [
'Use Browser to inspect the checkout flow and report any broken UI states.',
'Open <url> with Browser, interact with the page, and summarize what changed after each step.',
];
export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUsePanelProps) {
const [status, setStatus] = useState<BrowserUseStatus | null>(null);
const [sessions, setSessions] = useState<BrowserUseSession[]>([]);
const [selectedSessionId, setSelectedSessionId] = useState<string | null>(null);
function getBrowserUseCacheKey(projectId?: string | null): string {
return projectId ? `browser-use:project:${projectId}` : 'browser-use:global';
}
function getFreshCacheEntry(cacheKey: string): BrowserUsePanelCacheEntry | null {
const entry = browserUsePanelCache.get(cacheKey);
if (!entry || Date.now() - entry.updatedAt > BROWSER_USE_CACHE_TTL_MS) {
return null;
}
return entry;
}
export default function BrowserUsePanel({ isVisible, projectId, onShowSettings }: BrowserUsePanelProps) {
const cacheKey = getBrowserUseCacheKey(projectId);
const initialCacheEntry = getFreshCacheEntry(cacheKey);
const [status, setStatus] = useState<BrowserUseStatus | null>(() => initialCacheEntry?.status ?? null);
const [sessions, setSessions] = useState<BrowserUseSession[]>(() => initialCacheEntry?.sessions ?? []);
const [selectedSessionId, setSelectedSessionId] = useState<string | null>(() => (
initialCacheEntry?.selectedSessionId || initialCacheEntry?.sessions[0]?.id || null
));
const [hasLoadedOnce, setHasLoadedOnce] = useState(Boolean(initialCacheEntry));
const [isRefreshing, setIsRefreshing] = useState(false);
const [isBusy, setIsBusy] = useState(false);
const [isInstalling, setIsInstalling] = useState(false);
const [isFullscreen, setIsFullscreen] = useState(false);
const [error, setError] = useState<string | null>(null);
const activeLoadIdRef = useRef(0);
const selectedSession = useMemo(
() => sessions.find((session) => session.id === selectedSessionId) || sessions[0] || null,
@@ -140,8 +202,12 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
);
const activeSessions = sessions.filter((session) => session.status === 'ready');
const needsBrowserBinaries = Boolean(status?.enabled && (!status.playwrightInstalled || !status.chromiumInstalled));
const runtimeLabel = !status?.enabled
const isInitialLoading = isRefreshing && !hasLoadedOnce && sessions.length === 0;
const isBackgroundRefreshing = isRefreshing && !isInitialLoading;
const needsBrowserBinaries = Boolean(status?.enabled && !status.available);
const runtimeLabel = isInitialLoading
? 'Loading'
: !status?.enabled
? 'Disabled'
: status.available
? 'Ready'
@@ -157,29 +223,72 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
: null;
const refresh = useCallback(async () => {
const loadId = activeLoadIdRef.current + 1;
activeLoadIdRef.current = loadId;
setIsRefreshing(true);
try {
const [statusResponse, sessionsResponse] = await Promise.all([
authenticatedFetch('/api/browser-use/status'),
authenticatedFetch('/api/browser-use/sessions'),
]);
const statusData = await readJson<{ data: BrowserUseStatus }>(statusResponse);
const sessionsData = await readJson<{ data: { sessions: BrowserUseSession[] } }>(sessionsResponse);
const nextSessions = sessionsData.data.sessions;
setStatus(statusData.data);
let nextData: Awaited<ReturnType<typeof fetchBrowserPanelData>>;
try {
nextData = await fetchBrowserPanelData();
} catch (error) {
if (loadId !== activeLoadIdRef.current) {
return;
}
await new Promise((resolve) => setTimeout(resolve, 350));
nextData = await fetchBrowserPanelData();
}
if (activeLoadIdRef.current !== loadId) {
return;
}
const nextSessions = nextData.sessions;
setStatus(nextData.status);
setSessions(nextSessions);
setSelectedSessionId((current) => (
current && nextSessions.some((session) => session.id === current)
setHasLoadedOnce(true);
let nextSelectedSessionId: string | null = null;
setSelectedSessionId((current) => {
nextSelectedSessionId = current && nextSessions.some((session) => session.id === current)
? current
: nextSessions[0]?.id || null
));
: nextSessions[0]?.id || null;
return nextSelectedSessionId;
});
browserUsePanelCache.set(cacheKey, {
status: nextData.status,
sessions: nextSessions,
selectedSessionId: nextSelectedSessionId,
updatedAt: Date.now(),
});
setError(null);
} catch (err) {
if (activeLoadIdRef.current !== loadId) {
return;
}
setHasLoadedOnce(true);
setError(err instanceof Error ? err.message : 'Failed to load Browser');
} finally {
setIsRefreshing(false);
if (activeLoadIdRef.current === loadId) {
setIsRefreshing(false);
}
}
}, []);
}, [cacheKey]);
useEffect(() => {
const cachedEntry = browserUsePanelCache.get(cacheKey);
if (!cachedEntry) return;
browserUsePanelCache.set(cacheKey, {
...cachedEntry,
selectedSessionId,
});
}, [cacheKey, selectedSessionId]);
useEffect(() => {
const cachedEntry = getFreshCacheEntry(cacheKey);
setStatus(cachedEntry?.status ?? null);
setSessions(cachedEntry?.sessions ?? []);
setSelectedSessionId(cachedEntry?.selectedSessionId || cachedEntry?.sessions[0]?.id || null);
setHasLoadedOnce(Boolean(cachedEntry));
setError(null);
activeLoadIdRef.current += 1;
}, [cacheKey]);
useEffect(() => {
if (!isVisible) return;
@@ -253,6 +362,10 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
<span>{formatRelativeTime(session.updatedAt)}</span>
<span className="truncate">- {formatAction(session.lastAction)}</span>
</div>
<div className="mt-2 flex flex-wrap gap-1.5 pl-3.5 text-[10px] text-muted-foreground">
<span className="rounded border border-border/70 bg-background/70 px-1.5 py-0.5">{getEngineLabel(session.backend)}</span>
<span className="rounded border border-border/70 bg-background/70 px-1.5 py-0.5">{session.profileName || 'Temporary'}</span>
</div>
</button>
);
};
@@ -270,9 +383,18 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
</div>
<p className="mt-1 max-w-xl text-sm leading-6 text-muted-foreground">
{status?.enabled
? 'Agent browser sessions appear here while an AI task is using Browser.'
: 'Enable Browser in settings to let agents open monitored browser sessions.'}
? 'When an agent opens a browser, you can watch the latest screenshot, take control in a new tab, or end the running session.'
: 'Enable Browser to let agents open websites, test flows, capture screenshots, and debug UI from a real page.'}
</p>
<a
href={BROWSER_USE_GUIDE_URL}
target="_blank"
rel="noopener noreferrer"
className="mt-2 inline-flex items-center gap-1.5 text-sm font-medium text-primary hover:underline"
>
Read the Browser guide
<ExternalLink className="h-3.5 w-3.5" />
</a>
</div>
</div>
@@ -312,10 +434,19 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
</div>
);
const renderLoadingState = () => (
<div className="flex min-h-0 flex-1 items-center justify-center p-6">
<div className="flex items-center gap-3 rounded-md border border-border bg-card/40 px-4 py-3 text-sm text-muted-foreground shadow-sm">
<Loader2 className="h-4 w-4 animate-spin text-primary" />
Loading browser sessions...
</div>
</div>
);
const renderBrowserSurface = (fullscreen = false) => (
<div className={cn('flex flex-1 items-center justify-center bg-neutral-950', fullscreen ? 'min-h-[80vh]' : 'min-h-[420px]')}>
{selectedSession?.screenshotDataUrl ? (
<div className="relative inline-block max-h-full">
<div className="group relative inline-block max-h-full">
<img
src={selectedSession.screenshotDataUrl}
alt="Browser session screenshot"
@@ -329,6 +460,18 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
<div className="absolute left-1/2 top-1/2 h-2 w-2 -translate-x-1/2 -translate-y-1/2 rounded-full bg-white" />
</div>
)}
{selectedSession?.viewerEmbedUrl && selectedSession.status === 'ready' && (
<button
type="button"
onClick={() => window.open(selectedSession.viewerUrl || selectedSession.viewerEmbedUrl || '', '_blank', 'noopener,noreferrer')}
className="absolute inset-0 flex items-center justify-center bg-black/0 opacity-0 transition focus-visible:bg-black/30 focus-visible:opacity-100 focus-visible:outline-none group-hover:bg-black/30 group-hover:opacity-100"
>
<span className="inline-flex items-center gap-2 rounded-md border border-white/20 bg-black/80 px-3 py-2 text-sm font-medium text-white shadow-lg">
<MousePointer2 className="h-4 w-4" />
Take control
</span>
</button>
)}
</div>
) : (
<div className="px-6 text-center">
@@ -350,10 +493,29 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
<Badge variant="outline" className={cn('text-[10px]', getRuntimeTone(status, isInstalling))}>
{runtimeLabel}
</Badge>
<Badge variant="outline" className="border-border bg-background text-[10px] text-muted-foreground">
{getEngineLabel(status?.backend)}
</Badge>
</div>
<p className="mt-0.5 text-xs text-muted-foreground">Monitor browser sessions opened by AI agents.</p>
<p className="mt-0.5 text-xs text-muted-foreground">Watch and manage browser sessions agents use to test real websites.</p>
{isBackgroundRefreshing && (
<div className="mt-1 flex items-center gap-1.5 text-xs text-muted-foreground">
<RefreshCw className="h-3 w-3 animate-spin" />
Refreshing sessions...
</div>
)}
</div>
<div className="flex items-center gap-1.5">
<Button
variant="ghost"
size="sm"
className="h-7 w-7 p-0"
onClick={() => window.open(BROWSER_USE_GUIDE_URL, '_blank', 'noopener,noreferrer')}
title="Open Browser guide"
aria-label="Open Browser guide"
>
<ExternalLink className="h-3.5 w-3.5" />
</Button>
{onShowSettings && (
<Button
variant="ghost"
@@ -425,7 +587,7 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
</div>
{sessions.length === 0 ? (
renderEmptyState()
isInitialLoading ? renderLoadingState() : renderEmptyState()
) : (
<div className="min-h-0 flex-1 overflow-auto bg-muted/20 p-4">
<div className="mx-auto flex min-h-[500px] max-w-7xl flex-col overflow-hidden rounded-md border border-border bg-background shadow-sm">
@@ -441,14 +603,32 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
<ExternalLink className="h-3.5 w-3.5 shrink-0" />
<span className="truncate">{selectedSession?.url || 'No page loaded'}</span>
</div>
<div className="mt-1 flex flex-wrap gap-1.5 text-[10px] text-muted-foreground">
<span className="rounded border border-border/70 bg-muted/30 px-1.5 py-0.5">{getEngineLabel(selectedSession?.backend || status?.backend)}</span>
<span className="rounded border border-border/70 bg-muted/30 px-1.5 py-0.5">Profile: {selectedSession?.profileName || 'Temporary'}</span>
<span className="rounded border border-border/70 bg-muted/30 px-1.5 py-0.5">Updated {formatRelativeTime(selectedSession?.updatedAt || null)}</span>
</div>
</div>
<div className="hidden text-xs text-muted-foreground md:block">
{formatAction(selectedSession?.lastAction || null)}
</div>
{selectedSession?.viewerUrl && selectedSession.status === 'ready' && (
<Button
variant="outline"
size="sm"
className="h-8"
onClick={() => window.open(selectedSession.viewerUrl || '', '_blank', 'noopener,noreferrer')}
title="Open live browser control in a new tab"
aria-label="Open live browser control in a new tab"
>
<MousePointer2 className="h-4 w-4" />
Take control
</Button>
)}
<Button variant="ghost" size="sm" className="h-8 w-8 p-0" onClick={() => setIsFullscreen(true)} disabled={!selectedSession?.screenshotDataUrl} title="Full screen" aria-label="Full screen">
<Expand className="h-4 w-4" />
</Button>
<Button variant="ghost" size="sm" className="h-8 w-8 p-0 lg:hidden" onClick={stopSession} disabled={isBusy || !selectedSession || selectedSession.status !== 'ready'} title="Stop session" aria-label="Stop session">
<Button variant="ghost" size="sm" className="h-8 w-8 p-0 lg:hidden" onClick={stopSession} disabled={isBusy || !selectedSession || selectedSession.status !== 'ready'} title="End session" aria-label="End session">
<Square className="h-4 w-4" />
</Button>
<Button variant="ghost" size="sm" className="h-8 w-8 p-0 lg:hidden" onClick={deleteSession} disabled={isBusy || !selectedSession} title="Delete session" aria-label="Delete session">
@@ -475,6 +655,11 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
<div className="min-h-0 flex-1 overflow-y-auto p-3">
{sessions.length > 0 ? (
<div className="space-y-2">{sessions.map(renderSessionItem)}</div>
) : isInitialLoading ? (
<div className="flex items-center justify-center gap-2 rounded-md border border-dashed border-border/70 px-3 py-8 text-center text-xs text-muted-foreground">
<Loader2 className="h-3.5 w-3.5 animate-spin" />
Loading sessions...
</div>
) : (
<div className="rounded-md border border-dashed border-border/70 px-3 py-8 text-center text-xs text-muted-foreground">
No agent browser sessions.
@@ -505,7 +690,7 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
<div className="mt-3 grid grid-cols-2 gap-2">
<Button variant="outline" size="sm" onClick={stopSession} disabled={isBusy || !selectedSession || selectedSession.status !== 'ready'}>
<Square className="h-4 w-4" />
Stop
End
</Button>
<Button variant="outline" size="sm" onClick={deleteSession} disabled={isBusy || !selectedSession}>
<Trash2 className="h-4 w-4" />

View File

@@ -1,12 +0,0 @@
import type { LLMProvider, ProviderModelOption } from '../../../types/app';
export const DEFAULT_EFFORT_VALUE = 'default';
export const FALLBACK_PROVIDER_EFFORT_VALUES: Partial<Record<LLMProvider, readonly string[]>> = {
claude: ['low', 'medium', 'high', 'xhigh', 'max'],
codex: ['low', 'medium', 'high', 'xhigh'],
};
export const toProviderEffortOptions = (
values: readonly string[],
): NonNullable<ProviderModelOption['effort']>['values'] => values.map((value) => ({ value }));

View File

@@ -34,11 +34,9 @@ interface UseChatComposerStateArgs {
provider: LLMProvider;
permissionMode: PermissionMode | string;
cyclePermissionMode: () => void;
resolvePermissionModeForProvider: (provider: LLMProvider, requestedMode: PermissionMode | string) => PermissionMode;
cursorModel: string;
claudeModel: string;
codexModel: string;
currentProviderEffort: string;
geminiModel: string;
opencodeModel: string;
isLoading: boolean;
@@ -170,11 +168,9 @@ export function useChatComposerState({
provider,
permissionMode,
cyclePermissionMode,
resolvePermissionModeForProvider,
cursorModel,
claudeModel,
codexModel,
currentProviderEffort,
geminiModel,
opencodeModel,
isLoading,
@@ -732,9 +728,8 @@ export function useChatComposerState({
: provider === 'gemini'
? geminiModel
: provider === 'opencode'
? opencodeModel
: claudeModel;
const effort = currentProviderEffort;
? opencodeModel
: claudeModel;
// One message shape for every provider. The backend resolves the
// provider, project path, and provider-native resume id from the
@@ -745,8 +740,9 @@ export function useChatComposerState({
content: messageContent,
options: {
model,
effort,
permissionMode: resolvePermissionModeForProvider(provider, permissionMode),
// Codex has no plan mode; downgrade rather than sending an
// unsupported value to its runtime.
permissionMode: provider === 'codex' && permissionMode === 'plan' ? 'default' : permissionMode,
toolsSettings,
skipPermissions: toolsSettings?.skipPermissions || false,
sessionSummary,
@@ -773,7 +769,6 @@ export function useChatComposerState({
attachedImages,
claudeModel,
codexModel,
currentProviderEffort,
currentSessionId,
cursorModel,
executeCommand,
@@ -784,7 +779,6 @@ export function useChatComposerState({
onSessionEstablished,
permissionMode,
provider,
resolvePermissionModeForProvider,
resetCommandMenuState,
scrollToBottom,
selectedProject,

View File

@@ -1,31 +1,22 @@
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { useCallback, useEffect, useRef, useState } from 'react';
import { authenticatedFetch } from '../../../utils/api';
import type { PendingPermissionRequest, PermissionMode } from '../types/types';
import type {
ProjectSession,
LLMProvider,
Project,
ProviderModelOption,
ProviderModelsCacheInfo,
ProviderModelsDefinition,
} from '../../../types/app';
import {
DEFAULT_EFFORT_VALUE,
FALLBACK_PROVIDER_EFFORT_VALUES,
toProviderEffortOptions,
} from '../constants/providerEffort';
const FALLBACK_DEFAULT_MODEL: Record<LLMProvider, string> = {
claude: 'default',
claude: 'opus',
cursor: 'gpt-5.3-codex',
codex: 'gpt-5.4',
gemini: 'gemini-3.1-pro-preview',
opencode: 'anthropic/claude-sonnet-4-5',
};
const PROVIDERS: LLMProvider[] = ['claude', 'cursor', 'codex', 'gemini', 'opencode'];
/**
* Fallback permission-mode matrix used only until the backend capability
* matrix (`GET /api/providers/capabilities`) has loaded. The backend is the
@@ -48,7 +39,6 @@ type ProviderCapabilities = {
supportsAbort: boolean;
supportsPermissionRequests: boolean;
supportsTokenUsage: boolean;
supportsEffort?: boolean;
};
type ProviderCapabilitiesApiResponse = {
@@ -82,7 +72,7 @@ type ChangeActiveModelApiResponse = {
};
};
export function useChatProviderState({ selectedSession, selectedProject: _selectedProject }: UseChatProviderStateArgs) {
export function useChatProviderState({ selectedSession, selectedProject }: UseChatProviderStateArgs) {
const [permissionMode, setPermissionMode] = useState<PermissionMode>('default');
const [pendingPermissionRequests, setPendingPermissionRequests] = useState<PendingPermissionRequest[]>([]);
const [provider, setProvider] = useState<LLMProvider>(() => {
@@ -97,12 +87,6 @@ export function useChatProviderState({ selectedSession, selectedProject: _select
const [codexModel, setCodexModel] = useState<string>(() => {
return localStorage.getItem('codex-model') || FALLBACK_DEFAULT_MODEL.codex;
});
const [providerEfforts, setProviderEfforts] = useState<Partial<Record<LLMProvider, string>>>(() => {
return PROVIDERS.reduce<Partial<Record<LLMProvider, string>>>((acc, targetProvider) => {
acc[targetProvider] = localStorage.getItem(`${targetProvider}-effort`) || DEFAULT_EFFORT_VALUE;
return acc;
}, {});
});
const [geminiModel, setGeminiModel] = useState<string>(() => {
return localStorage.getItem('gemini-model') || FALLBACK_DEFAULT_MODEL.gemini;
});
@@ -161,16 +145,8 @@ export function useChatProviderState({ selectedSession, selectedProject: _select
localStorage.setItem('opencode-model', model);
}, []);
const setStoredProviderEffort = useCallback((targetProvider: LLMProvider, effort: string) => {
setProviderEfforts((previous) => (
previous[targetProvider] === effort
? previous
: { ...previous, [targetProvider]: effort }
));
localStorage.setItem(`${targetProvider}-effort`, effort);
}, []);
const loadProviderModels = useCallback(async (options: { bypassCache?: boolean } = {}) => {
const providers: LLMProvider[] = ['claude', 'cursor', 'codex', 'gemini', 'opencode'];
const requestId = providerModelsRequestIdRef.current + 1;
providerModelsRequestIdRef.current = requestId;
const isHardRefresh = options.bypassCache === true;
@@ -183,7 +159,7 @@ export function useChatProviderState({ selectedSession, selectedProject: _select
try {
const results = await Promise.all(
PROVIDERS.map(async (p) => {
providers.map(async (p) => {
const params = new URLSearchParams();
if (options.bypassCache) {
params.set('bypassCache', 'true');
@@ -207,7 +183,7 @@ export function useChatProviderState({ selectedSession, selectedProject: _select
const nextCatalog: Partial<Record<LLMProvider, ProviderModelsDefinition>> = {};
const nextCacheCatalog: Partial<Record<LLMProvider, ProviderModelsCacheInfo>> = {};
PROVIDERS.forEach((p, i) => {
providers.forEach((p, i) => {
const entry = results[i];
if (!entry) {
return;
@@ -268,23 +244,6 @@ export function useChatProviderState({ selectedSession, selectedProject: _select
return FALLBACK_PERMISSION_MODES[targetProvider] ?? ['default'];
}, [providerCapabilities]);
const getDefaultPermissionModeForProvider = useCallback((targetProvider: LLMProvider): PermissionMode => {
const modes = getPermissionModesForProvider(targetProvider);
const capabilityDefault = providerCapabilities?.[targetProvider]?.defaultPermissionMode as PermissionMode | undefined;
if (capabilityDefault && modes.includes(capabilityDefault)) {
return capabilityDefault;
}
return modes[0] ?? 'default';
}, [getPermissionModesForProvider, providerCapabilities]);
const getSupportsEffortForProvider = useCallback((targetProvider: LLMProvider): boolean => {
const capabilitySupport = providerCapabilities?.[targetProvider]?.supportsEffort;
if (typeof capabilitySupport === 'boolean') {
return capabilitySupport;
}
return Boolean(FALLBACK_PROVIDER_EFFORT_VALUES[targetProvider]?.length);
}, [providerCapabilities]);
const pickStoredOrCurrent = (
storageKey: string,
current: string,
@@ -300,70 +259,6 @@ export function useChatProviderState({ selectedSession, selectedProject: _select
return def.DEFAULT;
};
const getModelOption = useCallback((
targetProvider: LLMProvider,
model: string,
): ProviderModelOption | null => {
const definition = providerModelCatalog[targetProvider];
if (!definition) {
return null;
}
return definition.OPTIONS.find((option) => option.value === model) ?? null;
}, [providerModelCatalog]);
const getEffortOptionsForModel = useCallback((
targetProvider: LLMProvider,
model: string,
): NonNullable<ProviderModelOption['effort']>['values'] => {
if (!getSupportsEffortForProvider(targetProvider)) {
return [];
}
const option = getModelOption(targetProvider, model);
if (option) {
return option.effort?.values ?? [];
}
return toProviderEffortOptions(FALLBACK_PROVIDER_EFFORT_VALUES[targetProvider] ?? []);
}, [getModelOption, getSupportsEffortForProvider]);
const getAllowedEffortValues = useCallback((
targetProvider: LLMProvider,
model: string,
): string[] => (
getEffortOptionsForModel(targetProvider, model).map((value) => value.value)
), [getEffortOptionsForModel]);
const reconcileStoredEffort = useCallback((
targetProvider: LLMProvider,
model: string,
currentEffort: string,
): string => {
const allowedValues = getAllowedEffortValues(targetProvider, model);
if (allowedValues.length === 0) {
return DEFAULT_EFFORT_VALUE;
}
if (currentEffort === DEFAULT_EFFORT_VALUE || !currentEffort) {
return DEFAULT_EFFORT_VALUE;
}
if (allowedValues.includes(currentEffort)) {
return currentEffort;
}
return DEFAULT_EFFORT_VALUE;
}, [getAllowedEffortValues]);
const providerModels = useMemo<Record<LLMProvider, string>>(() => ({
claude: claudeModel,
cursor: cursorModel,
codex: codexModel,
gemini: geminiModel,
opencode: opencodeModel,
}), [claudeModel, cursorModel, codexModel, geminiModel, opencodeModel]);
useEffect(() => {
const claude = providerModelCatalog.claude;
if (claude) {
@@ -429,27 +324,6 @@ export function useChatProviderState({ selectedSession, selectedProject: _select
}
}, [providerModelCatalog.opencode, opencodeModel]);
useEffect(() => {
const nextEfforts: Partial<Record<LLMProvider, string>> = {};
let hasUpdates = false;
for (const targetProvider of PROVIDERS) {
const currentEffort = providerEfforts[targetProvider] ?? DEFAULT_EFFORT_VALUE;
const nextEffort = reconcileStoredEffort(targetProvider, providerModels[targetProvider], currentEffort);
if (nextEffort === currentEffort) {
continue;
}
nextEfforts[targetProvider] = nextEffort;
localStorage.setItem(`${targetProvider}-effort`, nextEffort);
hasUpdates = true;
}
if (hasUpdates) {
setProviderEfforts((previous) => ({ ...previous, ...nextEfforts }));
}
}, [providerEfforts, providerModels, reconcileStoredEffort]);
useEffect(() => {
if (!selectedSession?.id) {
return;
@@ -457,12 +331,8 @@ export function useChatProviderState({ selectedSession, selectedProject: _select
const savedMode = localStorage.getItem(`permissionMode-${selectedSession.id}`) as PermissionMode | null;
const validModes = getPermissionModesForProvider(provider);
setPermissionMode(
savedMode && validModes.includes(savedMode)
? savedMode
: getDefaultPermissionModeForProvider(provider),
);
}, [selectedSession?.id, provider, getDefaultPermissionModeForProvider, getPermissionModesForProvider]);
setPermissionMode(savedMode && validModes.includes(savedMode) ? savedMode : 'default');
}, [selectedSession?.id, provider, getPermissionModesForProvider]);
useEffect(() => {
if (!selectedSession?.__provider || selectedSession.__provider === provider) {
@@ -516,16 +386,6 @@ export function useChatProviderState({ selectedSession, selectedProject: _select
}
}, [permissionMode, provider, selectedSession?.id, getPermissionModesForProvider]);
const resolvePermissionModeForProvider = useCallback((
targetProvider: LLMProvider,
requestedMode: PermissionMode | string,
): PermissionMode => {
const validModes = getPermissionModesForProvider(targetProvider);
return validModes.includes(requestedMode as PermissionMode)
? requestedMode as PermissionMode
: getDefaultPermissionModeForProvider(targetProvider);
}, [getDefaultPermissionModeForProvider, getPermissionModesForProvider]);
const selectProviderModel = useCallback(async (
targetProvider: LLMProvider,
model: string,
@@ -561,17 +421,6 @@ export function useChatProviderState({ selectedSession, selectedProject: _select
};
}, [setStoredProviderModel]);
const currentProviderEffortOptions = useMemo(() => {
return getEffortOptionsForModel(provider, providerModels[provider]);
}, [getEffortOptionsForModel, provider, providerModels]);
const currentProviderEffort = useMemo(() => {
return reconcileStoredEffort(
provider,
providerModels[provider],
providerEfforts[provider] ?? DEFAULT_EFFORT_VALUE,
);
}, [provider, providerEfforts, providerModels, reconcileStoredEffort]);
return {
provider,
setProvider,
@@ -581,8 +430,6 @@ export function useChatProviderState({ selectedSession, selectedProject: _select
setClaudeModel,
codexModel,
setCodexModel,
currentProviderEffort,
currentProviderEffortOptions,
geminiModel,
setGeminiModel,
opencodeModel,
@@ -598,7 +445,5 @@ export function useChatProviderState({ selectedSession, selectedProject: _select
providerModelsRefreshing,
hardRefreshProviderModels: () => loadProviderModels({ bypassCache: true }),
selectProviderModel,
setStoredProviderEffort,
resolvePermissionModeForProvider,
};
}

View File

@@ -17,6 +17,7 @@ import ChatMessagesPane from './subcomponents/ChatMessagesPane';
import ChatComposer from './subcomponents/ChatComposer';
import CommandResultModal from './subcomponents/CommandResultModal';
function ChatInterface({
selectedProject,
selectedSession,
@@ -69,8 +70,6 @@ function ChatInterface({
setClaudeModel,
codexModel,
setCodexModel,
currentProviderEffort,
currentProviderEffortOptions,
geminiModel,
setGeminiModel,
opencodeModel,
@@ -85,8 +84,6 @@ function ChatInterface({
providerModelsRefreshing,
hardRefreshProviderModels,
selectProviderModel,
setStoredProviderEffort,
resolvePermissionModeForProvider,
} = useChatProviderState({
selectedSession,
selectedProject,
@@ -200,7 +197,6 @@ function ChatInterface({
cursorModel,
claudeModel,
codexModel,
currentProviderEffort,
geminiModel,
opencodeModel,
isLoading: isProcessing,
@@ -217,7 +213,6 @@ function ChatInterface({
addMessage,
setIsUserScrolledUp,
setPendingPermissionRequests,
resolvePermissionModeForProvider,
});
// On WebSocket reconnect, re-fetch the current session's messages from the
@@ -388,9 +383,6 @@ function ChatInterface({
onAbortSession={handleAbortSession}
permissionMode={permissionMode}
onModeSwitch={cyclePermissionMode}
effort={currentProviderEffort}
availableEffortOptions={currentProviderEffortOptions}
onSelectEffort={(nextEffort) => setStoredProviderEffort(provider, nextEffort)}
tokenBudget={tokenBudget}
onShowTokenUsage={showCostModal}
slashCommandsCount={slashCommandsCount}

View File

@@ -1,5 +1,6 @@
import { useTranslation } from 'react-i18next';
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { useMemo } from 'react';
import { useCallback, useEffect, useRef, useState } from 'react';
import type {
ChangeEvent,
ClipboardEvent,
@@ -10,13 +11,12 @@ import type {
RefObject,
TouchEvent,
} from 'react';
import { ImageIcon, MessageSquareIcon, XIcon, ArrowDownIcon, Loader2, ChevronDown, Check } from 'lucide-react';
import { ImageIcon, MessageSquareIcon, XIcon, Loader2 } from 'lucide-react';
import { useVoiceInput } from '../../hooks/useVoiceInput';
import { useVoiceAvailable } from '../../hooks/useVoiceAvailable';
import type { SessionActivity } from '../../../../hooks/useSessionProtection';
import type { PendingPermissionRequest, PermissionMode } from '../../types/types';
import type { ProviderModelOption } from '../../../../types/app';
import {
PromptInput,
PromptInputHeader,
@@ -62,9 +62,6 @@ interface ChatComposerProps {
onAbortSession: () => void;
permissionMode: PermissionMode | string;
onModeSwitch: () => void;
effort: string;
availableEffortOptions: NonNullable<ProviderModelOption['effort']>['values'];
onSelectEffort: (effort: string) => void;
tokenBudget: Record<string, unknown> | null;
onShowTokenUsage: () => void;
slashCommandsCount: number;
@@ -117,9 +114,6 @@ export default function ChatComposer({
onAbortSession,
permissionMode,
onModeSwitch,
effort,
availableEffortOptions,
onSelectEffort,
tokenBudget,
onShowTokenUsage,
slashCommandsCount,
@@ -173,7 +167,7 @@ export default function ChatComposer({
left: textareaRect ? textareaRect.left : 16,
bottom: textareaRect ? window.innerHeight - textareaRect.top + 8 : 90,
};
}, [isCommandMenuOpen, textareaRef]);
}, [input, isCommandMenuOpen, textareaRef]);
// Voice state is hosted here (not in the mic button) so the main Send button can stop
// recording and send the transcript in one tap, the way the mic button drops it in the box.
@@ -195,39 +189,6 @@ export default function ChatComposer({
);
const isRecording = voiceState === 'recording';
const isTranscribing = voiceState === 'transcribing';
const [isEffortDropdownOpen, setIsEffortDropdownOpen] = useState(false);
const effortDropdownRef = useRef<HTMLDivElement | null>(null);
const effortOptions = useMemo(
() => [{ value: 'default' }, ...availableEffortOptions],
[availableEffortOptions],
);
const selectedEffortLabel = effort === 'default' ? 'Default' : effort;
useEffect(() => {
if (!isEffortDropdownOpen) return;
const handlePointerDown = (event: PointerEvent) => {
if (!effortDropdownRef.current?.contains(event.target as Node)) {
setIsEffortDropdownOpen(false);
}
};
const handleKeyDown = (event: globalThis.KeyboardEvent) => {
if (event.key === 'Escape') {
event.preventDefault();
event.stopPropagation();
setIsEffortDropdownOpen(false);
}
};
document.addEventListener('pointerdown', handlePointerDown);
window.addEventListener('keydown', handleKeyDown, { capture: true });
return () => {
document.removeEventListener('pointerdown', handlePointerDown);
window.removeEventListener('keydown', handleKeyDown, { capture: true });
};
}, [isEffortDropdownOpen]);
// Detect if the AskUserQuestion interactive panel is active
const hasQuestionPanel = pendingPermissionRequests.some(
@@ -415,55 +376,6 @@ export default function ChatComposer({
</div>
</button>
{availableEffortOptions.length > 0 && (
<div ref={effortDropdownRef} className="relative">
<button
type="button"
onClick={() => setIsEffortDropdownOpen((current) => !current)}
className="flex h-8 items-center gap-1.5 rounded-lg border border-border/60 bg-muted/40 px-2 text-xs font-medium text-foreground transition-all duration-200 hover:bg-muted"
aria-haspopup="menu"
aria-expanded={isEffortDropdownOpen}
aria-label="Select reasoning effort"
title="Select reasoning effort"
>
<span className="hidden text-[11px] text-muted-foreground sm:inline">Effort</span>
<span className="max-w-16 truncate capitalize sm:max-w-20">{selectedEffortLabel}</span>
<ChevronDown className={`h-3 w-3 text-muted-foreground transition-transform ${isEffortDropdownOpen ? 'rotate-180' : ''}`} />
</button>
{isEffortDropdownOpen && (
<div className="absolute bottom-full left-0 z-50 mb-2 min-w-36 overflow-hidden rounded-lg border border-border bg-card p-1 shadow-lg">
{effortOptions.map((option) => {
const isSelected = option.value === effort;
const label = option.value === 'default' ? 'Default' : option.value;
return (
<button
key={option.value}
type="button"
role="menuitemradio"
aria-checked={isSelected}
onClick={() => {
onSelectEffort(option.value);
setIsEffortDropdownOpen(false);
}}
className={`flex w-full items-center gap-2 rounded px-2 py-1.5 text-left text-xs capitalize transition-colors ${
isSelected
? 'bg-accent text-foreground'
: 'text-muted-foreground hover:bg-accent/70 hover:text-foreground'
}`}
>
<span className="flex h-3 w-3 items-center justify-center">
{isSelected && <Check className="h-3 w-3 text-primary" />}
</span>
<span>{label}</span>
</button>
);
})}
</div>
)}
</div>
)}
<TokenUsageSummary usage={tokenBudget} onClick={onShowTokenUsage} />
<PromptInputButton

View File

@@ -263,6 +263,7 @@ function ModelsContent({
const availableModels = Array.isArray(data?.availableModels) ? data.availableModels : [];
return availableModels.map((model) => ({ value: model, label: model }));
}, [data, liveDefinition]);
const filteredOptions = useMemo(() => {
const normalized = query.trim().toLowerCase();
if (!normalized) {

View File

@@ -207,7 +207,11 @@ function MainContent({
{shouldShowBrowserTab && activeTab === 'browser' && (
<div className="h-full overflow-hidden">
<BrowserUsePanel isVisible={activeTab === 'browser'} onShowSettings={onShowSettings} />
<BrowserUsePanel
isVisible={activeTab === 'browser'}
projectId={selectedProject.projectId}
onShowSettings={onShowSettings}
/>
</div>
)}

View File

@@ -73,7 +73,15 @@ export default function MainContentTitle({
<h2 title={getSessionTitle(selectedSession)} className="truncate text-sm font-semibold leading-tight text-foreground">
{getSessionTitle(selectedSession)}
</h2>
<div className="truncate text-[11px] leading-tight text-muted-foreground">{selectedProject.displayName}</div>
<div className="flex min-w-0 items-center gap-2 text-[11px] leading-tight text-muted-foreground">
<span className="min-w-0 truncate">{selectedProject.displayName}</span>
<span
className="hidden min-w-0 max-w-[45%] flex-shrink truncate border-l border-border/60 pl-2 font-mono text-[10px] sm:block"
title={selectedSession.id}
>
{selectedSession.id}
</span>
</div>
</div>
) : showChatNewSession ? (
<div className="min-w-0">

View File

@@ -1,22 +1,32 @@
import { useCallback, useEffect, useState } from 'react';
import { Download, Loader2 } from 'lucide-react';
import { Download, ExternalLink, Eye, Loader2, Zap } from 'lucide-react';
import { Button } from '../../../../../shared/view/ui';
import { Button, Input } from '../../../../../shared/view/ui';
import { authenticatedFetch } from '../../../../../utils/api';
import SettingsCard from '../../SettingsCard';
import SettingsRow from '../../SettingsRow';
import SettingsSection from '../../SettingsSection';
import SettingsToggle from '../../SettingsToggle';
const BROWSER_USE_GUIDE_URL = 'https://cloudcli.ai/docs/browser-use';
type BrowserUseSettings = {
enabled: boolean;
persistSessions: boolean;
defaultProfileName: string;
browserBackend: 'playwright' | 'camoufox-vnc';
};
type BrowserUseStatus = {
enabled: boolean;
available: boolean;
backend: 'playwright' | 'camoufox-vnc';
browserBackend: 'playwright' | 'camoufox-vnc';
playwrightInstalled: boolean;
chromiumInstalled: boolean;
camoufoxInstalled: boolean;
noVncInstalled: boolean;
x11vncInstalled: boolean;
installInProgress: boolean;
message: string;
};
@@ -32,16 +42,20 @@ async function readJson<T>(response: Response): Promise<T> {
export default function BrowserUseSettingsTab() {
const [settings, setSettings] = useState<BrowserUseSettings | null>(null);
const [status, setStatus] = useState<BrowserUseStatus | null>(null);
const [hasLoadedSettings, setHasLoadedSettings] = useState(false);
const [isSettingsLoading, setIsSettingsLoading] = useState(true);
const [isStatusLoading, setIsStatusLoading] = useState(true);
const [isSaving, setIsSaving] = useState(false);
const [isInstalling, setIsInstalling] = useState(false);
const [error, setError] = useState<string | null>(null);
const [profileNameDraft, setProfileNameDraft] = useState('default');
const loadSettings = useCallback(async () => {
const settingsResponse = await authenticatedFetch('/api/browser-use/settings');
const settingsData = await readJson<{ data: { settings: BrowserUseSettings } }>(settingsResponse);
setSettings(settingsData.data.settings);
setHasLoadedSettings(true);
setProfileNameDraft(settingsData.data.settings.defaultProfileName || 'default');
}, []);
const loadStatus = useCallback(async () => {
@@ -52,6 +66,7 @@ export default function BrowserUseSettingsTab() {
useEffect(() => {
setError(null);
setHasLoadedSettings(false);
setIsSettingsLoading(true);
setIsStatusLoading(true);
@@ -74,6 +89,7 @@ export default function BrowserUseSettingsTab() {
});
const data = await readJson<{ data: { settings: BrowserUseSettings } }>(response);
setSettings(data.data.settings);
setHasLoadedSettings(true);
window.dispatchEvent(new Event('browserUseSettingsChanged'));
setIsStatusLoading(true);
await loadStatus();
@@ -101,8 +117,21 @@ export default function BrowserUseSettingsTab() {
}
};
const saveProfileName = async () => {
const nextName = profileNameDraft.trim() || 'default';
setProfileNameDraft(nextName);
if (nextName === settings?.defaultProfileName) {
return;
}
await updateSettings({ defaultProfileName: nextName });
};
const browserEnabled = settings?.enabled === true;
const needsBrowserBinaries = Boolean(browserEnabled && status && (!status.playwrightInstalled || !status.chromiumInstalled));
const browserDisabled = hasLoadedSettings && settings?.enabled === false;
const persistSessions = settings?.persistSessions === true;
const selectedBackend = settings?.browserBackend || 'playwright';
const effectiveBackend = status?.backend || 'playwright';
const needsBrowserBinaries = Boolean(browserEnabled && status && !status.available);
const runtimeLabel = (installed?: boolean) => {
if (isStatusLoading && !status) {
return 'checking...';
@@ -114,33 +143,165 @@ export default function BrowserUseSettingsTab() {
<div className="space-y-8">
<SettingsSection
title="Browser"
description="Allow agents to create guarded Playwright browser sessions that you can monitor from the Browser tab."
description="Give coding agents a working browser so they can open websites, test flows, capture screenshots, and help debug what users actually see."
>
<SettingsCard divided>
<SettingsRow
label="Enable Browser"
description="Registers Browser for supported agents. Agents can create browser sessions; you can watch, stop, and delete them."
label="Give Agents Browser Access"
description="Let agents use a browser during coding tasks while you can watch live sessions, open them in a tab, and stop them at any time."
>
{isSettingsLoading && !settings ? (
{isSettingsLoading && !hasLoadedSettings ? (
<Loader2 className="h-4 w-4 animate-spin text-muted-foreground" />
) : (
) : hasLoadedSettings ? (
<SettingsToggle
checked={browserEnabled}
onChange={(value) => void updateSettings({ enabled: value })}
ariaLabel="Enable Browser"
ariaLabel="Give Agents Browser Access"
disabled={isSaving}
/>
) : (
<span className="text-sm text-muted-foreground">Unavailable</span>
)}
</SettingsRow>
{browserDisabled && (
<div className="px-4 py-4">
<a
href={BROWSER_USE_GUIDE_URL}
target="_blank"
rel="noopener noreferrer"
className="inline-flex items-center gap-1.5 text-sm font-medium text-primary hover:underline"
>
Read the Browser guide
<ExternalLink className="h-3.5 w-3.5" />
</a>
</div>
)}
{error && (
<div className="px-4 py-4">
<div className="rounded-md border border-red-200 bg-red-50 px-3 py-2 text-sm text-red-700 dark:border-red-900/50 dark:bg-red-950/30 dark:text-red-200">
{error}
</div>
</div>
)}
{browserEnabled && (
<>
<div className="space-y-3 px-4 py-4">
<div className="min-w-0">
<div className="text-sm font-medium text-foreground">Browser Engine</div>
<div className="mt-0.5 text-sm text-muted-foreground">
Pick the kind of browser experience agents should use for new sessions.
</div>
</div>
<div className="grid gap-2 sm:grid-cols-2">
{([
{
value: 'playwright' as const,
label: 'Playwright',
description: 'Best for quick checks, screenshots, and automated page interaction when no manual login is needed.',
icon: Zap,
},
{
value: 'camoufox-vnc' as const,
label: 'Camoufox + noVNC',
description: 'Best when a person may need to log in, approve a step, or watch the browser session live.',
icon: Eye,
},
]).map((option) => {
const Icon = option.icon;
const selected = selectedBackend === option.value;
return (
<button
key={option.value}
type="button"
onClick={() => void updateSettings({ browserBackend: option.value })}
disabled={isSaving || isSettingsLoading}
className={[
'group flex min-h-[88px] items-start gap-3 rounded-lg border px-3 py-3 text-left transition-colors',
'focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 focus-visible:ring-offset-background',
selected
? 'border-primary bg-primary/5 text-foreground shadow-sm'
: 'border-border bg-background hover:border-foreground/20 hover:bg-muted/40',
(isSaving || isSettingsLoading) ? 'cursor-not-allowed opacity-60' : '',
].join(' ')}
aria-pressed={selected}
>
<span className={[
'mt-0.5 flex h-8 w-8 flex-shrink-0 items-center justify-center rounded-md border',
selected ? 'border-primary/30 bg-primary/10 text-primary' : 'border-border bg-muted/40 text-muted-foreground',
].join(' ')}
>
<Icon className="h-4 w-4" />
</span>
<span className="min-w-0">
<span className="block text-sm font-medium">{option.label}</span>
<span className="mt-1 block text-xs leading-relaxed text-muted-foreground">{option.description}</span>
</span>
</button>
);
})}
</div>
</div>
<SettingsRow
label="Remember Browser Logins"
description="Keep cookies and site storage in a named profile so agents can reuse signed-in sessions instead of starting from scratch."
>
{isSettingsLoading && !settings ? (
<Loader2 className="h-4 w-4 animate-spin text-muted-foreground" />
) : (
<SettingsToggle
checked={persistSessions}
onChange={(value) => void updateSettings({ persistSessions: value })}
ariaLabel="Remember Browser Logins"
disabled={isSaving}
/>
)}
</SettingsRow>
{persistSessions && (
<SettingsRow
label="Default Browser Profile"
description="New browser sessions use this profile by default, so saved logins stay tied to a predictable workspace."
>
<Input
value={profileNameDraft}
onChange={(event) => setProfileNameDraft(event.target.value)}
onBlur={() => void saveProfileName()}
onKeyDown={(event) => {
if (event.key === 'Enter') {
event.currentTarget.blur();
}
}}
disabled={isSaving || isSettingsLoading}
className="w-40"
aria-label="Default Browser Profile"
/>
</SettingsRow>
)}
</>
)}
{browserEnabled && (
<div className="space-y-4 px-4 py-4">
<div className="flex flex-wrap gap-2 text-xs text-muted-foreground">
<span className="rounded-md border border-border px-2 py-1">
Backend: {effectiveBackend === 'camoufox-vnc' ? 'Camoufox + noVNC' : 'Playwright'}
</span>
<span className="rounded-md border border-border px-2 py-1">
Playwright: {runtimeLabel(status?.playwrightInstalled)}
</span>
<span className="rounded-md border border-border px-2 py-1">
Chromium: {runtimeLabel(status?.chromiumInstalled)}
</span>
<span className="rounded-md border border-border px-2 py-1">
Camoufox: {runtimeLabel(status?.camoufoxInstalled)}
</span>
<span className="rounded-md border border-border px-2 py-1">
noVNC: {runtimeLabel(status?.noVncInstalled)}
</span>
<span className="rounded-md border border-border px-2 py-1">
Status: {isStatusLoading && !status ? 'checking...' : status?.available ? 'ready' : browserEnabled ? 'setup required' : 'disabled'}
</span>
@@ -172,12 +333,17 @@ export default function BrowserUseSettingsTab() {
</div>
)}
{error && (
<div className="rounded-md border border-red-200 bg-red-50 px-3 py-2 text-sm text-red-700 dark:border-red-900/50 dark:bg-red-950/30 dark:text-red-200">
{error}
</div>
)}
<a
href={BROWSER_USE_GUIDE_URL}
target="_blank"
rel="noopener noreferrer"
className="inline-flex items-center gap-1.5 text-sm font-medium text-primary hover:underline"
>
Read the Browser guide
<ExternalLink className="h-3.5 w-3.5" />
</a>
</div>
)}
</SettingsCard>
</SettingsSection>
</div>

View File

@@ -4,13 +4,6 @@ export type ProviderModelOption = {
value: string;
label: string;
description?: string;
effort?: {
default?: string;
values: {
value: string;
description?: string;
}[];
};
};
export type ProviderModelsDefinition = {