Compare commits

...

88 Commits

Author SHA1 Message Date
Simos Mikelatos
d5a1467d03 Merge branch 'main' into camoufox-novnc-browser-use 2026-07-10 10:00:54 +02:00
Simos Mikelatos
b389ecb24b feat(browser-use): label Camoufox mode as co-browse 2026-07-09 20:23:31 +00:00
Simos Mikelatos
1dfca21558 fix(browser-use): support visible Camoufox on desktop platforms 2026-07-09 20:14:49 +00:00
Simos Mikelatos
f7171be383 fix(browser-use): allocate visible displays atomically 2026-07-09 19:51:41 +00:00
LeChristopher Blackwell
5884573a69 fix: validate X-Refreshed-Token before storing it as the auth token (#971)
* fix: validate X-Refreshed-Token before storing it as the auth token

authenticatedFetch stored any X-Refreshed-Token response header value
straight into localStorage. A malformed or injected header value would
silently overwrite the stored auth token, breaking every subsequent
authenticated request. Only accept values that match the JWT shape the
server actually issues (three base64url segments).

* fix: apply refreshed-token validation to file upload path

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GYpHw7VYW1d2HwfL9EZKx4

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 15:38:23 +03:00
Simos Mikelatos
123d244a51 fix: harden docker cloudcli install 2026-07-08 09:22:10 +00:00
Haile
bdf1193c43 Merge branch 'main' into camoufox-novnc-browser-use 2026-07-03 23:34:36 +03:00
Simos Mikelatos
9bcb9436a1 ci: drop redundant desktop-macos-release workflow (consolidated on main)
Main consolidated per-OS desktop release workflows into desktop-release.yml
(d882f80). This standalone macOS workflow survived on the branch via a
modify/delete merge conflict and duplicates the build-macos job now in
desktop-release.yml.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 16:03:39 +00:00
Simos Mikelatos
19e4bf9d80 chore: remove computer use (re-applied after main merge)
The merge of main into this branch (0907d87) hit a modify/delete conflict
for the computer-use module — main removed it in 6761f31 while this branch
had extended it — and the conflict was resolved by keeping the branch's
copies, silently resurrecting the whole feature.

This re-applies 6761f31's removal on top of the browser-use work:
- deletes the computer-use server module, client panel, settings tab,
  desktop-agent websocket service, electron computerAgent, and the
  computer-semantics build scripts
- strips computer-use wiring from shared files (server/index.js, electron
  launcher/main, MainContent, settings, i18n) while preserving browser-use
- removes src/constants/featureFlags.ts (existed only for the CU menu flag)

browser-use (camoufox/noVNC) is untouched. typecheck passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 15:54:22 +00:00
Haile
a34249497e Merge branch 'main' into camoufox-novnc-browser-use 2026-07-01 16:57:16 +03:00
Simos Mikelatos
0907d873f6 Merge remote-tracking branch 'origin/main' into camoufox-novnc-browser-use
# Conflicts:
#	.github/workflows/release.yml
#	electron/desktopWindow.js
#	electron/launcher/launcher.js
#	electron/main.js
#	electron/preload.cjs
#	package.json
#	scripts/release/prepare-desktop-app.js
#	server/modules/websocket/services/websocket-server.service.ts
#	src/components/main-content/view/MainContent.tsx
#	src/components/main-content/view/subcomponents/MainContentTitle.tsx
2026-06-29 12:34:48 +00:00
Simos Mikelatos
ec437072ad Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	server/index.js
2026-06-29 07:43:33 +00:00
Haile
54f4d8aa36 Chat & sidebar UX improvements (#929) 2026-06-29 09:27:04 +02:00
Simos Mikelatos
261690935f Merge pull request #891 from siteboon/electron-app 2026-06-29 09:07:57 +02:00
Simos Mikelatos
46ba8e56b4 Potential fix for pull request finding 'Useless assignment to local variable'
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
2026-06-26 16:14:44 +02:00
Simos Mikelatos
a0899a252e Merge branch 'main' into electron-app 2026-06-26 16:09:19 +02:00
Simos Mikelatos
fff89e6132 Merge branch 'main' into camoufox-novnc-browser-use 2026-06-26 16:07:22 +02:00
Simos Mikelatos
3bc2c777a3 fix: await desktop auth token lookup 2026-06-26 10:33:10 +00:00
Simos Mikelatos
63f3c3941d feat: add desktop notifications and skills updates 2026-06-26 10:25:47 +00:00
Simos Mikelatos
e6c6f89dda Merge branch 'main' into electron-app 2026-06-26 10:02:48 +02:00
Simos Mikelatos
8adcdaa0e5 Merge branch 'main' into camoufox-novnc-browser-use 2026-06-24 23:16:13 +02:00
Simos Mikelatos
6f712269e8 fix: remove invalid windows build options 2026-06-24 21:05:59 +00:00
Simos Mikelatos
52244404a3 chore: remove windows icon generator 2026-06-24 20:50:45 +00:00
Simos Mikelatos
8ad18f8587 fix: improve desktop chat performance 2026-06-24 20:49:24 +00:00
Simos Mikelatos
fe116a7138 ci: restore notarized macOS branch builds 2026-06-24 20:25:53 +00:00
Simos Mikelatos
490e66ebdb fix: stabilize desktop environment auth navigation 2026-06-24 20:09:41 +00:00
Simos Mikelatos
81eb966904 ci: skip notarization for macOS branch builds 2026-06-24 20:05:52 +00:00
Simos Mikelatos
0d68dc2cd0 fix: add Electron tab diagnostics 2026-06-24 20:00:45 +00:00
Simos Mikelatos
0610cc8333 fix: browser use set profile root folder 2026-06-24 19:21:52 +00:00
Simos Mikelatos
9457651fdd fix(browser-use): harden browser settings state 2026-06-24 15:36:25 +00:00
Simos Mikelatos
8c31ebcc63 feat(browser-use): add Camoufox noVNC session viewer 2026-06-24 14:39:41 +00:00
Simos Mikelatos
bb630ef739 fix: hide computer use menus 2026-06-20 01:50:02 +00:00
Simos Mikelatos
1c05fe0905 fix: stabilize cloud computer use mcp 2026-06-19 20:47:53 +00:00
Simos Mikelatos
077baee5f2 fix: authenticate desktop agent websocket 2026-06-19 15:52:49 +00:00
coderabbitai[bot]
f150fa6b09 fix: apply CodeRabbit auto-fixes
Fixed 1 file(s) based on 1 unresolved review comment.

Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
2026-06-19 15:22:43 +00:00
Simos Mikelatos
9f8cee8919 fix: restore macos semantic helper cast 2026-06-19 15:05:47 +00:00
Simos Mikelatos
bb323fc566 fix: respect cloud computer use setting 2026-06-19 15:02:07 +00:00
Simos Mikelatos
5ef40be2d3 fix: macos release 2026-06-19 14:46:58 +00:00
Simos Mikelatos
cf4b28273e fix: compile macos semantic helper 2026-06-19 14:22:47 +00:00
Simos Mikelatos
f4c68942a5 fix: repair desktop launcher local view 2026-06-19 14:20:23 +00:00
Simos Mikelatos
4d70a2588c feat: improve Computer Use linking status 2026-06-19 13:47:16 +00:00
Simos Mikelatos
218e8e2e38 chore: update Codex SDK to latest 2026-06-19 13:12:53 +00:00
Simos Mikelatos
53c3c4c27a Fix long-running desktop resource leaks 2026-06-19 13:07:08 +00:00
Simos Mikelatos
901c6fc956 chore: simplify desktop release artifacts 2026-06-19 13:04:53 +00:00
Simos Mikelatos
278fe4f7b1 Fix semantic review issues and release action runtime 2026-06-19 12:46:40 +00:00
Simos Mikelatos
d7f4d4c342 Fix desktop release review findings 2026-06-19 12:29:46 +00:00
Simos Mikelatos
d1930fecdb fix: build semantic helpers on macos and windows 2026-06-19 12:17:32 +00:00
Simos Mikelatos
1726705459 feat: add CloudCLI computer use semantics, desktop helper packaging, and permission onboarding 2026-06-19 12:09:55 +00:00
Simos Mikelatos
a35200f340 Harden computer use MCP handling 2026-06-19 08:06:26 +00:00
Simos Mikelatos
06c9745489 Update src/i18n/locales/zh-CN/settings.json
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-06-19 09:56:21 +02:00
Simos Mikelatos
0dd22db2bb Update src/i18n/locales/zh-TW/settings.json
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-06-19 09:56:01 +02:00
Simos Mikelatos
e7aa72c41e Update src/i18n/locales/tr/settings.json
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-06-19 09:55:45 +02:00
Simos Mikelatos
9f24f80f33 Fix computer use session error status 2026-06-19 07:47:56 +00:00
Simos Mikelatos
25ab273b05 Publish branch server bundles for thin desktop builds 2026-06-19 07:08:19 +00:00
Simos Mikelatos
5be100ea1b Keep branch desktop artifacts thin 2026-06-19 06:49:28 +00:00
Simos Mikelatos
2af3d38afe Harden desktop workflows and computer use handling 2026-06-19 06:21:13 +00:00
Simos Mikelatos
531833bc87 Merge branch 'main' into electron-app 2026-06-19 08:19:36 +02:00
Simos Mikelatos
b2333e7d93 Fix launcher CodeQL unused helpers 2026-06-18 21:17:09 +00:00
Simos Mikelatos
f75ae385dd Add on-demand desktop server bundle 2026-06-18 21:08:29 +00:00
Simos Mikelatos
7786763dd1 Fix desktop settings modal behavior 2026-06-18 06:15:17 +00:00
Simos Mikelatos
1dbf545fd9 Authenticate ripgrep install in desktop workflows 2026-06-17 22:29:55 +00:00
Simos Mikelatos
ac37213269 Run desktop branch builds on electron app pushes 2026-06-17 22:26:47 +00:00
Simos Mikelatos
65fdc38f2e Add desktop app packaging and settings updates 2026-06-17 22:15:36 +00:00
Simos Mikelatos
6c2652aee6 Merge remote-tracking branch 'origin/main' into electron-app
# Conflicts:
#	package-lock.json
#	package.json
#	server/index.js
#	src/components/main-content/types/types.ts
#	src/components/main-content/view/MainContent.tsx
#	src/components/main-content/view/subcomponents/MainContentHeader.tsx
#	src/components/main-content/view/subcomponents/MainContentTabSwitcher.tsx
#	src/components/main-content/view/subcomponents/MainContentTitle.tsx
#	src/components/settings/hooks/useSettingsController.ts
#	src/components/settings/types/types.ts
#	src/components/settings/view/Settings.tsx
#	src/components/settings/view/SettingsSidebar.tsx
#	src/hooks/useProjectsState.ts
#	src/i18n/locales/de/common.json
#	src/i18n/locales/en/common.json
#	src/i18n/locales/en/settings.json
#	src/i18n/locales/it/common.json
#	src/i18n/locales/ja/common.json
#	src/i18n/locales/ko/common.json
#	src/i18n/locales/ru/common.json
#	src/i18n/locales/tr/common.json
#	src/i18n/locales/zh-CN/common.json
#	src/i18n/locales/zh-TW/common.json
#	src/types/app.ts
2026-06-17 20:18:09 +00:00
Simos Mikelatos
bf50d29c20 Merge remote-tracking branch 'origin/browser-use' into electron-app
# Conflicts:
#	src/i18n/locales/en/settings.json
2026-06-17 20:17:38 +00:00
Simos Mikelatos
ffc0cd7501 Improve Browser settings load and managed MCP display 2026-06-17 20:04:44 +00:00
Simos Mikelatos
59194d1502 Refine Browser naming and managed MCP UX
- Rename Browser Use surfaces to Browser
- Register Browser MCP under the new server name
- Mark CloudCLI-managed MCP servers read-only
- Adjust MCP stdio framing and sidebar footer sizing
2026-06-17 19:18:23 +00:00
Simos Mikelatos
7e6028b113 feat: add desktop computer use runtime 2026-06-17 19:01:15 +00:00
Simos Mikelatos
9881e5e366 feat(browser-use): improve mobile monitoring ux 2026-06-17 18:19:12 +00:00
Simos Mikelatos
496a895e8a feat(browser-use): refine monitoring panel ux 2026-06-17 17:39:55 +00:00
Simos Mikelatos
086df034b4 feat(browser-use): simplify agent session monitoring 2026-06-17 17:04:11 +00:00
Simos Mikelatos
fc71fc7d2b Merge branch 'pr889-fixes' into electron-app
# Conflicts:
#	server/index.js
2026-06-17 15:45:07 +00:00
Simos Mikelatos
a0d56429a7 fix browser use 2026-06-17 15:43:21 +00:00
Simos Mikelatos
6af4afe6f2 Merge branch 'main' into browser-use 2026-06-16 19:02:36 +02:00
Simos Mikelatos
7aeca52669 Merge branch 'browser-use' into electron-app
# Conflicts:
#	src/components/main-content/view/subcomponents/MainContentTabSwitcher.tsx
2026-06-16 06:51:35 +00:00
Simos Mikelatos
56532af33a feat: add browser use guide links 2026-06-15 21:22:49 +00:00
Simos Mikelatos
9438a365f2 feat: improve browser use session controls 2026-06-15 21:14:10 +00:00
Simos Mikelatos
e5c6e5e596 fix: hide browser use runtime mode 2026-06-15 20:20:44 +00:00
Simos Mikelatos
0426522406 feat: expose browser use to agents via MCP 2026-06-15 19:47:58 +00:00
Simos Mikelatos
6e7e2ff4c1 feat: make browser use opt-in 2026-06-15 18:12:27 +00:00
Simos Mikelatos
e6263dbd1f refactor: store browser use settings in database 2026-06-15 17:57:00 +00:00
Simos Mikelatos
260070bae0 feat: add browser use runtime setup settings 2026-06-15 17:52:27 +00:00
Simos Mikelatos
daac6e3fd3 ci: add macos desktop release workflow 2026-06-15 17:26:53 +00:00
Simos Mikelatos
861cfecbaa feat: add electron app support 2026-06-15 16:21:05 +00:00
Simos Mikelatos
a182765e10 Merge branch 'browser-use' into electron-app 2026-06-15 16:15:03 +00:00
Simos Mikelatos
828d1a2302 Merge remote-tracking branch 'origin/feat/unify-websocket-2' into browser-use-independent 2026-06-15 16:12:10 +00:00
Simos Mikelatos
d427004bd7 Merge browser use branch 2026-06-14 20:34:36 +00:00
Simos Mikelatos
243e6cecd5 Add browser use workspace panel 2026-06-14 20:34:16 +00:00
24 changed files with 1552 additions and 194 deletions

View File

@@ -14,18 +14,22 @@ jobs:
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
template: [claude-code, codex, gemini] template: [claude-code, codex]
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@v6
- name: Set up QEMU - name: Set up QEMU
uses: docker/setup-qemu-action@v3 uses: docker/setup-qemu-action@v4
with:
platforms: arm64
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v4
with:
version: latest
- name: Log in to Docker Hub - name: Log in to Docker Hub
uses: docker/login-action@v3 uses: docker/login-action@v4
with: with:
username: ${{ secrets.DOCKERHUB_USERNAME }} username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }} password: ${{ secrets.DOCKERHUB_TOKEN }}
@@ -40,7 +44,7 @@ jobs:
echo "tags=$TAGS" >> "$GITHUB_OUTPUT" echo "tags=$TAGS" >> "$GITHUB_OUTPUT"
- name: Build and push - name: Build and push
uses: docker/build-push-action@v6 uses: docker/build-push-action@v7
with: with:
context: ./docker context: ./docker
file: ./docker/${{ matrix.template }}/Dockerfile file: ./docker/${{ matrix.template }}/Dockerfile

View File

@@ -5,7 +5,16 @@ COPY shared/install-cloudcli.sh /tmp/install-cloudcli.sh
RUN chmod +x /tmp/install-cloudcli.sh && /tmp/install-cloudcli.sh RUN chmod +x /tmp/install-cloudcli.sh && /tmp/install-cloudcli.sh
USER agent USER agent
RUN npm install -g @cloudcli-ai/cloudcli && cloudcli --version RUN --mount=type=cache,target=/tmp/npm-cache,sharing=locked,mode=0777 \
npm install -g @cloudcli-ai/cloudcli \
--cache=/tmp/npm-cache \
--fetch-retries=5 \
--fetch-retry-mintimeout=20000 \
--fetch-retry-maxtimeout=120000 \
--fetch-timeout=600000 \
--no-audit \
--no-fund && \
cloudcli --version
COPY --chown=agent:agent shared/start-cloudcli.sh /home/agent/.cloudcli-start.sh COPY --chown=agent:agent shared/start-cloudcli.sh /home/agent/.cloudcli-start.sh
RUN echo '. ~/.cloudcli-start.sh' >> /home/agent/.bashrc RUN echo '. ~/.cloudcli-start.sh' >> /home/agent/.bashrc

View File

@@ -5,7 +5,16 @@ COPY shared/install-cloudcli.sh /tmp/install-cloudcli.sh
RUN chmod +x /tmp/install-cloudcli.sh && /tmp/install-cloudcli.sh RUN chmod +x /tmp/install-cloudcli.sh && /tmp/install-cloudcli.sh
USER agent USER agent
RUN npm install -g @cloudcli-ai/cloudcli && cloudcli --version RUN --mount=type=cache,target=/tmp/npm-cache,sharing=locked,mode=0777 \
npm install -g @cloudcli-ai/cloudcli \
--cache=/tmp/npm-cache \
--fetch-retries=5 \
--fetch-retry-mintimeout=20000 \
--fetch-retry-maxtimeout=120000 \
--fetch-timeout=600000 \
--no-audit \
--no-fund && \
cloudcli --version
COPY --chown=agent:agent shared/start-cloudcli.sh /home/agent/.cloudcli-start.sh COPY --chown=agent:agent shared/start-cloudcli.sh /home/agent/.cloudcli-start.sh
RUN echo '. ~/.cloudcli-start.sh' >> /home/agent/.bashrc RUN echo '. ~/.cloudcli-start.sh' >> /home/agent/.bashrc

View File

@@ -69,7 +69,7 @@ const sessionIdSchema = {
const tools: ToolDefinition[] = [ const tools: ToolDefinition[] = [
{ {
name: 'browser_create_session', name: 'browser_create_session',
description: 'Create a temporary Browser session that the agent can control. Optionally provide a background profileName to reuse cookies and storage.', description: 'Create a Browser session that the agent can control. Provide profileName to use a specific persistent profile; when omitted, the configured persistent profile is used only if session persistence is enabled, otherwise a temporary session is created.',
inputSchema: { inputSchema: {
type: 'object', type: 'object',
properties: { properties: {

View File

@@ -62,7 +62,7 @@ import voiceRoutes from './voice-proxy.js';
import browserUseRoutes from './modules/browser-use/browser-use.routes.js'; import browserUseRoutes from './modules/browser-use/browser-use.routes.js';
import { assetsRoutes } from './modules/assets/index.js'; import { assetsRoutes } from './modules/assets/index.js';
import browserUseMcpRoutes from './modules/browser-use/browser-use-mcp.routes.js'; import browserUseMcpRoutes from './modules/browser-use/browser-use-mcp.routes.js';
import { browserUseService } from './modules/browser-use/browser-use.service.js'; import { browserUseService, VIEWER_COOKIE_NAME } from './modules/browser-use/index.js';
import { startEnabledPluginServers, stopAllPlugins, getPluginPort } from './utils/plugin-process-manager.js'; import { startEnabledPluginServers, stopAllPlugins, getPluginPort } from './utils/plugin-process-manager.js';
import { initializeDatabase, projectsDb, sessionsDb } from './modules/database/index.js'; import { initializeDatabase, projectsDb, sessionsDb } from './modules/database/index.js';
import { configureWebPush } from './services/vapid-keys.js'; import { configureWebPush } from './services/vapid-keys.js';
@@ -139,6 +139,8 @@ const wss = createWebSocketServer(server, {
shouldAutoOpenUrlFromOutput, shouldAutoOpenUrlFromOutput,
}, },
getPluginPort, getPluginPort,
browserUseViewer: (ws, pathname) => browserUseService.handleViewerWebSocket(ws, pathname),
authenticateBrowserUseViewer: authenticateBrowserUseViewerPath,
}); });
// Make WebSocket server available to routes // Make WebSocket server available to routes
@@ -206,11 +208,42 @@ app.use('/api/user', authenticateToken, userRoutes);
// Plugins API Routes (protected) // Plugins API Routes (protected)
app.use('/api/plugins', authenticateToken, pluginsRoutes); app.use('/api/plugins', authenticateToken, pluginsRoutes);
function readCookieValue(header, name) {
if (!header) return null;
const prefix = `${name}=`;
const cookie = String(header).split(';').map((part) => part.trim()).find((part) => part.startsWith(prefix));
return cookie ? decodeURIComponent(cookie.slice(prefix.length)) : null;
}
function authenticateBrowserUseViewerPath(pathname, token) {
const parts = String(pathname || '').split('/');
const sessionId = parts[4];
if (parts[1] !== 'api' || parts[2] !== 'browser-use' || parts[3] !== 'sessions' || parts[5] !== 'viewer' || parts[6] !== 'websockify') {
return false;
}
return browserUseService.validateViewerToken(decodeURIComponent(sessionId), token);
}
function authenticateBrowserUse(req, res, next) {
const match = /^\/sessions\/([^/]+)\/viewer(?:\/|$)/.exec(req.path || '');
if (match) {
const sessionId = decodeURIComponent(match[1]);
const token = typeof req.query.viewerToken === 'string'
? req.query.viewerToken
: readCookieValue(req.headers.cookie, VIEWER_COOKIE_NAME);
if (browserUseService.validateViewerToken(sessionId, token)) {
return next();
}
return res.status(401).json({ error: 'Browser viewer access requires a valid session token.' });
}
return authenticateToken(req, res, next);
}
// Browser MCP bridge API (local token protected) // Browser MCP bridge API (local token protected)
app.use('/api/browser-use-mcp', browserUseMcpRoutes); app.use('/api/browser-use-mcp', browserUseMcpRoutes);
// Browser API Routes (protected) // Browser API Routes (protected)
app.use('/api/browser-use', authenticateToken, browserUseRoutes); app.use('/api/browser-use', authenticateBrowserUse, browserUseRoutes);
// Unified provider MCP routes (protected) // Unified provider MCP routes (protected)
app.use('/api/providers', authenticateToken, providerRoutes); app.use('/api/providers', authenticateToken, providerRoutes);

View File

@@ -1,6 +1,7 @@
import express from 'express'; import express from 'express';
import { browserUseService } from '@/modules/browser-use/browser-use.service.js'; import { browserUseService } from '@/modules/browser-use/browser-use.service.js';
import { VIEWER_COOKIE_NAME, VIEWER_TOKEN_TTL_MS } from '@/modules/browser-use/browser-use.viewer.js';
const router = express.Router(); const router = express.Router();
@@ -8,6 +9,45 @@ function readParam(value: string | string[] | undefined): string {
return Array.isArray(value) ? value[0] || '' : value || ''; return Array.isArray(value) ? value[0] || '' : value || '';
} }
const SAFE_VIEWER_ROOT_FILES = new Set(['vnc.html', 'favicon.ico', 'manifest.json']);
const SAFE_VIEWER_ROOT_DIRS = new Set(['app', 'core', 'vendor', 'assets', 'images', 'utils']);
function isSafeViewerPath(viewerPath: string): boolean {
if (!viewerPath || viewerPath.startsWith('/') || viewerPath.includes('..') || viewerPath.includes('\\')) {
return false;
}
if (!/^[A-Za-z0-9][A-Za-z0-9._~/-]*$/.test(viewerPath)) {
return false;
}
if (SAFE_VIEWER_ROOT_FILES.has(viewerPath)) {
return true;
}
const [rootDir] = viewerPath.split('/');
return Boolean(rootDir && SAFE_VIEWER_ROOT_DIRS.has(rootDir));
}
function isSecureRequest(req: express.Request): boolean {
const forwardedProto = String(req.headers['x-forwarded-proto'] || '')
.split(',')[0]
.trim()
.toLowerCase();
return req.secure || forwardedProto === 'https';
}
function readQueryString(originalUrl: string): string {
const queryIndex = originalUrl.indexOf('?');
if (queryIndex < 0) {
return '';
}
const params = new URLSearchParams(originalUrl.slice(queryIndex + 1));
params.delete('viewerToken');
const nextQuery = params.toString();
return nextQuery ? `?${nextQuery}` : '';
}
router.get('/status', async (_req, res) => { router.get('/status', async (_req, res) => {
try { try {
res.json({ success: true, data: await browserUseService.getStatus() }); res.json({ success: true, data: await browserUseService.getStatus() });
@@ -62,13 +102,60 @@ router.get('/sessions', async (_req, res) => {
try { try {
res.json({ success: true, data: { sessions: await browserUseService.listSessions() } }); res.json({ success: true, data: { sessions: await browserUseService.listSessions() } });
} catch (error) { } catch (error) {
res.status(401).json({ res.status(500).json({
success: false, success: false,
error: error instanceof Error ? error.message : 'Failed to list browser sessions.', error: error instanceof Error ? error.message : 'Failed to list browser sessions.',
}); });
} }
}); });
router.get('/sessions/:sessionId/viewer/*', async (req, res) => {
try {
const sessionId = readParam(req.params.sessionId);
const originalPath = req.originalUrl.split('?')[0] || '';
const viewerMarker = `/sessions/${sessionId}/viewer/`;
const markerIndex = originalPath.indexOf(viewerMarker);
const rawViewerPath = markerIndex >= 0 ? originalPath.slice(markerIndex + viewerMarker.length) : 'vnc.html';
const viewerPath = decodeURIComponent(rawViewerPath).replace(/^\/+/, '') || 'vnc.html';
if (!isSafeViewerPath(viewerPath)) {
res.status(400).json({ success: false, error: 'Invalid Browser viewer path.' });
return;
}
const viewerToken = readParam(req.query.viewerToken as string | string[] | undefined);
if (viewerPath === 'vnc.html' && browserUseService.validateViewerToken(sessionId, viewerToken)) {
res.cookie(VIEWER_COOKIE_NAME, viewerToken, {
httpOnly: true,
sameSite: 'lax',
secure: isSecureRequest(req),
maxAge: VIEWER_TOKEN_TTL_MS,
path: '/api/browser-use/sessions/' + encodeURIComponent(sessionId) + '/viewer',
});
}
const target = browserUseService.getViewerProxyTarget(sessionId);
const query = readQueryString(req.originalUrl);
const upstream = await fetch(`http://127.0.0.1:${target.websockifyPort}/${viewerPath}${query}`, {
headers: {
accept: String(req.headers.accept || '*/*'),
},
});
const contentType = upstream.headers.get('content-type');
if (contentType) {
res.setHeader('content-type', contentType);
}
const cacheControl = viewerPath === 'vnc.html' ? 'no-store' : 'public, max-age=3600';
res.setHeader('cache-control', cacheControl);
res.status(upstream.status);
const body = Buffer.from(await upstream.arrayBuffer());
res.send(body);
} catch (error) {
res.status(404).json({
success: false,
error: error instanceof Error ? error.message : 'Browser viewer is not available.',
});
}
});
router.post('/sessions/:sessionId/stop', async (req, res) => { router.post('/sessions/:sessionId/stop', async (req, res) => {
try { try {
const result = await browserUseService.stopSession(readParam(req.params.sessionId)); const result = await browserUseService.stopSession(readParam(req.params.sessionId));

View File

@@ -1,128 +1,117 @@
import { createRequire } from 'node:module'; import { createRequire } from 'node:module';
import { execFileSync, spawn } from 'node:child_process';
import { randomBytes, randomUUID } from 'node:crypto'; import { randomBytes, randomUUID } from 'node:crypto';
import fs from 'node:fs'; import fs from 'node:fs';
import net from 'node:net';
import os from 'node:os'; import os from 'node:os';
import path from 'node:path'; import path from 'node:path';
// cross-spawn: drop-in spawn with Windows .cmd/PATHEXT resolution. import { WebSocket } from 'ws';
import spawn from 'cross-spawn'; import spawn from 'cross-spawn';
import { appConfigDb } from '@/modules/database/index.js'; import { appConfigDb } from '@/modules/database/index.js';
import { providerMcpService } from '@/modules/providers/index.js'; import { providerMcpService } from '@/modules/providers/index.js';
import { getModuleDir } from '@/utils/runtime-paths.js'; import { getModuleDir } from '@/utils/runtime-paths.js';
import {
getOrCreateMcpToken,
getProfilePath,
normalizeBrowserBackend,
PROFILE_ROOT,
readSettings,
resolveSessionProfileName,
useVisibleCamoufoxBackend,
writeSettings,
} from './browser-use.settings.js';
import type {
BrowserUseSession,
BrowserUseSettings,
PublicBrowserUseSession,
RuntimeHandle,
RuntimeProbe,
RuntimeReadiness,
} from './browser-use.types.js';
import { getViewerUrl, handleViewerWebSocket, VIEWER_TOKEN_TTL_MS } from './browser-use.viewer.js';
const require = createRequire(import.meta.url); const require = createRequire(import.meta.url);
const __dirname = getModuleDir(import.meta.url); const __dirname = getModuleDir(import.meta.url);
const IS_PLATFORM = process.env.VITE_IS_PLATFORM === 'true'; const IS_PLATFORM = process.env.VITE_IS_PLATFORM === 'true';
const MAX_SESSIONS_PER_OWNER = Number.parseInt(process.env.CLOUDCLI_BROWSER_USE_MAX_SESSIONS_PER_OWNER || '3', 10); const MAX_SESSIONS_PER_OWNER = Number.parseInt(process.env.CLOUDCLI_BROWSER_USE_MAX_SESSIONS_PER_OWNER || '3', 10);
const SESSION_TTL_MS = Number.parseInt(process.env.CLOUDCLI_BROWSER_USE_SESSION_TTL_MS || String(30 * 60 * 1000), 10); const SESSION_TTL_MS = Number.parseInt(process.env.CLOUDCLI_BROWSER_USE_SESSION_TTL_MS || String(30 * 60 * 1000), 10);
const BROWSER_USE_SETTINGS_KEY = 'browser_use_settings';
const BROWSER_USE_MCP_TOKEN_KEY = 'browser_use_mcp_token';
type BrowserUseRuntime = 'cloud' | 'local';
type BrowserUseSessionStatus = 'ready' | 'stopped' | 'unavailable';
type BrowserUseSession = {
id: string;
ownerId: string;
createdBy: 'agent';
runtime: BrowserUseRuntime;
status: BrowserUseSessionStatus;
url: string | null;
title: string | null;
screenshotDataUrl: string | null;
createdAt: string;
updatedAt: string;
lastAction: string | null;
message: string | null;
profileName: string | null;
viewport: {
width: number;
height: number;
} | null;
cursor: {
x: number;
y: number;
actor: 'agent';
} | null;
};
type PublicBrowserUseSession = Omit<BrowserUseSession, 'ownerId'>;
type RuntimeHandle = {
browser?: any;
context?: any;
page?: any;
};
type BrowserUseSettings = {
enabled: boolean;
};
type RuntimeReadiness = {
playwright: any | null;
playwrightInstalled: boolean;
chromiumInstalled: boolean;
chromiumExecutablePath: string | null;
installInProgress: boolean;
installMessage: string | null;
};
type RuntimeProbe = Omit<RuntimeReadiness, 'installInProgress' | 'installMessage'>;
const sessions = new Map<string, BrowserUseSession>(); const sessions = new Map<string, BrowserUseSession>();
const handles = new Map<string, RuntimeHandle>(); const handles = new Map<string, RuntimeHandle>();
const viewerTokens = new Map<string, { token: string; expiresAt: number }>();
let installPromise: Promise<{ success: boolean; message: string }> | null = null; let installPromise: Promise<{ success: boolean; message: string }> | null = null;
let lastInstallMessage: string | null = null; let lastInstallMessage: string | null = null;
let runtimeProbeCache: { value: RuntimeProbe; updatedAt: number } | null = null; let runtimeProbeCache: { value: RuntimeProbe; updatedAt: number } | null = null;
const DEFAULT_SETTINGS: BrowserUseSettings = {
enabled: false,
};
const AGENT_OWNER_ID = 'agent'; const AGENT_OWNER_ID = 'agent';
const PROFILE_ROOT = path.join(os.homedir(), '.cloudcli', 'browser-use', 'profiles');
const MCP_SERVER_NAME = 'cloudcli-browser'; const MCP_SERVER_NAME = 'cloudcli-browser';
const LEGACY_MCP_SERVER_NAMES = ['cloudcli-browser-use']; const LEGACY_MCP_SERVER_NAMES = ['cloudcli-browser-use'];
const RUNTIME_READINESS_CACHE_TTL_MS = 30_000; const RUNTIME_READINESS_CACHE_TTL_MS = 30_000;
const VISIBLE_BROWSER_ENABLED = process.env.CLOUDCLI_BROWSER_USE_VISIBLE !== 'false';
// The noVNC viewer pipeline (Xvfb + x11vnc + websockify) only exists on Linux.
// On other platforms visible Camoufox sessions open a normal window on the host desktop.
const VNC_VIEWER_SUPPORTED = process.platform === 'linux';
const RUNTIME_ROOT = process.env.CLOUDCLI_BROWSER_USE_RUNTIME_ROOT || '/opt/claudecodeui/.runtime-browser';
const NOVNC_ROOT = process.env.CLOUDCLI_BROWSER_USE_NOVNC_ROOT || path.join(RUNTIME_ROOT, 'novnc');
const X11VNC_BIN = process.env.CLOUDCLI_BROWSER_USE_X11VNC_BIN || path.join(RUNTIME_ROOT, 'rootfs/usr/bin/x11vnc');
const X11VNC_LIB_DIR = process.env.CLOUDCLI_BROWSER_USE_X11VNC_LIB_DIR || path.join(RUNTIME_ROOT, 'rootfs/usr/lib/x86_64-linux-gnu');
const X11VNC_EXTRA_LIB_DIR = process.env.CLOUDCLI_BROWSER_USE_X11VNC_EXTRA_LIB_DIR || path.join(RUNTIME_ROOT, 'rootfs/lib/x86_64-linux-gnu');
const LOG_RUNTIME_PROCESS_OUTPUT = process.env.CLOUDCLI_BROWSER_USE_RUNTIME_LOGS === 'true';
const RUNTIME_PROCESS_SHUTDOWN_TIMEOUT_MS = 1_500;
function getRuntime(): BrowserUseRuntime { function getRuntime(): 'cloud' | 'local' {
return IS_PLATFORM ? 'cloud' : 'local'; return IS_PLATFORM ? 'cloud' : 'local';
} }
function readSettings(): BrowserUseSettings { const CAMOUFOX_CLI_CANDIDATES = [
try { path.join(os.homedir(), '.local/bin/camoufox'),
const raw = appConfigDb.get(BROWSER_USE_SETTINGS_KEY); // PATH lookup covers pipx on Linux/macOS and the pip Scripts dir on Windows.
if (!raw) { 'camoufox',
return DEFAULT_SETTINGS; ];
const CAMOUFOX_BINARY_CANDIDATES = process.platform === 'win32'
? ['camoufox.exe', 'camoufox']
: process.platform === 'darwin'
? ['Camoufox.app/Contents/MacOS/camoufox', 'camoufox']
: ['camoufox'];
function findCamoufoxBinary(installPath: string): string | null {
if (!fs.statSync(installPath).isDirectory()) {
return fs.existsSync(installPath) ? installPath : null;
}
for (const candidate of CAMOUFOX_BINARY_CANDIDATES) {
const executablePath = path.join(installPath, candidate);
if (fs.existsSync(executablePath)) {
return executablePath;
} }
const parsed = JSON.parse(raw) as Partial<BrowserUseSettings>;
return {
enabled: parsed.enabled === true,
};
} catch (error: any) {
console.warn('[Browser] Failed to read settings:', error?.message || error);
return DEFAULT_SETTINGS;
} }
return null;
} }
function writeSettings(settings: BrowserUseSettings): BrowserUseSettings { function getCamoufoxExecutablePath(): string | null {
const normalized = { const configured = process.env.CLOUDCLI_BROWSER_USE_CAMOUFOX_EXECUTABLE;
enabled: settings.enabled === true, if (configured && fs.existsSync(configured)) {
}; return findCamoufoxBinary(configured);
appConfigDb.set(BROWSER_USE_SETTINGS_KEY, JSON.stringify(normalized));
return normalized;
}
function getOrCreateMcpToken(): string {
const existing = appConfigDb.get(BROWSER_USE_MCP_TOKEN_KEY);
if (existing) {
return existing;
} }
const token = randomBytes(32).toString('hex');
appConfigDb.set(BROWSER_USE_MCP_TOKEN_KEY, token); for (const cli of CAMOUFOX_CLI_CANDIDATES) {
return token; try {
const output = execFileSync(cli, ['path'], {
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'ignore'],
}).trim();
const executablePath = findCamoufoxBinary(output);
if (executablePath) {
return executablePath;
}
} catch {
// Try the next candidate.
}
}
return null;
} }
function getSetupMessage(settings: BrowserUseSettings, readiness: RuntimeReadiness): string { function getSetupMessage(settings: BrowserUseSettings, readiness: RuntimeReadiness): string {
@@ -134,6 +123,28 @@ function getSetupMessage(settings: BrowserUseSettings, readiness: RuntimeReadine
return 'Install Playwright and Chromium to use browser sessions.'; return 'Install Playwright and Chromium to use browser sessions.';
} }
if (settings.browserBackend === 'camoufox-vnc' && !getCamoufoxExecutablePath()) {
return 'Camoufox is selected, but Camoufox is not installed.';
}
if (useVisibleCamoufoxBackend(settings)) {
if (!VISIBLE_BROWSER_ENABLED) {
return 'Camoufox is selected, but visible browser sessions are disabled.';
}
if (!getCamoufoxExecutablePath()) {
return 'Camoufox is selected, but Camoufox is not installed.';
}
if (VNC_VIEWER_SUPPORTED) {
if (!fs.existsSync(X11VNC_BIN)) {
return 'Camoufox is selected, but x11vnc is missing.';
}
if (!fs.existsSync(path.join(NOVNC_ROOT, 'vnc.html'))) {
return 'Camoufox is selected, but noVNC is missing.';
}
}
return readiness.installMessage || 'Camoufox runtime is not ready.';
}
if (!readiness.chromiumInstalled) { if (!readiness.chromiumInstalled) {
return 'Playwright is installed, but Chromium is missing. Install the Chromium runtime to continue.'; return 'Playwright is installed, but Chromium is missing. Install the Chromium runtime to continue.';
} }
@@ -141,6 +152,16 @@ function getSetupMessage(settings: BrowserUseSettings, readiness: RuntimeReadine
return readiness.installMessage || 'Browser runtime is not ready.'; return readiness.installMessage || 'Browser runtime is not ready.';
} }
function isVncRuntimeInstalled(): boolean {
return fs.existsSync(X11VNC_BIN) && fs.existsSync(path.join(NOVNC_ROOT, 'vnc.html'));
}
function isVisibleCamoufoxReady(): boolean {
return VISIBLE_BROWSER_ENABLED
&& Boolean(getCamoufoxExecutablePath())
&& (!VNC_VIEWER_SUPPORTED || isVncRuntimeInstalled());
}
function getPlaywright(): any | null { function getPlaywright(): any | null {
try { try {
return require('playwright'); return require('playwright');
@@ -178,24 +199,6 @@ async function removeMcpServerFromAllProviders(name: string) {
return results.map((result) => ({ ...result, name })); return results.map((result) => ({ ...result, name }));
} }
function normalizeProfileName(profileName?: string | null): string | null {
const normalized = String(profileName || '').trim();
if (!normalized) {
return null;
}
return normalized.slice(0, 80);
}
function getProfilePath(profileName: string): string {
const safeName = profileName
.toLowerCase()
.replace(/[^a-z0-9._-]+/g, '-')
.replace(/^-+|-+$/g, '')
.slice(0, 80) || 'default';
return path.join(PROFILE_ROOT, safeName);
}
function probeRuntime(): RuntimeProbe { function probeRuntime(): RuntimeProbe {
const playwright = getPlaywright(); const playwright = getPlaywright();
const readiness: RuntimeProbe = { const readiness: RuntimeProbe = {
@@ -240,6 +243,271 @@ function getRuntimeReadiness(options: { force?: boolean } = {}): RuntimeReadines
}; };
} }
function findAvailablePort(): Promise<number> {
return new Promise((resolve, reject) => {
const server = net.createServer();
server.on('error', reject);
server.listen(0, '127.0.0.1', () => {
const address = server.address();
server.close(() => {
if (typeof address === 'object' && address?.port) {
resolve(address.port);
} else {
reject(new Error('Failed to reserve a browser runtime port.'));
}
});
});
});
}
function delay(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function isRuntimeProcessAlive(child: ReturnType<typeof spawn>): boolean {
return child.exitCode === null && child.signalCode === null && !child.killed;
}
function isRuntimeProcessRunning(child: ReturnType<typeof spawn>): boolean {
return child.exitCode === null && child.signalCode === null;
}
function assertRuntimeProcessesAlive(processes: Array<ReturnType<typeof spawn>>, label: string) {
const exited = processes.find((child) => !isRuntimeProcessAlive(child));
if (exited) {
throw new Error(`${label} exited before the Browser viewer runtime was ready.`);
}
}
async function isPortListening(port: number): Promise<boolean> {
return new Promise((resolve) => {
const socket = net.createConnection({ host: '127.0.0.1', port });
let settled = false;
const finish = (listening: boolean) => {
if (settled) {
return;
}
settled = true;
socket.destroy();
resolve(listening);
};
socket.setTimeout(250);
socket.once('connect', () => finish(true));
socket.once('timeout', () => finish(false));
socket.once('error', () => finish(false));
});
}
async function waitForRuntimePort(
port: number,
label: string,
processes: Array<ReturnType<typeof spawn>>,
timeoutMs = 5_000,
) {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
assertRuntimeProcessesAlive(processes, label);
if (await isPortListening(port)) {
return;
}
await delay(100);
}
assertRuntimeProcessesAlive(processes, label);
throw new Error(`${label} did not start listening on 127.0.0.1:${port}.`);
}
function waitForRuntimeProcessExit(child: ReturnType<typeof spawn>, timeoutMs: number): Promise<boolean> {
if (!isRuntimeProcessRunning(child)) {
return Promise.resolve(true);
}
return new Promise((resolve) => {
let settled = false;
const finish = (exited: boolean) => {
if (settled) {
return;
}
settled = true;
clearTimeout(timer);
child.off('exit', onExit);
resolve(exited);
};
const onExit = () => finish(true);
const timer = setTimeout(() => finish(false), timeoutMs);
timer.unref?.();
child.once('exit', onExit);
});
}
async function stopRuntimeProcesses(processes?: Array<ReturnType<typeof spawn>>) {
const liveProcesses = processes?.filter(isRuntimeProcessRunning) || [];
if (liveProcesses.length === 0) {
return;
}
liveProcesses.forEach((child) => {
try {
child.kill('SIGTERM');
} catch {
// Process may have exited between filtering and signalling.
}
});
const exited = await Promise.all(
liveProcesses.map((child) => waitForRuntimeProcessExit(child, RUNTIME_PROCESS_SHUTDOWN_TIMEOUT_MS)),
);
const stubbornProcesses = liveProcesses.filter((child, index) => !exited[index] && isRuntimeProcessRunning(child));
if (stubbornProcesses.length === 0) {
return;
}
console.warn(`[Browser runtime] Force stopping ${stubbornProcesses.length} runtime process(es).`);
stubbornProcesses.forEach((child) => {
try {
child.kill('SIGKILL');
} catch {
// Process may have exited before escalation.
}
});
await Promise.all(stubbornProcesses.map((child) => waitForRuntimeProcessExit(child, 500)));
}
function spawnRuntimeProcess(command: string, args: string[], options: { env?: NodeJS.ProcessEnv } = {}) {
const child = spawn(command, args, {
env: { ...process.env, ...options.env },
stdio: ['ignore', 'ignore', 'pipe'],
});
child.stderr?.on('data', (chunk) => {
if (!LOG_RUNTIME_PROCESS_OUTPUT) {
return;
}
const text = String(chunk).trim();
if (text) {
console.warn(`[Browser runtime] ${path.basename(command)}: ${text}`);
}
});
child.on('error', (error) => {
console.warn(`[Browser runtime] ${path.basename(command)} failed:`, error.message);
});
return child;
}
function startXvfbProcess(): Promise<{ child: ReturnType<typeof spawn>; display: string }> {
return new Promise((resolve, reject) => {
const child = spawn('Xvfb', [
'-displayfd',
'3',
'-screen',
'0',
'1440x900x24',
'-ac',
'-nolisten',
'tcp',
], {
env: process.env,
stdio: ['ignore', 'ignore', 'pipe', 'pipe'],
});
let settled = false;
let displayOutput = '';
const displayPipe = child.stdio[3];
const finish = (fn: () => void) => {
if (settled) {
return;
}
settled = true;
clearTimeout(timer);
fn();
};
const timer = setTimeout(() => {
void stopRuntimeProcesses([child]);
finish(() => reject(new Error('Xvfb did not report an available display.')));
}, 5_000);
timer.unref?.();
child.stderr?.on('data', (chunk) => {
if (!LOG_RUNTIME_PROCESS_OUTPUT) {
return;
}
const text = String(chunk).trim();
if (text) {
console.warn(`[Browser runtime] Xvfb: ${text}`);
}
});
child.on('error', (error) => {
console.warn('[Browser runtime] Xvfb failed:', error.message);
finish(() => reject(error));
});
child.on('exit', (code, signal) => {
finish(() => reject(new Error(`Xvfb exited before reporting a display (${signal || code}).`)));
});
displayPipe?.on('data', (chunk) => {
displayOutput += String(chunk);
const match = displayOutput.match(/\d+/);
if (!match) {
return;
}
finish(() => resolve({ child, display: `:${match[0]}` }));
});
});
}
async function startVisibleRuntime(): Promise<NonNullable<RuntimeHandle['viewer']> & { processes: Array<ReturnType<typeof spawn>> }> {
const processes: Array<ReturnType<typeof spawn>> = [];
try {
const xvfb = await startXvfbProcess();
const display = xvfb.display;
processes.push(xvfb.child);
const vncPort = await findAvailablePort();
const websockifyPort = await findAvailablePort();
assertRuntimeProcessesAlive(processes, 'Xvfb');
if (!fs.existsSync(X11VNC_BIN)) {
throw new Error(`x11vnc is missing at ${X11VNC_BIN}.`);
}
processes.push(spawnRuntimeProcess(X11VNC_BIN, [
'-display',
display,
'-localhost',
'-forever',
'-shared',
'-rfbport',
String(vncPort),
'-nopw',
'-quiet',
], {
env: {
LD_LIBRARY_PATH: `${X11VNC_LIB_DIR}:${X11VNC_EXTRA_LIB_DIR}:${process.env.LD_LIBRARY_PATH || ''}`,
},
}));
await waitForRuntimePort(vncPort, 'x11vnc', processes);
if (!fs.existsSync(path.join(NOVNC_ROOT, 'vnc.html'))) {
throw new Error(`noVNC is missing at ${NOVNC_ROOT}.`);
}
processes.push(spawnRuntimeProcess(path.join(os.homedir(), '.local/bin/websockify'), [
'--web',
NOVNC_ROOT,
`127.0.0.1:${websockifyPort}`,
`127.0.0.1:${vncPort}`,
]));
await waitForRuntimePort(websockifyPort, 'websockify', processes);
return {
display,
vncPort,
websockifyPort,
noVncRoot: NOVNC_ROOT,
processes,
};
} catch (error) {
await stopRuntimeProcesses(processes);
throw error;
}
}
const INSTALL_COMMAND_TIMEOUT_MS = Number.parseInt( const INSTALL_COMMAND_TIMEOUT_MS = Number.parseInt(
process.env.CLOUDCLI_BROWSER_USE_INSTALL_TIMEOUT_MS || String(10 * 60 * 1000), process.env.CLOUDCLI_BROWSER_USE_INSTALL_TIMEOUT_MS || String(10 * 60 * 1000),
10, 10,
@@ -354,6 +622,45 @@ function publicSession(session: BrowserUseSession): PublicBrowserUseSession {
return publicFields; return publicFields;
} }
function getSessionViewer(sessionId: string): RuntimeHandle['viewer'] | null {
const session = sessions.get(sessionId);
if (!session || session.ownerId !== AGENT_OWNER_ID || session.status !== 'ready') {
return null;
}
return handles.get(sessionId)?.viewer || null;
}
function createViewerToken(sessionId: string): string {
const token = randomUUID();
viewerTokens.set(sessionId, {
token,
expiresAt: Date.now() + VIEWER_TOKEN_TTL_MS,
});
return token;
}
function deleteViewerToken(sessionId: string) {
viewerTokens.delete(sessionId);
}
function validateViewerTokenForSession(sessionId: string, token: string | null | undefined): boolean {
if (!token) {
return false;
}
const session = sessions.get(sessionId);
const viewer = session?.ownerId === AGENT_OWNER_ID && session.status === 'ready'
? handles.get(sessionId)?.viewer || null
: null;
const stored = viewerTokens.get(sessionId);
if (!viewer || !stored || stored.token !== token || stored.expiresAt < Date.now()) {
if (stored?.expiresAt && stored.expiresAt < Date.now()) {
viewerTokens.delete(sessionId);
}
return false;
}
return true;
}
function ownerSessions(ownerId: string): BrowserUseSession[] { function ownerSessions(ownerId: string): BrowserUseSession[] {
return [...sessions.values()].filter((session) => session.ownerId === ownerId); return [...sessions.values()].filter((session) => session.ownerId === ownerId);
} }
@@ -361,8 +668,10 @@ function ownerSessions(ownerId: string): BrowserUseSession[] {
async function closeHandle(sessionId: string): Promise<void> { async function closeHandle(sessionId: string): Promise<void> {
const handle = handles.get(sessionId); const handle = handles.get(sessionId);
handles.delete(sessionId); handles.delete(sessionId);
deleteViewerToken(sessionId);
await handle?.context?.close?.().catch(() => undefined); await handle?.context?.close?.().catch(() => undefined);
await handle?.browser?.close().catch(() => undefined); await handle?.browser?.close().catch(() => undefined);
await stopRuntimeProcesses(handle?.processes);
} }
async function expireStaleSessions(now = Date.now()): Promise<void> { async function expireStaleSessions(now = Date.now()): Promise<void> {
@@ -428,6 +737,11 @@ export const browserUseService = {
const current = readSettings(); const current = readSettings();
const nextSettings = { const nextSettings = {
enabled: typeof settings.enabled === 'boolean' ? settings.enabled : current.enabled, enabled: typeof settings.enabled === 'boolean' ? settings.enabled : current.enabled,
persistSessions: typeof settings.persistSessions === 'boolean' ? settings.persistSessions : current.persistSessions,
defaultProfileName: typeof settings.defaultProfileName === 'string'
? settings.defaultProfileName
: current.defaultProfileName,
browserBackend: settings.browserBackend ? normalizeBrowserBackend(settings.browserBackend) : current.browserBackend,
}; };
const next = writeSettings(nextSettings); const next = writeSettings(nextSettings);
@@ -443,14 +757,26 @@ export const browserUseService = {
async getStatus() { async getStatus() {
const settings = readSettings(); const settings = readSettings();
const readiness = getRuntimeReadiness(); const readiness = getRuntimeReadiness();
const available = settings.enabled && readiness.playwrightInstalled && readiness.chromiumInstalled; const useVisibleBackend = useVisibleCamoufoxBackend(settings);
const visibleCamoufoxReady = useVisibleBackend
&& readiness.playwrightInstalled
&& isVisibleCamoufoxReady();
const available = settings.enabled
&& readiness.playwrightInstalled
&& (useVisibleBackend ? visibleCamoufoxReady : readiness.chromiumInstalled);
return { return {
enabled: settings.enabled, enabled: settings.enabled,
runtime: getRuntime(), runtime: getRuntime(),
backend: useVisibleBackend ? 'camoufox-vnc' : 'playwright',
browserBackend: settings.browserBackend,
viewerMode: VNC_VIEWER_SUPPORTED ? 'novnc' : 'window',
available, available,
playwrightInstalled: readiness.playwrightInstalled, playwrightInstalled: readiness.playwrightInstalled,
chromiumInstalled: readiness.chromiumInstalled, chromiumInstalled: readiness.chromiumInstalled,
camoufoxInstalled: Boolean(getCamoufoxExecutablePath()),
noVncInstalled: fs.existsSync(path.join(NOVNC_ROOT, 'vnc.html')),
x11vncInstalled: fs.existsSync(X11VNC_BIN),
installInProgress: readiness.installInProgress, installInProgress: readiness.installInProgress,
sessionCount: sessions.size, sessionCount: sessions.size,
message: available message: available
@@ -509,7 +835,7 @@ export const browserUseService = {
} }
await expireStaleSessions(); await expireStaleSessions();
const profileName = normalizeProfileName(options?.profileName); const profileName = resolveSessionProfileName(settings, options?.profileName);
const now = new Date().toISOString(); const now = new Date().toISOString();
const session: BrowserUseSession = { const session: BrowserUseSession = {
@@ -525,6 +851,9 @@ export const browserUseService = {
updatedAt: now, updatedAt: now,
lastAction: 'create', lastAction: 'create',
message: null, message: null,
backend: useVisibleCamoufoxBackend(settings) ? 'camoufox-vnc' : 'playwright',
viewerUrl: null,
viewerEmbedUrl: null,
profileName, profileName,
viewport: { width: 1440, height: 900 }, viewport: { width: 1440, height: 900 },
cursor: null, cursor: null,
@@ -536,7 +865,9 @@ export const browserUseService = {
} }
const readiness = getRuntimeReadiness(); const readiness = getRuntimeReadiness();
if (!settings.enabled || !readiness.playwrightInstalled || !readiness.chromiumInstalled || !readiness.playwright) { const useVisibleBackend = useVisibleCamoufoxBackend(settings);
const visibleCamoufoxReady = useVisibleBackend && isVisibleCamoufoxReady();
if (!settings.enabled || !readiness.playwrightInstalled || !readiness.playwright || (useVisibleBackend ? !visibleCamoufoxReady : !readiness.chromiumInstalled)) {
session.message = getSetupMessage(settings, readiness); session.message = getSetupMessage(settings, readiness);
sessions.set(session.id, session); sessions.set(session.id, session);
return publicSession(session); return publicSession(session);
@@ -545,31 +876,76 @@ export const browserUseService = {
let browser: any | undefined; let browser: any | undefined;
let context: any | undefined; let context: any | undefined;
let page: any; let page: any;
const launchOptions = { let viewer: RuntimeHandle['viewer'];
headless: true, let processes: RuntimeHandle['processes'] = [];
const launchOptions: Record<string, unknown> = {
headless: !useVisibleBackend,
args: ['--disable-dev-shm-usage'], args: ['--disable-dev-shm-usage'],
}; };
const contextOptions = { const contextOptions = useVisibleBackend
viewport: { width: 1440, height: 900 }, ? { viewport: null }
serviceWorkers: 'block', : {
}; viewport: { width: 1440, height: 900 },
serviceWorkers: 'block',
};
if (profileName) { try {
fs.mkdirSync(PROFILE_ROOT, { recursive: true }); if (useVisibleBackend) {
context = await readiness.playwright.chromium.launchPersistentContext(getProfilePath(profileName), { const camoufoxExecutable = getCamoufoxExecutablePath();
...launchOptions, if (!camoufoxExecutable) {
...contextOptions, throw new Error('Camoufox is not installed.');
}); }
page = context.pages()[0] || await context.newPage(); launchOptions.executablePath = camoufoxExecutable;
} else { launchOptions.args = [];
browser = await readiness.playwright.chromium.launch(launchOptions); session.backend = 'camoufox-vnc';
context = await browser.newContext(contextOptions);
page = await context.newPage(); if (VNC_VIEWER_SUPPORTED) {
const runtime = await startVisibleRuntime();
viewer = {
display: runtime.display,
vncPort: runtime.vncPort,
websockifyPort: runtime.websockifyPort,
noVncRoot: runtime.noVncRoot,
};
processes = runtime.processes;
launchOptions.env = {
...process.env,
DISPLAY: runtime.display,
LD_LIBRARY_PATH: `${X11VNC_LIB_DIR}:${X11VNC_EXTRA_LIB_DIR}:${process.env.LD_LIBRARY_PATH || ''}`,
};
const viewerToken = createViewerToken(session.id);
session.viewerUrl = getViewerUrl(session.id, viewerToken);
session.viewerEmbedUrl = session.viewerUrl;
} else {
// Without a VNC pipeline the browser opens as a normal window on the host desktop.
session.message = 'Browser window is open on the machine running CloudCLI.';
}
}
if (profileName) {
fs.mkdirSync(PROFILE_ROOT, { recursive: true });
const browserType = useVisibleBackend ? readiness.playwright.firefox : readiness.playwright.chromium;
context = await browserType.launchPersistentContext(getProfilePath(profileName), {
...launchOptions,
...contextOptions,
});
page = context.pages()[0] || await context.newPage();
} else {
const browserType = useVisibleBackend ? readiness.playwright.firefox : readiness.playwright.chromium;
browser = await browserType.launch(launchOptions);
context = await browser.newContext(contextOptions);
page = await context.newPage();
}
} catch (error) {
await context?.close?.().catch(() => undefined);
await browser?.close?.().catch(() => undefined);
await stopRuntimeProcesses(processes);
throw error;
} }
session.status = 'ready'; session.status = 'ready';
session.message = 'Browser session is ready.'; session.message = session.message || 'Browser session is ready.';
sessions.set(session.id, session); sessions.set(session.id, session);
handles.set(session.id, { browser, context, page }); handles.set(session.id, { browser, context, page, processes, viewer });
await captureSession(session, page); await captureSession(session, page);
return publicSession(session); return publicSession(session);
}, },
@@ -816,6 +1192,25 @@ export const browserUseService = {
return { deleted: true, sessionId }; return { deleted: true, sessionId };
}, },
getViewerProxyTarget(sessionId: string) {
const viewer = getSessionViewer(sessionId);
if (!viewer) {
throw new Error('Browser viewer is not available for this session.');
}
return {
websockifyPort: viewer.websockifyPort,
noVncRoot: viewer.noVncRoot,
};
},
validateViewerToken(sessionId: string, token: string | null | undefined) {
return validateViewerTokenForSession(sessionId, token);
},
handleViewerWebSocket(clientWs: WebSocket, pathname: string) {
handleViewerWebSocket(clientWs, pathname, getSessionViewer);
},
async agentStopSession(sessionId: string) { async agentStopSession(sessionId: string) {
await this.getAgentSession(sessionId); await this.getAgentSession(sessionId);
return this.stopSession(sessionId); return this.stopSession(sessionId);

View File

@@ -0,0 +1,147 @@
import { randomBytes } from 'node:crypto';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { appConfigDb } from '@/modules/database/index.js';
import type { BrowserUseBackend, BrowserUseSettings } from './browser-use.types.js';
const IS_PLATFORM = process.env.VITE_IS_PLATFORM === 'true';
const BROWSER_USE_SETTINGS_KEY = 'browser_use_settings';
const BROWSER_USE_MCP_TOKEN_KEY = 'browser_use_mcp_token';
const MAX_PROFILE_NAME_LENGTH = 80;
export const DEFAULT_BROWSER_USE_SETTINGS: BrowserUseSettings = {
enabled: false,
persistSessions: false,
defaultProfileName: 'default',
browserBackend: IS_PLATFORM ? 'camoufox-vnc' : 'playwright',
};
export const PROFILE_ROOT = process.env.CLOUDCLI_BROWSER_USE_PROFILE_ROOT
|| path.join(os.homedir(), '.cloudcli', 'browser-use', 'profiles');
export function normalizeBrowserBackend(value: unknown): BrowserUseBackend {
return value === 'playwright' || value === 'camoufox-vnc'
? value
: DEFAULT_BROWSER_USE_SETTINGS.browserBackend;
}
function trimEdgeDashes(value: string): string {
let start = 0;
let end = value.length;
while (start < end && value[start] === '-') {
start += 1;
}
while (end > start && value[end - 1] === '-') {
end -= 1;
}
return value.slice(start, end);
}
export function normalizeProfileName(profileName?: string | null): string | null {
const sanitized = trimEdgeDashes(String(profileName || '')
.trim()
.toLowerCase()
.replace(/[^a-z0-9._-]+/g, '-'));
const normalized = sanitized
.slice(0, MAX_PROFILE_NAME_LENGTH)
.replace(/^-+|-+$/g, '');
if (!normalized) {
return null;
}
return /[a-z0-9]/.test(normalized) ? normalized : null;
}
export function normalizeDefaultProfileName(profileName?: string | null): string {
return normalizeProfileName(profileName) || DEFAULT_BROWSER_USE_SETTINGS.defaultProfileName;
}
export function resolveSessionProfileName(settings: BrowserUseSettings, profileName?: string | null): string | null {
const requestedProfileName = normalizeProfileName(profileName);
if (String(profileName || '').trim() && !requestedProfileName) {
throw new Error('Browser profile name must include at least one letter or number.');
}
if (requestedProfileName) {
validateRequestedProfileName(profileName, requestedProfileName);
return requestedProfileName;
}
return settings.persistSessions ? normalizeDefaultProfileName(settings.defaultProfileName) : null;
}
export function getProfilePath(profileName: string): string {
return path.join(PROFILE_ROOT, normalizeDefaultProfileName(profileName));
}
function validateRequestedProfileName(profileName: string | null | undefined, normalizedProfileName: string): void {
const requestedProfileName = String(profileName || '').trim();
const existingProfileName = findExistingProfileName(normalizedProfileName);
if (
existingProfileName
&& (requestedProfileName !== normalizedProfileName || existingProfileName !== normalizedProfileName)
) {
throw new Error(`Browser profile "${requestedProfileName}" resolves to existing profile "${existingProfileName}". Use "${normalizedProfileName}" instead.`);
}
}
function findExistingProfileName(normalizedProfileName: string): string | null {
try {
if (!fs.existsSync(PROFILE_ROOT)) {
return null;
}
const entries = fs.readdirSync(PROFILE_ROOT, { withFileTypes: true });
const match = entries.find((entry) => entry.isDirectory() && normalizeProfileName(entry.name) === normalizedProfileName);
return match?.name || null;
} catch {
return null;
}
}
export function useVisibleCamoufoxBackend(settings: BrowserUseSettings): boolean {
return settings.browserBackend === 'camoufox-vnc';
}
export function readSettings(): BrowserUseSettings {
try {
const raw = appConfigDb.get(BROWSER_USE_SETTINGS_KEY);
if (!raw) {
return DEFAULT_BROWSER_USE_SETTINGS;
}
const parsed = JSON.parse(raw) as Partial<BrowserUseSettings>;
return {
enabled: parsed.enabled === true,
persistSessions: parsed.persistSessions === true,
defaultProfileName: normalizeDefaultProfileName(parsed.defaultProfileName),
browserBackend: normalizeBrowserBackend(parsed.browserBackend),
};
} catch (error: any) {
console.warn('[Browser] Failed to read settings:', error?.message || error);
return DEFAULT_BROWSER_USE_SETTINGS;
}
}
export function writeSettings(settings: BrowserUseSettings): BrowserUseSettings {
const normalized = {
enabled: settings.enabled === true,
persistSessions: settings.persistSessions === true,
defaultProfileName: normalizeDefaultProfileName(settings.defaultProfileName),
browserBackend: normalizeBrowserBackend(settings.browserBackend),
};
appConfigDb.set(BROWSER_USE_SETTINGS_KEY, JSON.stringify(normalized));
return normalized;
}
export function getOrCreateMcpToken(): string {
const existing = appConfigDb.get(BROWSER_USE_MCP_TOKEN_KEY);
if (existing) {
return existing;
}
const token = randomBytes(32).toString('hex');
appConfigDb.set(BROWSER_USE_MCP_TOKEN_KEY, token);
return token;
}

View File

@@ -0,0 +1,66 @@
import type { spawn } from 'node:child_process';
export type BrowserUseRuntime = 'cloud' | 'local';
export type BrowserUseBackend = 'playwright' | 'camoufox-vnc';
export type BrowserUseSessionStatus = 'ready' | 'stopped' | 'unavailable';
export type BrowserUseSession = {
id: string;
ownerId: string;
createdBy: 'agent';
runtime: BrowserUseRuntime;
status: BrowserUseSessionStatus;
url: string | null;
title: string | null;
screenshotDataUrl: string | null;
createdAt: string;
updatedAt: string;
lastAction: string | null;
message: string | null;
backend: BrowserUseBackend;
viewerUrl: string | null;
viewerEmbedUrl: string | null;
profileName: string | null;
viewport: {
width: number;
height: number;
} | null;
cursor: {
x: number;
y: number;
actor: 'agent';
} | null;
};
export type PublicBrowserUseSession = Omit<BrowserUseSession, 'ownerId'>;
export type RuntimeHandle = {
browser?: any;
context?: any;
page?: any;
processes?: Array<ReturnType<typeof spawn>>;
viewer?: {
display: string;
vncPort: number;
websockifyPort: number;
noVncRoot: string;
};
};
export type BrowserUseSettings = {
enabled: boolean;
persistSessions: boolean;
defaultProfileName: string;
browserBackend: BrowserUseBackend;
};
export type RuntimeReadiness = {
playwright: any | null;
playwrightInstalled: boolean;
chromiumInstalled: boolean;
chromiumExecutablePath: string | null;
installInProgress: boolean;
installMessage: string | null;
};
export type RuntimeProbe = Omit<RuntimeReadiness, 'installInProgress' | 'installMessage'>;

View File

@@ -0,0 +1,76 @@
import { WebSocket } from 'ws';
import type { RuntimeHandle } from './browser-use.types.js';
type BrowserUseViewer = NonNullable<RuntimeHandle['viewer']>;
export const VIEWER_COOKIE_NAME = 'browser_use_viewer_token';
const DEFAULT_VIEWER_TOKEN_TTL_MS = 30 * 60 * 1000;
const parsedViewerTokenTtlMs = Number.parseInt(
process.env.CLOUDCLI_BROWSER_USE_VIEWER_TOKEN_TTL_MS || String(DEFAULT_VIEWER_TOKEN_TTL_MS),
10,
);
export const VIEWER_TOKEN_TTL_MS =
Number.isFinite(parsedViewerTokenTtlMs) && parsedViewerTokenTtlMs > 0
? parsedViewerTokenTtlMs
: DEFAULT_VIEWER_TOKEN_TTL_MS;
export function getViewerUrl(sessionId: string, viewerToken?: string): string {
const basePath = `/api/browser-use/sessions/${encodeURIComponent(sessionId)}/viewer`;
const websockifyPath = viewerToken
? `${basePath}/websockify?viewerToken=${encodeURIComponent(viewerToken)}`
: `${basePath}/websockify`;
const params = new URLSearchParams({
autoconnect: '1',
resize: 'scale',
reconnect: '1',
path: websockifyPath,
});
if (viewerToken) {
params.set('viewerToken', viewerToken);
}
return `${basePath}/vnc.html?${params.toString()}`;
}
export function handleViewerWebSocket(
clientWs: WebSocket,
pathname: string,
getSessionViewer: (sessionId: string) => BrowserUseViewer | null | undefined,
) {
const match = /^\/api\/browser-use\/sessions\/([^/]+)\/viewer\/websockify\/?$/.exec(pathname);
const sessionId = match ? decodeURIComponent(match[1]) : '';
const viewer = sessionId ? getSessionViewer(sessionId) : null;
if (!viewer) {
clientWs.close(4404, 'Browser viewer not found');
return;
}
const upstream = new WebSocket(`ws://127.0.0.1:${viewer.websockifyPort}`);
upstream.on('open', () => {
clientWs.on('message', (data) => {
if (upstream.readyState === WebSocket.OPEN) {
upstream.send(data);
}
});
upstream.on('message', (data) => {
if (clientWs.readyState === WebSocket.OPEN) {
clientWs.send(data);
}
});
});
upstream.on('close', (code, reason) => {
if (clientWs.readyState === WebSocket.OPEN) {
clientWs.close(code, reason);
}
});
upstream.on('error', () => {
if (clientWs.readyState === WebSocket.OPEN) {
clientWs.close(4502, 'Browser viewer upstream error');
}
});
clientWs.on('close', () => {
if (upstream.readyState === WebSocket.OPEN) {
upstream.close();
}
});
}

View File

@@ -0,0 +1,2 @@
export { browserUseService } from './browser-use.service.js';
export { VIEWER_COOKIE_NAME } from './browser-use.viewer.js';

View File

@@ -0,0 +1,73 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import test from 'node:test';
const originalProfileRoot = process.env.CLOUDCLI_BROWSER_USE_PROFILE_ROOT;
const testProfileRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'browser-use-profiles-'));
process.env.CLOUDCLI_BROWSER_USE_PROFILE_ROOT = testProfileRoot;
const {
getProfilePath,
normalizeDefaultProfileName,
normalizeProfileName,
PROFILE_ROOT,
resolveSessionProfileName,
} = await import('@/modules/browser-use/browser-use.settings.js');
test.after(() => {
if (originalProfileRoot === undefined) {
delete process.env.CLOUDCLI_BROWSER_USE_PROFILE_ROOT;
} else {
process.env.CLOUDCLI_BROWSER_USE_PROFILE_ROOT = originalProfileRoot;
}
fs.rmSync(testProfileRoot, { recursive: true, force: true });
});
test('browser profile names are canonicalized before storage and path resolution', () => {
assert.equal(normalizeProfileName(' Work Profile!! '), 'work-profile');
assert.equal(normalizeProfileName(`${'-'.repeat(100)}Work Profile`), 'work-profile');
assert.equal(normalizeDefaultProfileName(' Work Profile!! '), 'work-profile');
assert.equal(
getProfilePath(' Work Profile!! '),
`${PROFILE_ROOT}/work-profile`,
);
assert.equal(
resolveSessionProfileName({
enabled: true,
persistSessions: true,
defaultProfileName: ' Work Profile!! ',
browserBackend: 'playwright',
}),
'work-profile',
);
});
test('browser profile aliases are rejected when the normalized profile already exists', () => {
const profileName = `alias-test-${Date.now()}`;
fs.mkdirSync(getProfilePath(profileName), { recursive: true });
try {
assert.throws(
() => resolveSessionProfileName({
enabled: true,
persistSessions: false,
defaultProfileName: 'default',
browserBackend: 'playwright',
}, profileName.toUpperCase()),
/resolves to existing profile/,
);
assert.equal(
resolveSessionProfileName({
enabled: true,
persistSessions: false,
defaultProfileName: 'default',
browserBackend: 'playwright',
}, profileName),
profileName,
);
} finally {
fs.rmSync(getProfilePath(profileName), { recursive: true, force: true });
}
});

View File

@@ -1,8 +1,9 @@
import type { Server as HttpServer } from 'node:http'; import type { Server as HttpServer } from 'node:http';
import { WebSocketServer, type VerifyClientCallbackSync } from 'ws'; import { WebSocket, WebSocketServer, type VerifyClientCallbackSync } from 'ws';
import { handleChatConnection } from '@/modules/websocket/services/chat-websocket.service.js'; import { handleChatConnection } from '@/modules/websocket/services/chat-websocket.service.js';
import { VIEWER_COOKIE_NAME } from '@/modules/browser-use/index.js';
import { verifyWebSocketClient } from '@/modules/websocket/services/websocket-auth.service.js'; import { verifyWebSocketClient } from '@/modules/websocket/services/websocket-auth.service.js';
import { handlePluginWsProxy } from '@/modules/websocket/services/plugin-websocket-proxy.service.js'; import { handlePluginWsProxy } from '@/modules/websocket/services/plugin-websocket-proxy.service.js';
import { handleShellConnection } from '@/modules/websocket/services/shell-websocket.service.js'; import { handleShellConnection } from '@/modules/websocket/services/shell-websocket.service.js';
@@ -14,8 +15,21 @@ type WebSocketServerDependencies = {
chat: Parameters<typeof handleChatConnection>[2]; chat: Parameters<typeof handleChatConnection>[2];
shell: Parameters<typeof handleShellConnection>[1]; shell: Parameters<typeof handleShellConnection>[1];
getPluginPort: Parameters<typeof handlePluginWsProxy>[2]; getPluginPort: Parameters<typeof handlePluginWsProxy>[2];
browserUseViewer?: (ws: WebSocket, pathname: string) => void;
authenticateBrowserUseViewer?: (pathname: string, token: string | null) => boolean;
}; };
function readCookieValue(header: unknown, name: string): string | null {
if (!header) return null;
const prefix = `${name}=`;
const cookie = String(header).split(';').map((part) => part.trim()).find((part) => part.startsWith(prefix));
return cookie ? decodeURIComponent(cookie.slice(prefix.length)) : null;
}
function getBrowserUseViewerToken(url: URL, headers: Record<string, unknown>): string | null {
return url.searchParams.get('viewerToken') || readCookieValue(headers.cookie, VIEWER_COOKIE_NAME);
}
/** /**
* Creates and wires the server-wide websocket gateway used for chat, shell, and * Creates and wires the server-wide websocket gateway used for chat, shell, and
* plugin proxy routes. * plugin proxy routes.
@@ -28,7 +42,17 @@ export function createWebSocketServer(
server, server,
verifyClient: (( verifyClient: ((
info: Parameters<VerifyClientCallbackSync<AuthenticatedWebSocketRequest>>[0] info: Parameters<VerifyClientCallbackSync<AuthenticatedWebSocketRequest>>[0]
) => verifyWebSocketClient(info, dependencies.verifyClient)), ) => {
const requestUrl = new URL(info.req.url ?? '/', 'http://localhost');
if (
requestUrl.pathname.startsWith('/api/browser-use/sessions/')
&& requestUrl.pathname.endsWith('/viewer/websockify')
) {
const token = getBrowserUseViewerToken(requestUrl, info.req.headers as Record<string, unknown>);
return Boolean(dependencies.authenticateBrowserUseViewer?.(requestUrl.pathname, token));
}
return verifyWebSocketClient(info, dependencies.verifyClient);
}),
}); });
wss.on('connection', (ws, request) => { wss.on('connection', (ws, request) => {
@@ -74,6 +98,11 @@ export function createWebSocketServer(
return; return;
} }
if (pathname.startsWith('/api/browser-use/sessions/') && pathname.endsWith('/viewer/websockify')) {
dependencies.browserUseViewer?.(ws, pathname);
return;
}
console.log('[WARN] Unknown WebSocket path:', pathname); console.log('[WARN] Unknown WebSocket path:', pathname);
ws.close(); ws.close();
}); });

View File

@@ -1,4 +1,4 @@
import { useCallback, useEffect, useMemo, useState } from 'react'; import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { import {
Bot, Bot,
Clock3, Clock3,
@@ -7,6 +7,7 @@ import {
ExternalLink, ExternalLink,
Loader2, Loader2,
MonitorPlay, MonitorPlay,
MousePointer2,
RefreshCw, RefreshCw,
Settings, Settings,
Square, Square,
@@ -19,9 +20,15 @@ import { Badge, Button } from '../../../shared/view/ui';
import { authenticatedFetch } from '../../../utils/api'; import { authenticatedFetch } from '../../../utils/api';
import type { SettingsMainTab } from '../../settings/types/types'; import type { SettingsMainTab } from '../../settings/types/types';
const BROWSER_USE_GUIDE_URL = 'https://cloudcli.ai/docs/browser-use';
const BROWSER_USE_CACHE_TTL_MS = 30_000;
type BrowserUseStatus = { type BrowserUseStatus = {
enabled: boolean; enabled: boolean;
available: boolean; available: boolean;
backend: 'playwright' | 'camoufox-vnc';
browserBackend: 'playwright' | 'camoufox-vnc';
viewerMode?: 'novnc' | 'window';
playwrightInstalled: boolean; playwrightInstalled: boolean;
chromiumInstalled: boolean; chromiumInstalled: boolean;
installInProgress: boolean; installInProgress: boolean;
@@ -39,6 +46,9 @@ type BrowserUseSession = {
updatedAt: string; updatedAt: string;
lastAction: string | null; lastAction: string | null;
message: string | null; message: string | null;
backend?: 'playwright' | 'camoufox-vnc';
viewerUrl?: string | null;
viewerEmbedUrl?: string | null;
createdBy: 'agent'; createdBy: 'agent';
profileName: string | null; profileName: string | null;
viewport: { viewport: {
@@ -54,17 +64,48 @@ type BrowserUseSession = {
type BrowserUsePanelProps = { type BrowserUsePanelProps = {
isVisible: boolean; isVisible: boolean;
projectId?: string | null;
onShowSettings?: (tab?: SettingsMainTab) => void; onShowSettings?: (tab?: SettingsMainTab) => void;
}; };
type BrowserUsePanelCacheEntry = {
status: BrowserUseStatus | null;
sessions: BrowserUseSession[];
selectedSessionId: string | null;
updatedAt: number;
};
const browserUsePanelCache = new Map<string, BrowserUsePanelCacheEntry>();
async function readJson<T>(response: Response): Promise<T> { async function readJson<T>(response: Response): Promise<T> {
const data = await response.json(); const text = await response.text();
let data: any = {};
if (text) {
try {
data = JSON.parse(text);
} catch {
throw new Error(response.ok ? 'Received an invalid Browser response.' : `Browser request failed (${response.status}).`);
}
}
if (!response.ok || data.success === false) { if (!response.ok || data.success === false) {
throw new Error(data.error || data.details || `Request failed (${response.status})`); throw new Error(data.error || data.details || `Request failed (${response.status})`);
} }
return data as T; return data as T;
} }
async function fetchBrowserPanelData() {
const [statusResponse, sessionsResponse] = await Promise.all([
authenticatedFetch('/api/browser-use/status'),
authenticatedFetch('/api/browser-use/sessions'),
]);
const statusData = await readJson<{ data: BrowserUseStatus }>(statusResponse);
const sessionsData = await readJson<{ data: { sessions: BrowserUseSession[] } }>(sessionsResponse);
return {
status: statusData.data,
sessions: [...sessionsData.data.sessions].sort((a, b) => Date.parse(b.createdAt) - Date.parse(a.createdAt)),
};
}
function formatRelativeTime(value: string | null): string { function formatRelativeTime(value: string | null): string {
if (!value) return 'Never'; if (!value) return 'Never';
@@ -119,20 +160,42 @@ function getStatusDot(status: BrowserUseSession['status']): string {
return 'bg-border'; return 'bg-border';
} }
function getEngineLabel(backend?: BrowserUseStatus['backend'] | BrowserUseSession['backend']): string {
return backend === 'camoufox-vnc' ? 'Co-browse' : 'Playwright';
}
const PROMPTS = [ const PROMPTS = [
'Use Browser to inspect the checkout flow and report any broken UI states.', 'Use Browser to inspect the checkout flow and report any broken UI states.',
'Open <url> with Browser, interact with the page, and summarize what changed after each step.', 'Open <url> with Browser, interact with the page, and summarize what changed after each step.',
]; ];
export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUsePanelProps) { function getBrowserUseCacheKey(projectId?: string | null): string {
const [status, setStatus] = useState<BrowserUseStatus | null>(null); return projectId ? `browser-use:project:${projectId}` : 'browser-use:global';
const [sessions, setSessions] = useState<BrowserUseSession[]>([]); }
const [selectedSessionId, setSelectedSessionId] = useState<string | null>(null);
function getFreshCacheEntry(cacheKey: string): BrowserUsePanelCacheEntry | null {
const entry = browserUsePanelCache.get(cacheKey);
if (!entry || Date.now() - entry.updatedAt > BROWSER_USE_CACHE_TTL_MS) {
return null;
}
return entry;
}
export default function BrowserUsePanel({ isVisible, projectId, onShowSettings }: BrowserUsePanelProps) {
const cacheKey = getBrowserUseCacheKey(projectId);
const initialCacheEntry = getFreshCacheEntry(cacheKey);
const [status, setStatus] = useState<BrowserUseStatus | null>(() => initialCacheEntry?.status ?? null);
const [sessions, setSessions] = useState<BrowserUseSession[]>(() => initialCacheEntry?.sessions ?? []);
const [selectedSessionId, setSelectedSessionId] = useState<string | null>(() => (
initialCacheEntry?.selectedSessionId || initialCacheEntry?.sessions[0]?.id || null
));
const [hasLoadedOnce, setHasLoadedOnce] = useState(Boolean(initialCacheEntry));
const [isRefreshing, setIsRefreshing] = useState(false); const [isRefreshing, setIsRefreshing] = useState(false);
const [isBusy, setIsBusy] = useState(false); const [isBusy, setIsBusy] = useState(false);
const [isInstalling, setIsInstalling] = useState(false); const [isInstalling, setIsInstalling] = useState(false);
const [isFullscreen, setIsFullscreen] = useState(false); const [isFullscreen, setIsFullscreen] = useState(false);
const [error, setError] = useState<string | null>(null); const [error, setError] = useState<string | null>(null);
const activeLoadIdRef = useRef(0);
const selectedSession = useMemo( const selectedSession = useMemo(
() => sessions.find((session) => session.id === selectedSessionId) || sessions[0] || null, () => sessions.find((session) => session.id === selectedSessionId) || sessions[0] || null,
@@ -140,8 +203,14 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
); );
const activeSessions = sessions.filter((session) => session.status === 'ready'); const activeSessions = sessions.filter((session) => session.status === 'ready');
const needsBrowserBinaries = Boolean(status?.enabled && (!status.playwrightInstalled || !status.chromiumInstalled)); const isInitialLoading = isRefreshing && !hasLoadedOnce && sessions.length === 0;
const runtimeLabel = !status?.enabled const isBackgroundRefreshing = isRefreshing && !isInitialLoading;
const needsBrowserBinaries = Boolean(status?.enabled && !status.available);
const usesLocalWindowViewer = status?.viewerMode === 'window';
const panelTitle = status?.browserBackend === 'camoufox-vnc' ? 'Co-browse' : 'Browser';
const runtimeLabel = isInitialLoading
? 'Loading'
: !status?.enabled
? 'Disabled' ? 'Disabled'
: status.available : status.available
? 'Ready' ? 'Ready'
@@ -157,29 +226,72 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
: null; : null;
const refresh = useCallback(async () => { const refresh = useCallback(async () => {
const loadId = activeLoadIdRef.current + 1;
activeLoadIdRef.current = loadId;
setIsRefreshing(true); setIsRefreshing(true);
try { try {
const [statusResponse, sessionsResponse] = await Promise.all([ let nextData: Awaited<ReturnType<typeof fetchBrowserPanelData>>;
authenticatedFetch('/api/browser-use/status'), try {
authenticatedFetch('/api/browser-use/sessions'), nextData = await fetchBrowserPanelData();
]); } catch (error) {
const statusData = await readJson<{ data: BrowserUseStatus }>(statusResponse); if (loadId !== activeLoadIdRef.current) {
const sessionsData = await readJson<{ data: { sessions: BrowserUseSession[] } }>(sessionsResponse); return;
const nextSessions = sessionsData.data.sessions; }
setStatus(statusData.data); await new Promise((resolve) => setTimeout(resolve, 350));
nextData = await fetchBrowserPanelData();
}
if (activeLoadIdRef.current !== loadId) {
return;
}
const nextSessions = nextData.sessions;
setStatus(nextData.status);
setSessions(nextSessions); setSessions(nextSessions);
setSelectedSessionId((current) => ( setHasLoadedOnce(true);
current && nextSessions.some((session) => session.id === current) let nextSelectedSessionId: string | null = null;
setSelectedSessionId((current) => {
nextSelectedSessionId = current && nextSessions.some((session) => session.id === current)
? current ? current
: nextSessions[0]?.id || null : nextSessions[0]?.id || null;
)); return nextSelectedSessionId;
});
browserUsePanelCache.set(cacheKey, {
status: nextData.status,
sessions: nextSessions,
selectedSessionId: nextSelectedSessionId,
updatedAt: Date.now(),
});
setError(null); setError(null);
} catch (err) { } catch (err) {
if (activeLoadIdRef.current !== loadId) {
return;
}
setHasLoadedOnce(true);
setError(err instanceof Error ? err.message : 'Failed to load Browser'); setError(err instanceof Error ? err.message : 'Failed to load Browser');
} finally { } finally {
setIsRefreshing(false); if (activeLoadIdRef.current === loadId) {
setIsRefreshing(false);
}
} }
}, []); }, [cacheKey]);
useEffect(() => {
const cachedEntry = browserUsePanelCache.get(cacheKey);
if (!cachedEntry) return;
browserUsePanelCache.set(cacheKey, {
...cachedEntry,
selectedSessionId,
});
}, [cacheKey, selectedSessionId]);
useEffect(() => {
const cachedEntry = getFreshCacheEntry(cacheKey);
setStatus(cachedEntry?.status ?? null);
setSessions(cachedEntry?.sessions ?? []);
setSelectedSessionId(cachedEntry?.selectedSessionId || cachedEntry?.sessions[0]?.id || null);
setHasLoadedOnce(Boolean(cachedEntry));
setError(null);
activeLoadIdRef.current += 1;
}, [cacheKey]);
useEffect(() => { useEffect(() => {
if (!isVisible) return; if (!isVisible) return;
@@ -253,6 +365,10 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
<span>{formatRelativeTime(session.updatedAt)}</span> <span>{formatRelativeTime(session.updatedAt)}</span>
<span className="truncate">- {formatAction(session.lastAction)}</span> <span className="truncate">- {formatAction(session.lastAction)}</span>
</div> </div>
<div className="mt-2 flex flex-wrap gap-1.5 pl-3.5 text-[10px] text-muted-foreground">
<span className="rounded border border-border/70 bg-background/70 px-1.5 py-0.5">{getEngineLabel(session.backend)}</span>
<span className="rounded border border-border/70 bg-background/70 px-1.5 py-0.5">{session.profileName || 'Temporary'}</span>
</div>
</button> </button>
); );
}; };
@@ -270,9 +386,18 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
</div> </div>
<p className="mt-1 max-w-xl text-sm leading-6 text-muted-foreground"> <p className="mt-1 max-w-xl text-sm leading-6 text-muted-foreground">
{status?.enabled {status?.enabled
? 'Agent browser sessions appear here while an AI task is using Browser.' ? 'When an agent opens a browser, you can watch the latest screenshot, take control in a new tab, or end the running session.'
: 'Enable Browser in settings to let agents open monitored browser sessions.'} : 'Enable Browser to let agents open websites, test flows, capture screenshots, and debug UI from a real page.'}
</p> </p>
<a
href={BROWSER_USE_GUIDE_URL}
target="_blank"
rel="noopener noreferrer"
className="mt-2 inline-flex items-center gap-1.5 text-sm font-medium text-primary hover:underline"
>
Read the Browser guide
<ExternalLink className="h-3.5 w-3.5" />
</a>
</div> </div>
</div> </div>
@@ -312,10 +437,19 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
</div> </div>
); );
const renderLoadingState = () => (
<div className="flex min-h-0 flex-1 items-center justify-center p-6">
<div className="flex items-center gap-3 rounded-md border border-border bg-card/40 px-4 py-3 text-sm text-muted-foreground shadow-sm">
<Loader2 className="h-4 w-4 animate-spin text-primary" />
Loading browser sessions...
</div>
</div>
);
const renderBrowserSurface = (fullscreen = false) => ( const renderBrowserSurface = (fullscreen = false) => (
<div className={cn('flex flex-1 items-center justify-center bg-neutral-950', fullscreen ? 'min-h-[80vh]' : 'min-h-[420px]')}> <div className={cn('flex flex-1 items-center justify-center bg-neutral-950', fullscreen ? 'min-h-[80vh]' : 'min-h-[420px]')}>
{selectedSession?.screenshotDataUrl ? ( {selectedSession?.screenshotDataUrl ? (
<div className="relative inline-block max-h-full"> <div className="group relative inline-block max-h-full">
<img <img
src={selectedSession.screenshotDataUrl} src={selectedSession.screenshotDataUrl}
alt="Browser session screenshot" alt="Browser session screenshot"
@@ -329,6 +463,18 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
<div className="absolute left-1/2 top-1/2 h-2 w-2 -translate-x-1/2 -translate-y-1/2 rounded-full bg-white" /> <div className="absolute left-1/2 top-1/2 h-2 w-2 -translate-x-1/2 -translate-y-1/2 rounded-full bg-white" />
</div> </div>
)} )}
{selectedSession?.viewerEmbedUrl && selectedSession.status === 'ready' && (
<button
type="button"
onClick={() => window.open(selectedSession.viewerUrl || selectedSession.viewerEmbedUrl || '', '_blank', 'noopener,noreferrer')}
className="absolute inset-0 flex items-center justify-center bg-black/0 opacity-0 transition focus-visible:bg-black/30 focus-visible:opacity-100 focus-visible:outline-none group-hover:bg-black/30 group-hover:opacity-100"
>
<span className="inline-flex items-center gap-2 rounded-md border border-white/20 bg-black/80 px-3 py-2 text-sm font-medium text-white shadow-lg">
<MousePointer2 className="h-4 w-4" />
Take control
</span>
</button>
)}
</div> </div>
) : ( ) : (
<div className="px-6 text-center"> <div className="px-6 text-center">
@@ -346,14 +492,37 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
<div className="min-w-0"> <div className="min-w-0">
<div className="flex items-center gap-2"> <div className="flex items-center gap-2">
<MonitorPlay className="h-4 w-4 text-primary" /> <MonitorPlay className="h-4 w-4 text-primary" />
<h3 className="text-sm font-semibold text-foreground">Browser</h3> <h3 className="text-sm font-semibold text-foreground">{panelTitle}</h3>
<Badge variant="outline" className={cn('text-[10px]', getRuntimeTone(status, isInstalling))}> <Badge variant="outline" className={cn('text-[10px]', getRuntimeTone(status, isInstalling))}>
{runtimeLabel} {runtimeLabel}
</Badge> </Badge>
<Badge variant="outline" className="border-border bg-background text-[10px] text-muted-foreground">
{getEngineLabel(status?.backend)}
</Badge>
</div> </div>
<p className="mt-0.5 text-xs text-muted-foreground">Monitor browser sessions opened by AI agents.</p> <p className="mt-0.5 text-xs text-muted-foreground">
{status?.browserBackend === 'camoufox-vnc'
? 'Watch and manage co-browse sessions agents use to test real websites.'
: 'Watch and manage browser sessions agents use to test real websites.'}
</p>
{isBackgroundRefreshing && (
<div className="mt-1 flex items-center gap-1.5 text-xs text-muted-foreground">
<RefreshCw className="h-3 w-3 animate-spin" />
Refreshing sessions...
</div>
)}
</div> </div>
<div className="flex items-center gap-1.5"> <div className="flex items-center gap-1.5">
<Button
variant="ghost"
size="sm"
className="h-7 w-7 p-0"
onClick={() => window.open(BROWSER_USE_GUIDE_URL, '_blank', 'noopener,noreferrer')}
title="Open Browser guide"
aria-label="Open Browser guide"
>
<ExternalLink className="h-3.5 w-3.5" />
</Button>
{onShowSettings && ( {onShowSettings && (
<Button <Button
variant="ghost" variant="ghost"
@@ -425,7 +594,7 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
</div> </div>
{sessions.length === 0 ? ( {sessions.length === 0 ? (
renderEmptyState() isInitialLoading ? renderLoadingState() : renderEmptyState()
) : ( ) : (
<div className="min-h-0 flex-1 overflow-auto bg-muted/20 p-4"> <div className="min-h-0 flex-1 overflow-auto bg-muted/20 p-4">
<div className="mx-auto flex min-h-[500px] max-w-7xl flex-col overflow-hidden rounded-md border border-border bg-background shadow-sm"> <div className="mx-auto flex min-h-[500px] max-w-7xl flex-col overflow-hidden rounded-md border border-border bg-background shadow-sm">
@@ -441,14 +610,40 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
<ExternalLink className="h-3.5 w-3.5 shrink-0" /> <ExternalLink className="h-3.5 w-3.5 shrink-0" />
<span className="truncate">{selectedSession?.url || 'No page loaded'}</span> <span className="truncate">{selectedSession?.url || 'No page loaded'}</span>
</div> </div>
<div className="mt-1 flex flex-wrap gap-1.5 text-[10px] text-muted-foreground">
<span className="rounded border border-border/70 bg-muted/30 px-1.5 py-0.5">{getEngineLabel(selectedSession?.backend || status?.backend)}</span>
<span className="rounded border border-border/70 bg-muted/30 px-1.5 py-0.5">Profile: {selectedSession?.profileName || 'Temporary'}</span>
<span className="rounded border border-border/70 bg-muted/30 px-1.5 py-0.5">Updated {formatRelativeTime(selectedSession?.updatedAt || null)}</span>
</div>
</div> </div>
<div className="hidden text-xs text-muted-foreground md:block"> <div className="hidden text-xs text-muted-foreground md:block">
{formatAction(selectedSession?.lastAction || null)} {formatAction(selectedSession?.lastAction || null)}
</div> </div>
{selectedSession?.viewerUrl && selectedSession.status === 'ready' && (
<Button
variant="outline"
size="sm"
className="h-8"
onClick={() => window.open(selectedSession.viewerUrl || '', '_blank', 'noopener,noreferrer')}
title="Open live browser control in a new tab"
aria-label="Open live browser control in a new tab"
>
<MousePointer2 className="h-4 w-4" />
Take control
</Button>
)}
{usesLocalWindowViewer && selectedSession?.backend === 'camoufox-vnc' && !selectedSession.viewerUrl && selectedSession.status === 'ready' && (
<span
className="hidden rounded border border-border/70 bg-muted/30 px-2 py-1 text-[10px] text-muted-foreground md:inline"
title="This visible session runs as a browser window on the machine running CloudCLI"
>
Window on host
</span>
)}
<Button variant="ghost" size="sm" className="h-8 w-8 p-0" onClick={() => setIsFullscreen(true)} disabled={!selectedSession?.screenshotDataUrl} title="Full screen" aria-label="Full screen"> <Button variant="ghost" size="sm" className="h-8 w-8 p-0" onClick={() => setIsFullscreen(true)} disabled={!selectedSession?.screenshotDataUrl} title="Full screen" aria-label="Full screen">
<Expand className="h-4 w-4" /> <Expand className="h-4 w-4" />
</Button> </Button>
<Button variant="ghost" size="sm" className="h-8 w-8 p-0 lg:hidden" onClick={stopSession} disabled={isBusy || !selectedSession || selectedSession.status !== 'ready'} title="Stop session" aria-label="Stop session"> <Button variant="ghost" size="sm" className="h-8 w-8 p-0 lg:hidden" onClick={stopSession} disabled={isBusy || !selectedSession || selectedSession.status !== 'ready'} title="End session" aria-label="End session">
<Square className="h-4 w-4" /> <Square className="h-4 w-4" />
</Button> </Button>
<Button variant="ghost" size="sm" className="h-8 w-8 p-0 lg:hidden" onClick={deleteSession} disabled={isBusy || !selectedSession} title="Delete session" aria-label="Delete session"> <Button variant="ghost" size="sm" className="h-8 w-8 p-0 lg:hidden" onClick={deleteSession} disabled={isBusy || !selectedSession} title="Delete session" aria-label="Delete session">
@@ -475,6 +670,11 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
<div className="min-h-0 flex-1 overflow-y-auto p-3"> <div className="min-h-0 flex-1 overflow-y-auto p-3">
{sessions.length > 0 ? ( {sessions.length > 0 ? (
<div className="space-y-2">{sessions.map(renderSessionItem)}</div> <div className="space-y-2">{sessions.map(renderSessionItem)}</div>
) : isInitialLoading ? (
<div className="flex items-center justify-center gap-2 rounded-md border border-dashed border-border/70 px-3 py-8 text-center text-xs text-muted-foreground">
<Loader2 className="h-3.5 w-3.5 animate-spin" />
Loading sessions...
</div>
) : ( ) : (
<div className="rounded-md border border-dashed border-border/70 px-3 py-8 text-center text-xs text-muted-foreground"> <div className="rounded-md border border-dashed border-border/70 px-3 py-8 text-center text-xs text-muted-foreground">
No agent browser sessions. No agent browser sessions.
@@ -505,7 +705,7 @@ export default function BrowserUsePanel({ isVisible, onShowSettings }: BrowserUs
<div className="mt-3 grid grid-cols-2 gap-2"> <div className="mt-3 grid grid-cols-2 gap-2">
<Button variant="outline" size="sm" onClick={stopSession} disabled={isBusy || !selectedSession || selectedSession.status !== 'ready'}> <Button variant="outline" size="sm" onClick={stopSession} disabled={isBusy || !selectedSession || selectedSession.status !== 'ready'}>
<Square className="h-4 w-4" /> <Square className="h-4 w-4" />
Stop End
</Button> </Button>
<Button variant="outline" size="sm" onClick={deleteSession} disabled={isBusy || !selectedSession}> <Button variant="outline" size="sm" onClick={deleteSession} disabled={isBusy || !selectedSession}>
<Trash2 className="h-4 w-4" /> <Trash2 className="h-4 w-4" />

View File

@@ -3,6 +3,7 @@ import type { DragEvent } from 'react';
import { IS_PLATFORM } from '../../../constants/config'; import { IS_PLATFORM } from '../../../constants/config';
import type { Project } from '../../../types/app'; import type { Project } from '../../../types/app';
import { isValidRefreshedToken } from '../../../utils/api';
import { import {
MAX_FILE_UPLOAD_COUNT, MAX_FILE_UPLOAD_COUNT,
MAX_FILE_UPLOAD_SIZE_BYTES, MAX_FILE_UPLOAD_SIZE_BYTES,
@@ -131,7 +132,7 @@ const uploadFormDataWithProgress = (
xhr.onload = () => { xhr.onload = () => {
const refreshedToken = xhr.getResponseHeader('X-Refreshed-Token'); const refreshedToken = xhr.getResponseHeader('X-Refreshed-Token');
if (refreshedToken) { if (isValidRefreshedToken(refreshedToken)) {
localStorage.setItem('auth-token', refreshedToken); localStorage.setItem('auth-token', refreshedToken);
} }

View File

@@ -66,6 +66,7 @@ export type MainContentHeaderProps = {
selectedSession: ProjectSession | null; selectedSession: ProjectSession | null;
shouldShowTasksTab: boolean; shouldShowTasksTab: boolean;
shouldShowBrowserTab: boolean; shouldShowBrowserTab: boolean;
shouldLabelBrowserTabAsCobrowse: boolean;
isMobile: boolean; isMobile: boolean;
onMenuClick: () => void; onMenuClick: () => void;
}; };

View File

@@ -59,6 +59,7 @@ function MainContent({
const { currentProject, setCurrentProject } = useTaskMaster() as TaskMasterContextValue; const { currentProject, setCurrentProject } = useTaskMaster() as TaskMasterContextValue;
const { tasksEnabled, isTaskMasterInstalled } = useTasksSettings() as TasksSettingsContextValue; const { tasksEnabled, isTaskMasterInstalled } = useTasksSettings() as TasksSettingsContextValue;
const [browserUseEnabled, setBrowserUseEnabled] = useState(false); const [browserUseEnabled, setBrowserUseEnabled] = useState(false);
const [shouldLabelBrowserTabAsCobrowse, setShouldLabelBrowserTabAsCobrowse] = useState(false);
const shouldShowTasksTab = Boolean(tasksEnabled && isTaskMasterInstalled); const shouldShowTasksTab = Boolean(tasksEnabled && isTaskMasterInstalled);
const shouldShowBrowserTab = browserUseEnabled; const shouldShowBrowserTab = browserUseEnabled;
@@ -103,9 +104,13 @@ function MainContent({
try { try {
const response = await authenticatedFetch('/api/browser-use/settings'); const response = await authenticatedFetch('/api/browser-use/settings');
const data = await response.json(); const data = await response.json();
setBrowserUseEnabled(Boolean(response.ok && data?.success !== false && data?.data?.settings?.enabled)); const settings = data?.data?.settings;
const enabled = Boolean(response.ok && data?.success !== false && settings?.enabled);
setBrowserUseEnabled(enabled);
setShouldLabelBrowserTabAsCobrowse(enabled && settings?.browserBackend === 'camoufox-vnc');
} catch { } catch {
setBrowserUseEnabled(false); setBrowserUseEnabled(false);
setShouldLabelBrowserTabAsCobrowse(false);
} }
}, []); }, []);
@@ -149,6 +154,7 @@ function MainContent({
selectedSession={selectedSession} selectedSession={selectedSession}
shouldShowTasksTab={shouldShowTasksTab} shouldShowTasksTab={shouldShowTasksTab}
shouldShowBrowserTab={shouldShowBrowserTab} shouldShowBrowserTab={shouldShowBrowserTab}
shouldLabelBrowserTabAsCobrowse={shouldLabelBrowserTabAsCobrowse}
isMobile={isMobile} isMobile={isMobile}
onMenuClick={onMenuClick} onMenuClick={onMenuClick}
/> />
@@ -207,7 +213,11 @@ function MainContent({
{shouldShowBrowserTab && activeTab === 'browser' && ( {shouldShowBrowserTab && activeTab === 'browser' && (
<div className="h-full overflow-hidden"> <div className="h-full overflow-hidden">
<BrowserUsePanel isVisible={activeTab === 'browser'} onShowSettings={onShowSettings} /> <BrowserUsePanel
isVisible={activeTab === 'browser'}
projectId={selectedProject.projectId}
onShowSettings={onShowSettings}
/>
</div> </div>
)} )}

View File

@@ -11,6 +11,7 @@ export default function MainContentHeader({
selectedSession, selectedSession,
shouldShowTasksTab, shouldShowTasksTab,
shouldShowBrowserTab, shouldShowBrowserTab,
shouldLabelBrowserTabAsCobrowse,
isMobile, isMobile,
onMenuClick, onMenuClick,
}: MainContentHeaderProps) { }: MainContentHeaderProps) {
@@ -44,6 +45,7 @@ export default function MainContentHeader({
selectedProject={selectedProject} selectedProject={selectedProject}
selectedSession={selectedSession} selectedSession={selectedSession}
shouldShowTasksTab={shouldShowTasksTab} shouldShowTasksTab={shouldShowTasksTab}
shouldLabelBrowserTabAsCobrowse={shouldLabelBrowserTabAsCobrowse}
/> />
</div> </div>
@@ -61,6 +63,7 @@ export default function MainContentHeader({
setActiveTab={setActiveTab} setActiveTab={setActiveTab}
shouldShowTasksTab={shouldShowTasksTab} shouldShowTasksTab={shouldShowTasksTab}
shouldShowBrowserTab={shouldShowBrowserTab} shouldShowBrowserTab={shouldShowBrowserTab}
shouldLabelBrowserTabAsCobrowse={shouldLabelBrowserTabAsCobrowse}
/> />
</div> </div>
{canScrollRight && ( {canScrollRight && (

View File

@@ -12,6 +12,7 @@ type MainContentTabSwitcherProps = {
setActiveTab: Dispatch<SetStateAction<AppTab>>; setActiveTab: Dispatch<SetStateAction<AppTab>>;
shouldShowTasksTab: boolean; shouldShowTasksTab: boolean;
shouldShowBrowserTab: boolean; shouldShowBrowserTab: boolean;
shouldLabelBrowserTabAsCobrowse: boolean;
}; };
type BuiltInTab = { type BuiltInTab = {
@@ -57,9 +58,11 @@ export default function MainContentTabSwitcher({
setActiveTab, setActiveTab,
shouldShowTasksTab, shouldShowTasksTab,
shouldShowBrowserTab, shouldShowBrowserTab,
shouldLabelBrowserTabAsCobrowse,
}: MainContentTabSwitcherProps) { }: MainContentTabSwitcherProps) {
const { t } = useTranslation(); const { t } = useTranslation();
const { plugins } = usePlugins(); const { plugins } = usePlugins();
const coBrowseLabel = t('tabs.coBrowse', { defaultValue: 'Co-browse' });
const builtInTabs: BuiltInTab[] = [ const builtInTabs: BuiltInTab[] = [
...BASE_TABS, ...BASE_TABS,
@@ -83,7 +86,11 @@ export default function MainContentTabSwitcher({
<PillBar> <PillBar>
{tabs.map((tab) => { {tabs.map((tab) => {
const isActive = tab.id === activeTab; const isActive = tab.id === activeTab;
const displayLabel = tab.kind === 'builtin' ? t(tab.labelKey) : tab.label; const displayLabel = tab.kind === 'plugin'
? tab.label
: tab.id === 'browser' && shouldLabelBrowserTabAsCobrowse
? coBrowseLabel
: t(tab.labelKey);
return ( return (
<Tooltip key={tab.id} content={displayLabel} position="bottom"> <Tooltip key={tab.id} content={displayLabel} position="bottom">

View File

@@ -9,9 +9,17 @@ type MainContentTitleProps = {
selectedProject: Project; selectedProject: Project;
selectedSession: ProjectSession | null; selectedSession: ProjectSession | null;
shouldShowTasksTab: boolean; shouldShowTasksTab: boolean;
shouldLabelBrowserTabAsCobrowse: boolean;
}; };
function getTabTitle(activeTab: AppTab, shouldShowTasksTab: boolean, t: (key: string) => string, pluginDisplayName?: string) { function getTabTitle(
activeTab: AppTab,
shouldShowTasksTab: boolean,
shouldLabelBrowserTabAsCobrowse: boolean,
t: (key: string) => string,
coBrowseLabel: string,
pluginDisplayName?: string,
) {
if (activeTab.startsWith('plugin:') && pluginDisplayName) { if (activeTab.startsWith('plugin:') && pluginDisplayName) {
return pluginDisplayName; return pluginDisplayName;
} }
@@ -29,6 +37,9 @@ function getTabTitle(activeTab: AppTab, shouldShowTasksTab: boolean, t: (key: st
} }
if (activeTab === 'browser') { if (activeTab === 'browser') {
if (shouldLabelBrowserTabAsCobrowse) {
return coBrowseLabel;
}
return t('tabs.browser'); return t('tabs.browser');
} }
@@ -48,9 +59,11 @@ export default function MainContentTitle({
selectedProject, selectedProject,
selectedSession, selectedSession,
shouldShowTasksTab, shouldShowTasksTab,
shouldLabelBrowserTabAsCobrowse,
}: MainContentTitleProps) { }: MainContentTitleProps) {
const { t } = useTranslation(); const { t } = useTranslation();
const { plugins } = usePlugins(); const { plugins } = usePlugins();
const coBrowseLabel = t('tabs.coBrowse', { defaultValue: 'Co-browse' });
const pluginDisplayName = activeTab.startsWith('plugin:') const pluginDisplayName = activeTab.startsWith('plugin:')
? plugins.find((p) => p.name === activeTab.replace('plugin:', ''))?.displayName ? plugins.find((p) => p.name === activeTab.replace('plugin:', ''))?.displayName
@@ -73,7 +86,15 @@ export default function MainContentTitle({
<h2 title={getSessionTitle(selectedSession)} className="truncate text-sm font-semibold leading-tight text-foreground"> <h2 title={getSessionTitle(selectedSession)} className="truncate text-sm font-semibold leading-tight text-foreground">
{getSessionTitle(selectedSession)} {getSessionTitle(selectedSession)}
</h2> </h2>
<div className="truncate text-[11px] leading-tight text-muted-foreground">{selectedProject.displayName}</div> <div className="flex min-w-0 items-center gap-2 text-[11px] leading-tight text-muted-foreground">
<span className="min-w-0 truncate">{selectedProject.displayName}</span>
<span
className="hidden min-w-0 max-w-[45%] flex-shrink truncate border-l border-border/60 pl-2 font-mono text-[10px] sm:block"
title={selectedSession.id}
>
{selectedSession.id}
</span>
</div>
</div> </div>
) : showChatNewSession ? ( ) : showChatNewSession ? (
<div className="min-w-0"> <div className="min-w-0">
@@ -83,7 +104,7 @@ export default function MainContentTitle({
) : ( ) : (
<div className="min-w-0"> <div className="min-w-0">
<h2 className="text-sm font-semibold leading-tight text-foreground"> <h2 className="text-sm font-semibold leading-tight text-foreground">
{getTabTitle(activeTab, shouldShowTasksTab, t, pluginDisplayName)} {getTabTitle(activeTab, shouldShowTasksTab, shouldLabelBrowserTabAsCobrowse, t, coBrowseLabel, pluginDisplayName)}
</h2> </h2>
<div className="truncate text-[11px] leading-tight text-muted-foreground">{selectedProject.displayName}</div> <div className="truncate text-[11px] leading-tight text-muted-foreground">{selectedProject.displayName}</div>
</div> </div>

View File

@@ -33,7 +33,7 @@ export const SETTINGS_MAIN_TABS: SettingsMainTabMeta[] = [
{ id: 'git', label: 'Git', keywords: 'git github commits', icon: GitBranch }, { id: 'git', label: 'Git', keywords: 'git github commits', icon: GitBranch },
{ id: 'api', label: 'API Tokens', keywords: 'api tokens auth keys', icon: KeyRound }, { id: 'api', label: 'API Tokens', keywords: 'api tokens auth keys', icon: KeyRound },
{ id: 'tasks', label: 'Tasks', keywords: 'tasks taskmaster', icon: ListChecks }, { id: 'tasks', label: 'Tasks', keywords: 'tasks taskmaster', icon: ListChecks },
{ id: 'browser', label: 'Browser', keywords: 'browser playwright chromium automation', icon: MonitorPlay }, { id: 'browser', label: 'Browser', keywords: 'browser co-browse cobrowse playwright chromium camoufox automation', icon: MonitorPlay },
{ id: 'notifications', label: 'Notifications', keywords: 'notifications alerts push', icon: Bell }, { id: 'notifications', label: 'Notifications', keywords: 'notifications alerts push', icon: Bell },
{ id: 'plugins', label: 'Plugins', keywords: 'plugins extensions integrations', icon: Plug }, { id: 'plugins', label: 'Plugins', keywords: 'plugins extensions integrations', icon: Plug },
{ id: 'about', label: 'About', keywords: 'about version info', icon: Info }, { id: 'about', label: 'About', keywords: 'about version info', icon: Info },

View File

@@ -1,22 +1,33 @@
import { useCallback, useEffect, useState } from 'react'; import { useCallback, useEffect, useState } from 'react';
import { Download, Loader2 } from 'lucide-react'; import { Download, ExternalLink, Eye, Loader2, Zap } from 'lucide-react';
import { Button } from '../../../../../shared/view/ui'; import { Button, Input } from '../../../../../shared/view/ui';
import { authenticatedFetch } from '../../../../../utils/api'; import { authenticatedFetch } from '../../../../../utils/api';
import SettingsCard from '../../SettingsCard'; import SettingsCard from '../../SettingsCard';
import SettingsRow from '../../SettingsRow'; import SettingsRow from '../../SettingsRow';
import SettingsSection from '../../SettingsSection'; import SettingsSection from '../../SettingsSection';
import SettingsToggle from '../../SettingsToggle'; import SettingsToggle from '../../SettingsToggle';
const BROWSER_USE_GUIDE_URL = 'https://cloudcli.ai/docs/browser-use';
type BrowserUseSettings = { type BrowserUseSettings = {
enabled: boolean; enabled: boolean;
persistSessions: boolean;
defaultProfileName: string;
browserBackend: 'playwright' | 'camoufox-vnc';
}; };
type BrowserUseStatus = { type BrowserUseStatus = {
enabled: boolean; enabled: boolean;
available: boolean; available: boolean;
backend: 'playwright' | 'camoufox-vnc';
browserBackend: 'playwright' | 'camoufox-vnc';
viewerMode?: 'novnc' | 'window';
playwrightInstalled: boolean; playwrightInstalled: boolean;
chromiumInstalled: boolean; chromiumInstalled: boolean;
camoufoxInstalled: boolean;
noVncInstalled: boolean;
x11vncInstalled: boolean;
installInProgress: boolean; installInProgress: boolean;
message: string; message: string;
}; };
@@ -32,16 +43,20 @@ async function readJson<T>(response: Response): Promise<T> {
export default function BrowserUseSettingsTab() { export default function BrowserUseSettingsTab() {
const [settings, setSettings] = useState<BrowserUseSettings | null>(null); const [settings, setSettings] = useState<BrowserUseSettings | null>(null);
const [status, setStatus] = useState<BrowserUseStatus | null>(null); const [status, setStatus] = useState<BrowserUseStatus | null>(null);
const [hasLoadedSettings, setHasLoadedSettings] = useState(false);
const [isSettingsLoading, setIsSettingsLoading] = useState(true); const [isSettingsLoading, setIsSettingsLoading] = useState(true);
const [isStatusLoading, setIsStatusLoading] = useState(true); const [isStatusLoading, setIsStatusLoading] = useState(true);
const [isSaving, setIsSaving] = useState(false); const [isSaving, setIsSaving] = useState(false);
const [isInstalling, setIsInstalling] = useState(false); const [isInstalling, setIsInstalling] = useState(false);
const [error, setError] = useState<string | null>(null); const [error, setError] = useState<string | null>(null);
const [profileNameDraft, setProfileNameDraft] = useState('default');
const loadSettings = useCallback(async () => { const loadSettings = useCallback(async () => {
const settingsResponse = await authenticatedFetch('/api/browser-use/settings'); const settingsResponse = await authenticatedFetch('/api/browser-use/settings');
const settingsData = await readJson<{ data: { settings: BrowserUseSettings } }>(settingsResponse); const settingsData = await readJson<{ data: { settings: BrowserUseSettings } }>(settingsResponse);
setSettings(settingsData.data.settings); setSettings(settingsData.data.settings);
setHasLoadedSettings(true);
setProfileNameDraft(settingsData.data.settings.defaultProfileName || 'default');
}, []); }, []);
const loadStatus = useCallback(async () => { const loadStatus = useCallback(async () => {
@@ -52,6 +67,7 @@ export default function BrowserUseSettingsTab() {
useEffect(() => { useEffect(() => {
setError(null); setError(null);
setHasLoadedSettings(false);
setIsSettingsLoading(true); setIsSettingsLoading(true);
setIsStatusLoading(true); setIsStatusLoading(true);
@@ -74,6 +90,7 @@ export default function BrowserUseSettingsTab() {
}); });
const data = await readJson<{ data: { settings: BrowserUseSettings } }>(response); const data = await readJson<{ data: { settings: BrowserUseSettings } }>(response);
setSettings(data.data.settings); setSettings(data.data.settings);
setHasLoadedSettings(true);
window.dispatchEvent(new Event('browserUseSettingsChanged')); window.dispatchEvent(new Event('browserUseSettingsChanged'));
setIsStatusLoading(true); setIsStatusLoading(true);
await loadStatus(); await loadStatus();
@@ -101,8 +118,23 @@ export default function BrowserUseSettingsTab() {
} }
}; };
const saveProfileName = async () => {
const nextName = profileNameDraft.trim() || 'default';
setProfileNameDraft(nextName);
if (nextName === settings?.defaultProfileName) {
return;
}
await updateSettings({ defaultProfileName: nextName });
};
const browserEnabled = settings?.enabled === true; const browserEnabled = settings?.enabled === true;
const needsBrowserBinaries = Boolean(browserEnabled && status && (!status.playwrightInstalled || !status.chromiumInstalled)); const browserDisabled = hasLoadedSettings && settings?.enabled === false;
const persistSessions = settings?.persistSessions === true;
const selectedBackend = settings?.browserBackend || 'playwright';
const effectiveBackend = status?.backend || 'playwright';
const usesLocalWindowViewer = status?.viewerMode === 'window';
const camoufoxRuntimeLabel = usesLocalWindowViewer ? 'Camoufox visible window' : 'Camoufox + noVNC';
const needsBrowserBinaries = Boolean(browserEnabled && status && !status.available);
const runtimeLabel = (installed?: boolean) => { const runtimeLabel = (installed?: boolean) => {
if (isStatusLoading && !status) { if (isStatusLoading && !status) {
return 'checking...'; return 'checking...';
@@ -114,33 +146,169 @@ export default function BrowserUseSettingsTab() {
<div className="space-y-8"> <div className="space-y-8">
<SettingsSection <SettingsSection
title="Browser" title="Browser"
description="Allow agents to create guarded Playwright browser sessions that you can monitor from the Browser tab." description="Give coding agents a working browser so they can open websites, test flows, capture screenshots, and help debug what users actually see."
> >
<SettingsCard divided> <SettingsCard divided>
<SettingsRow <SettingsRow
label="Enable Browser" label="Give Agents Browser Access"
description="Registers Browser for supported agents. Agents can create browser sessions; you can watch, stop, and delete them." description="Let agents use a browser during coding tasks while you can watch live sessions, open them in a tab, and stop them at any time."
> >
{isSettingsLoading && !settings ? ( {isSettingsLoading && !hasLoadedSettings ? (
<Loader2 className="h-4 w-4 animate-spin text-muted-foreground" /> <Loader2 className="h-4 w-4 animate-spin text-muted-foreground" />
) : ( ) : hasLoadedSettings ? (
<SettingsToggle <SettingsToggle
checked={browserEnabled} checked={browserEnabled}
onChange={(value) => void updateSettings({ enabled: value })} onChange={(value) => void updateSettings({ enabled: value })}
ariaLabel="Enable Browser" ariaLabel="Give Agents Browser Access"
disabled={isSaving} disabled={isSaving}
/> />
) : (
<span className="text-sm text-muted-foreground">Unavailable</span>
)} )}
</SettingsRow> </SettingsRow>
{browserDisabled && (
<div className="px-4 py-4">
<a
href={BROWSER_USE_GUIDE_URL}
target="_blank"
rel="noopener noreferrer"
className="inline-flex items-center gap-1.5 text-sm font-medium text-primary hover:underline"
>
Read the Browser guide
<ExternalLink className="h-3.5 w-3.5" />
</a>
</div>
)}
{error && (
<div className="px-4 py-4">
<div className="rounded-md border border-red-200 bg-red-50 px-3 py-2 text-sm text-red-700 dark:border-red-900/50 dark:bg-red-950/30 dark:text-red-200">
{error}
</div>
</div>
)}
{browserEnabled && (
<>
<div className="space-y-3 px-4 py-4">
<div className="min-w-0">
<div className="text-sm font-medium text-foreground">Browser Engine</div>
<div className="mt-0.5 text-sm text-muted-foreground">
Pick the kind of browser experience agents should use for new sessions.
</div>
</div>
<div className="grid gap-2 sm:grid-cols-2">
{([
{
value: 'playwright' as const,
label: 'Playwright',
description: 'Best for quick checks, screenshots, and automated page interaction when no manual login is needed.',
icon: Zap,
},
{
value: 'camoufox-vnc' as const,
label: 'Co-browse',
description: usesLocalWindowViewer
? 'Use Camoufox for co-browse sessions where a person may need to log in or approve a step on the machine running CloudCLI.'
: 'Use Camoufox for live co-browse sessions where a person may need to watch, log in, or approve a step.',
icon: Eye,
},
]).map((option) => {
const Icon = option.icon;
const selected = selectedBackend === option.value;
return (
<button
key={option.value}
type="button"
onClick={() => void updateSettings({ browserBackend: option.value })}
disabled={isSaving || isSettingsLoading}
className={[
'group flex min-h-[88px] items-start gap-3 rounded-lg border px-3 py-3 text-left transition-colors',
'focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 focus-visible:ring-offset-background',
selected
? 'border-primary bg-primary/5 text-foreground shadow-sm'
: 'border-border bg-background hover:border-foreground/20 hover:bg-muted/40',
(isSaving || isSettingsLoading) ? 'cursor-not-allowed opacity-60' : '',
].join(' ')}
aria-pressed={selected}
>
<span className={[
'mt-0.5 flex h-8 w-8 flex-shrink-0 items-center justify-center rounded-md border',
selected ? 'border-primary/30 bg-primary/10 text-primary' : 'border-border bg-muted/40 text-muted-foreground',
].join(' ')}
>
<Icon className="h-4 w-4" />
</span>
<span className="min-w-0">
<span className="block text-sm font-medium">{option.label}</span>
<span className="mt-1 block text-xs leading-relaxed text-muted-foreground">{option.description}</span>
</span>
</button>
);
})}
</div>
</div>
<SettingsRow
label="Remember Browser Logins"
description="Keep cookies and site storage in a named profile so agents can reuse signed-in sessions instead of starting from scratch."
>
{isSettingsLoading && !settings ? (
<Loader2 className="h-4 w-4 animate-spin text-muted-foreground" />
) : (
<SettingsToggle
checked={persistSessions}
onChange={(value) => void updateSettings({ persistSessions: value })}
ariaLabel="Remember Browser Logins"
disabled={isSaving}
/>
)}
</SettingsRow>
{persistSessions && (
<SettingsRow
label="Default Browser Profile"
description="New browser sessions use this profile by default, so saved logins stay tied to a predictable workspace."
>
<Input
value={profileNameDraft}
onChange={(event) => setProfileNameDraft(event.target.value)}
onBlur={() => void saveProfileName()}
onKeyDown={(event) => {
if (event.key === 'Enter') {
event.currentTarget.blur();
}
}}
disabled={isSaving || isSettingsLoading}
className="w-40"
aria-label="Default Browser Profile"
/>
</SettingsRow>
)}
</>
)}
{browserEnabled && (
<div className="space-y-4 px-4 py-4"> <div className="space-y-4 px-4 py-4">
<div className="flex flex-wrap gap-2 text-xs text-muted-foreground"> <div className="flex flex-wrap gap-2 text-xs text-muted-foreground">
<span className="rounded-md border border-border px-2 py-1">
Backend: {effectiveBackend === 'camoufox-vnc' ? `Co-browse (${camoufoxRuntimeLabel})` : 'Playwright'}
</span>
<span className="rounded-md border border-border px-2 py-1"> <span className="rounded-md border border-border px-2 py-1">
Playwright: {runtimeLabel(status?.playwrightInstalled)} Playwright: {runtimeLabel(status?.playwrightInstalled)}
</span> </span>
<span className="rounded-md border border-border px-2 py-1"> <span className="rounded-md border border-border px-2 py-1">
Chromium: {runtimeLabel(status?.chromiumInstalled)} Chromium: {runtimeLabel(status?.chromiumInstalled)}
</span> </span>
<span className="rounded-md border border-border px-2 py-1">
Camoufox: {runtimeLabel(status?.camoufoxInstalled)}
</span>
{!usesLocalWindowViewer && (
<span className="rounded-md border border-border px-2 py-1">
noVNC: {runtimeLabel(status?.noVncInstalled)}
</span>
)}
<span className="rounded-md border border-border px-2 py-1"> <span className="rounded-md border border-border px-2 py-1">
Status: {isStatusLoading && !status ? 'checking...' : status?.available ? 'ready' : browserEnabled ? 'setup required' : 'disabled'} Status: {isStatusLoading && !status ? 'checking...' : status?.available ? 'ready' : browserEnabled ? 'setup required' : 'disabled'}
</span> </span>
@@ -172,12 +340,17 @@ export default function BrowserUseSettingsTab() {
</div> </div>
)} )}
{error && ( <a
<div className="rounded-md border border-red-200 bg-red-50 px-3 py-2 text-sm text-red-700 dark:border-red-900/50 dark:bg-red-950/30 dark:text-red-200"> href={BROWSER_USE_GUIDE_URL}
{error} target="_blank"
</div> rel="noopener noreferrer"
)} className="inline-flex items-center gap-1.5 text-sm font-medium text-primary hover:underline"
>
Read the Browser guide
<ExternalLink className="h-3.5 w-3.5" />
</a>
</div> </div>
)}
</SettingsCard> </SettingsCard>
</SettingsSection> </SettingsSection>
</div> </div>

View File

@@ -24,6 +24,7 @@
"git": "Source Control", "git": "Source Control",
"tasks": "Tasks", "tasks": "Tasks",
"browser": "Browser", "browser": "Browser",
"coBrowse": "Co-browse",
"computer": "Computer" "computer": "Computer"
}, },
"status": { "status": {

View File

@@ -1,5 +1,16 @@
import { IS_PLATFORM } from "../constants/config"; import { IS_PLATFORM } from "../constants/config";
// Only accept a refreshed token that has this app's issued JWT shape
// (three base64url segments). An attacker-injected/malformed header value
// must never overwrite the stored auth token.
/**
* @param {unknown} token
* @returns {token is string}
*/
export const isValidRefreshedToken = (token) =>
typeof token === 'string' &&
/^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/.test(token);
// Utility function for authenticated API calls // Utility function for authenticated API calls
export const authenticatedFetch = (url, options = {}) => { export const authenticatedFetch = (url, options = {}) => {
const token = localStorage.getItem('auth-token'); const token = localStorage.getItem('auth-token');
@@ -23,7 +34,7 @@ export const authenticatedFetch = (url, options = {}) => {
}, },
}).then((response) => { }).then((response) => {
const refreshedToken = response.headers.get('X-Refreshed-Token'); const refreshedToken = response.headers.get('X-Refreshed-Token');
if (refreshedToken) { if (isValidRefreshedToken(refreshedToken)) {
localStorage.setItem('auth-token', refreshedToken); localStorage.setItem('auth-token', refreshedToken);
} }
return response; return response;