mirror of
https://github.com/siteboon/claudecodeui.git
synced 2026-06-27 14:15:26 +08:00
fix: voice tts format settings (#919)
* feat(voice): add optional speech-to-text input and read-aloud TTS Adds a push-to-talk mic button in the composer and a read-aloud button on assistant messages. Both are opt-in and hidden unless a voice backend is configured via VOICE_SIDECAR_URL. The auth-gated /api/voice proxy forwards to a configurable backend exposing /transcribe and /tts (provider-agnostic); the frontend probes /api/voice/health and hides the controls when disabled. Adds i18n keys and docs/voice.md. Includes a local, no-API-key reference backend in voice-sidecar/ (faster-whisper for STT, Kokoro-82M for TTS, both CPU-capable). * refactor(voice): provider-agnostic backend and in-app config Switches the voice proxy to the OpenAI audio API (/v1/audio/transcriptions and /v1/audio/speech) so it works with OpenAI, Groq, or a local server. Adds a Settings -> Voice tab (base URL, API key, models, voice) plus a Quick Settings toggle, and removes the bundled Python sidecar. Review fixes: stop mic tracks on unmount, clear the global TTS stop handler and revoke leaked blob URLs, add fetch timeouts in the proxy, surface mic errors in the button, trim before appending transcripts, and drop the repo-wide wav ignore. * fix(voice): relax backend timeout and surface timeout errors Bumps the proxy timeout to 5 minutes (VOICE_TIMEOUT_MS) since local TTS can synthesize long messages at roughly real-time, and returns a clear timed-out message (504) instead of failing silently. The read-aloud button now shows backend errors. * fix(voice): play read-aloud through an app-level player to stop cutoffs Read-aloud now runs in a single module-level player outside the React tree instead of per-message component state. Switching chats or re-rendering a message no longer revokes the blob URL mid-play (the 'Invalid URI' cutoff). Adds content-keyed caching so re-listening doesn't regenerate, and reuses one audio element (also unlocks iOS once). * fix(voice): address review (SSRF guard, auth mapping, client timeout) Validates the user-supplied backend URL (http/https only, blocks the link-local metadata range) to prevent SSRF; remaps upstream 401/403 so a bad voice API key isn't read as the app's own auth failing; adds a client-side AbortController timeout on the read-aloud request so the button can't sit in loading if a request stalls. * docs(voice): provider-agnostic wording and jsdoc on proxy functions drop leftover sidecar/faster-whisper references now that the backend is any openai-compatible voice api, and add jsdoc to the voice-proxy functions so the docstring coverage check passes. * fix(voice): harden timeout parsing, tts input check, and player abort - fall back to the default when VOICE_TIMEOUT_MS is non-numeric or <= 0, so a bad override can't make the abort fire immediately - type-check the tts `text` before calling .trim() so a non-string body returns 400 instead of throwing - abort the in-flight TTS fetch on stop() and on a superseding play, so tapping read-aloud repeatedly doesn't leave orphaned requests generating audio * feat(voice): send transcript with the main send button while recording while dictating, the main send button stops recording, transcribes, and sends in one tap, matching the codex-style flow. the mic button still stops and drops the transcript into the input box to edit before sending. voice recording state is lifted into the composer so both buttons share it, and the send button is enabled (not grayed) while recording. also fix a pre-existing type error: the quick-settings preferences map was missing voiceEnabled. * fix(voice): make stop() idempotent so a double tap can't throw guard on the recorder's own state instead of react state, so a double tap or the mic and send buttons both firing won't call stop() on an already-inactive MediaRecorder. * fix(voice): expose TTS format in user settings * fix(voice): harden recording and backend behavior Redirects could bypass the backend URL guard, and TTS playback waited for full buffering. Recording could overlap or finish after teardown. Controls also ignored backend readiness. Explicit formats and config-aware cache keys prevent stale audio after settings change. * fix(voice): validate config and request boundaries Malformed stored settings could break voice requests instead of using safe defaults. Health results could outlive auth changes. URL checks also did not guard the fetch sink. Remove constant recorder branches so lifecycle cancellation stays clear. * fix(voice): separate client and server backends User-selected backend URLs must remain usable without letting clients control server requests. Call custom providers from the browser while keeping the server proxy bound to its configured host. This restores voice controls for frontend settings without reopening the SSRF path. * fix: hide voice options until enabled --------- Co-authored-by: newsbubbles <nathaniel.gibson@gmail.com> Co-authored-by: Simos Mikelatos <simosmik@gmail.com>
This commit is contained in:
@@ -775,6 +775,17 @@ export function useChatComposerState({
|
||||
handleSubmitRef.current = handleSubmit;
|
||||
}, [handleSubmit]);
|
||||
|
||||
// A voice transcript either fills the input (to edit before sending) or, when the
|
||||
// user tapped "stop and send", is submitted straight away. Mirror the value into
|
||||
// inputValueRef synchronously so handleSubmit reads the new text, not the stale state.
|
||||
const handleVoiceTranscript = useCallback((text: string, send?: boolean) => {
|
||||
const base = inputValueRef.current.trim();
|
||||
const next = base ? `${base} ${text}` : text;
|
||||
setInput(next);
|
||||
inputValueRef.current = next;
|
||||
if (send) handleSubmitRef.current?.(createFakeSubmitEvent());
|
||||
}, [setInput]);
|
||||
|
||||
useEffect(() => {
|
||||
inputValueRef.current = input;
|
||||
}, [input]);
|
||||
@@ -1013,6 +1024,7 @@ export function useChatComposerState({
|
||||
isDragActive,
|
||||
openImagePicker: open,
|
||||
handleSubmit,
|
||||
handleVoiceTranscript,
|
||||
handleInputChange,
|
||||
handleKeyDown,
|
||||
handlePaste,
|
||||
|
||||
33
src/components/chat/hooks/useTts.ts
Normal file
33
src/components/chat/hooks/useTts.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
import { useCallback, useEffect, useState } from 'react';
|
||||
import { voicePlayer, voiceId, type VoiceSnapshot } from '../../../lib/voicePlayer';
|
||||
|
||||
export type TtsState = VoiceSnapshot['state'];
|
||||
|
||||
/**
|
||||
* Thin adapter over the app-level voicePlayer. Playback lives outside React (see
|
||||
* lib/voicePlayer), so switching chats or re-rendering a message no longer cuts the
|
||||
* audio off. This hook just reflects the player's state for one message and forwards taps.
|
||||
*/
|
||||
export function useTts(getText: () => string) {
|
||||
const content = getText();
|
||||
const id = voiceId(content);
|
||||
|
||||
const [snap, setSnap] = useState<VoiceSnapshot>(() => voicePlayer.getSnapshot(id));
|
||||
|
||||
useEffect(() => {
|
||||
const update = () =>
|
||||
setSnap((prev) => {
|
||||
const next = voicePlayer.getSnapshot(id);
|
||||
return prev.state === next.state && prev.error === next.error ? prev : next;
|
||||
});
|
||||
update();
|
||||
return voicePlayer.subscribe(update);
|
||||
}, [id]);
|
||||
|
||||
const toggle = useCallback(() => {
|
||||
voicePlayer.unlock(); // synchronous, within the click gesture (iOS)
|
||||
voicePlayer.toggle(content);
|
||||
}, [content]);
|
||||
|
||||
return { state: snap.state, toggle, error: snap.error };
|
||||
}
|
||||
85
src/components/chat/hooks/useVoiceAvailable.ts
Normal file
85
src/components/chat/hooks/useVoiceAvailable.ts
Normal file
@@ -0,0 +1,85 @@
|
||||
import { useEffect, useState } from 'react';
|
||||
|
||||
import { authenticatedFetch } from '../../../utils/api';
|
||||
import { readVoiceConfig, VOICE_CONFIG_SYNC_EVENT } from '../../../hooks/useVoiceConfig';
|
||||
|
||||
// Voice UI is gated on the `voiceEnabled` UI preference (toggled in Quick Settings /
|
||||
// the Settings modal) and a configured voice backend.
|
||||
const STORAGE_KEY = 'uiPreferences';
|
||||
const SYNC_EVENT = 'ui-preferences:sync';
|
||||
let healthRequest: Promise<boolean> | null = null;
|
||||
|
||||
function checkVoiceHealth(): Promise<boolean> {
|
||||
if (healthRequest) return healthRequest;
|
||||
const request = authenticatedFetch('/api/voice/health')
|
||||
.then(async (response) => {
|
||||
if (!response.ok) throw new Error(`Voice health check failed (${response.status})`);
|
||||
const data = await response.json();
|
||||
return data?.configured === true;
|
||||
})
|
||||
.finally(() => {
|
||||
healthRequest = null;
|
||||
});
|
||||
healthRequest = request;
|
||||
return request;
|
||||
}
|
||||
|
||||
function readVoiceEnabled(): boolean {
|
||||
try {
|
||||
const raw = localStorage.getItem(STORAGE_KEY);
|
||||
if (!raw) return false;
|
||||
const parsed = JSON.parse(raw);
|
||||
return parsed?.voiceEnabled === true || parsed?.voiceEnabled === 'true';
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export function useVoiceAvailable(): boolean {
|
||||
const [enabled, setEnabled] = useState<boolean>(() =>
|
||||
typeof window === 'undefined' ? false : readVoiceEnabled(),
|
||||
);
|
||||
const [available, setAvailable] = useState(false);
|
||||
|
||||
useEffect(() => {
|
||||
const update = () => setEnabled(readVoiceEnabled());
|
||||
window.addEventListener('storage', update);
|
||||
window.addEventListener(SYNC_EVENT, update as EventListener);
|
||||
return () => {
|
||||
window.removeEventListener('storage', update);
|
||||
window.removeEventListener(SYNC_EVENT, update as EventListener);
|
||||
};
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
let active = true;
|
||||
let requestId = 0;
|
||||
|
||||
const check = async () => {
|
||||
if (!enabled) {
|
||||
setAvailable(false);
|
||||
return;
|
||||
}
|
||||
if (readVoiceConfig().baseUrl.trim()) {
|
||||
setAvailable(true);
|
||||
return;
|
||||
}
|
||||
const id = ++requestId;
|
||||
try {
|
||||
const result = await checkVoiceHealth();
|
||||
if (active && id === requestId) setAvailable(result);
|
||||
} catch {
|
||||
if (active && id === requestId) setAvailable(false);
|
||||
}
|
||||
};
|
||||
|
||||
void check();
|
||||
window.addEventListener(VOICE_CONFIG_SYNC_EVENT, check);
|
||||
return () => {
|
||||
active = false;
|
||||
window.removeEventListener(VOICE_CONFIG_SYNC_EVENT, check);
|
||||
};
|
||||
}, [enabled]);
|
||||
|
||||
return enabled && available;
|
||||
}
|
||||
149
src/components/chat/hooks/useVoiceInput.ts
Normal file
149
src/components/chat/hooks/useVoiceInput.ts
Normal file
@@ -0,0 +1,149 @@
|
||||
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||
|
||||
import { transcribeVoice } from '../../../lib/voiceApi';
|
||||
|
||||
// Mobile-safe recording: iOS Safari 18.4+ supports webm/opus; older iOS needs mp4.
|
||||
const MIME_CANDIDATES = [
|
||||
'audio/webm;codecs=opus',
|
||||
'audio/webm',
|
||||
'audio/mp4',
|
||||
'audio/ogg;codecs=opus',
|
||||
'audio/ogg',
|
||||
];
|
||||
|
||||
function pickMime(): string {
|
||||
for (const t of MIME_CANDIDATES) {
|
||||
try {
|
||||
if (typeof MediaRecorder !== 'undefined' && MediaRecorder.isTypeSupported(t)) return t;
|
||||
} catch {
|
||||
/* isTypeSupported can throw on some iOS versions */
|
||||
}
|
||||
}
|
||||
return '';
|
||||
}
|
||||
|
||||
export type VoiceInputState = 'idle' | 'recording' | 'transcribing';
|
||||
|
||||
/**
|
||||
* Push-to-talk dictation. Records the mic, uploads to /api/voice/transcribe
|
||||
* (an OpenAI-compatible speech-to-text backend via the Express proxy), and
|
||||
* returns the transcript through onTranscript.
|
||||
*/
|
||||
export function useVoiceInput(
|
||||
onTranscript: (text: string, send?: boolean) => void,
|
||||
onError?: (msg: string) => void,
|
||||
) {
|
||||
const [state, setState] = useState<VoiceInputState>('idle');
|
||||
const recorderRef = useRef<MediaRecorder | null>(null);
|
||||
const chunksRef = useRef<Blob[]>([]);
|
||||
const streamRef = useRef<MediaStream | null>(null);
|
||||
const cancelledRef = useRef(false);
|
||||
const startingRef = useRef(false);
|
||||
// Whether the in-progress stop should auto-send the transcript (vs just fill the box).
|
||||
const sendRef = useRef(false);
|
||||
|
||||
const stopTracks = () => {
|
||||
streamRef.current?.getTracks().forEach((t) => t.stop());
|
||||
streamRef.current = null;
|
||||
};
|
||||
|
||||
// Stop the mic if the component unmounts mid-recording.
|
||||
useEffect(() => {
|
||||
cancelledRef.current = false;
|
||||
return () => {
|
||||
cancelledRef.current = true;
|
||||
startingRef.current = false;
|
||||
streamRef.current?.getTracks().forEach((t) => t.stop());
|
||||
streamRef.current = null;
|
||||
recorderRef.current = null;
|
||||
};
|
||||
}, []);
|
||||
|
||||
const start = useCallback(async () => {
|
||||
if (startingRef.current || (recorderRef.current && recorderRef.current.state !== 'inactive')) return;
|
||||
startingRef.current = true;
|
||||
try {
|
||||
const stream = await navigator.mediaDevices.getUserMedia({
|
||||
audio: { echoCancellation: true, noiseSuppression: true },
|
||||
});
|
||||
if (cancelledRef.current) {
|
||||
stream.getTracks().forEach((t) => t.stop());
|
||||
return;
|
||||
}
|
||||
streamRef.current = stream;
|
||||
const mimeType = pickMime();
|
||||
const rec = mimeType ? new MediaRecorder(stream, { mimeType }) : new MediaRecorder(stream);
|
||||
recorderRef.current = rec;
|
||||
chunksRef.current = [];
|
||||
|
||||
rec.ondataavailable = (e) => {
|
||||
if (e.data.size > 0) chunksRef.current.push(e.data);
|
||||
};
|
||||
|
||||
rec.onstop = async () => {
|
||||
stopTracks();
|
||||
if (cancelledRef.current) return;
|
||||
// Capture and clear the send intent for this stop before any async work.
|
||||
const shouldSend = sendRef.current;
|
||||
sendRef.current = false;
|
||||
const type = rec.mimeType || 'audio/webm';
|
||||
const blob = new Blob(chunksRef.current, { type });
|
||||
if (blob.size < 800) {
|
||||
setState('idle');
|
||||
onError?.('Recording too short');
|
||||
return;
|
||||
}
|
||||
setState('transcribing');
|
||||
try {
|
||||
const ext = type.includes('mp4') ? 'm4a' : type.includes('ogg') ? 'ogg' : 'webm';
|
||||
const res = await transcribeVoice(blob, `recording.${ext}`);
|
||||
if (!res.ok) throw new Error(`transcribe ${res.status}`);
|
||||
const data = await res.json();
|
||||
if (cancelledRef.current) return;
|
||||
const text = String(data?.text || '').trim();
|
||||
if (text) onTranscript(text, shouldSend);
|
||||
else onError?.('No speech detected');
|
||||
} catch (e) {
|
||||
if (!cancelledRef.current) {
|
||||
onError?.(`Transcription failed: ${e instanceof Error ? e.message : String(e)}`);
|
||||
}
|
||||
} finally {
|
||||
if (!cancelledRef.current) setState('idle');
|
||||
}
|
||||
};
|
||||
|
||||
rec.start();
|
||||
setState('recording');
|
||||
} catch (e) {
|
||||
recorderRef.current = null;
|
||||
stopTracks();
|
||||
if (cancelledRef.current) return;
|
||||
const err = e as { name?: string; message?: string };
|
||||
let msg = `Mic error: ${err?.message || e}`;
|
||||
if (err?.name === 'NotAllowedError') msg = 'Microphone access denied.';
|
||||
else if (err?.name === 'NotFoundError') msg = 'No microphone found.';
|
||||
onError?.(msg);
|
||||
setState('idle');
|
||||
} finally {
|
||||
startingRef.current = false;
|
||||
}
|
||||
}, [onTranscript, onError]);
|
||||
|
||||
// Stop recording. Pass { send: true } to auto-send the transcript once it's ready.
|
||||
// Guard on the recorder's own state (not React state) so a double tap, or the mic
|
||||
// and Send buttons both firing, can't call stop() on an already-inactive recorder.
|
||||
const stop = useCallback((opts?: { send?: boolean }) => {
|
||||
const rec = recorderRef.current;
|
||||
if (rec && rec.state !== 'inactive') {
|
||||
sendRef.current = opts?.send ?? false;
|
||||
rec.stop();
|
||||
}
|
||||
}, []);
|
||||
|
||||
const toggle = useCallback(() => {
|
||||
if (state === 'recording') stop();
|
||||
else if (state === 'idle') start();
|
||||
}, [state, start, stop]);
|
||||
|
||||
return { state, toggle, stop };
|
||||
}
|
||||
@@ -173,6 +173,7 @@ function ChatInterface({
|
||||
isDragActive,
|
||||
openImagePicker,
|
||||
handleSubmit,
|
||||
handleVoiceTranscript,
|
||||
handleInputChange,
|
||||
handleKeyDown,
|
||||
handlePaste,
|
||||
@@ -406,6 +407,7 @@ function ChatInterface({
|
||||
renderInputWithMentions={renderInputWithMentions}
|
||||
textareaRef={textareaRef}
|
||||
input={input}
|
||||
onVoiceTranscript={handleVoiceTranscript}
|
||||
onInputChange={handleInputChange}
|
||||
onTextareaClick={handleTextareaClick}
|
||||
onTextareaKeyDown={handleKeyDown}
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||
import type {
|
||||
ChangeEvent,
|
||||
ClipboardEvent,
|
||||
@@ -9,8 +10,10 @@ import type {
|
||||
RefObject,
|
||||
TouchEvent,
|
||||
} from 'react';
|
||||
import { ImageIcon, MessageSquareIcon, XIcon, ArrowDownIcon } from 'lucide-react';
|
||||
import { ImageIcon, MessageSquareIcon, XIcon, ArrowDownIcon, Loader2 } from 'lucide-react';
|
||||
|
||||
import { useVoiceInput } from '../../hooks/useVoiceInput';
|
||||
import { useVoiceAvailable } from '../../hooks/useVoiceAvailable';
|
||||
import type { SessionActivity } from '../../../../hooks/useSessionProtection';
|
||||
import type { PendingPermissionRequest, PermissionMode } from '../../types/types';
|
||||
import {
|
||||
@@ -27,6 +30,7 @@ import {
|
||||
import CommandMenu from './CommandMenu';
|
||||
import ActivityIndicator from './ActivityIndicator';
|
||||
import ImageAttachment from './ImageAttachment';
|
||||
import VoiceInputButton from './VoiceInputButton';
|
||||
import PermissionRequestsBanner from './PermissionRequestsBanner';
|
||||
import TokenUsageSummary from './TokenUsageSummary';
|
||||
|
||||
@@ -89,6 +93,7 @@ interface ChatComposerProps {
|
||||
renderInputWithMentions: (text: string) => ReactNode;
|
||||
textareaRef: RefObject<HTMLTextAreaElement>;
|
||||
input: string;
|
||||
onVoiceTranscript?: (text: string, send?: boolean) => void;
|
||||
onInputChange: (event: ChangeEvent<HTMLTextAreaElement>) => void;
|
||||
onTextareaClick: (event: MouseEvent<HTMLTextAreaElement>) => void;
|
||||
onTextareaKeyDown: (event: KeyboardEvent<HTMLTextAreaElement>) => void;
|
||||
@@ -142,6 +147,7 @@ export default function ChatComposer({
|
||||
renderInputWithMentions,
|
||||
textareaRef,
|
||||
input,
|
||||
onVoiceTranscript,
|
||||
onInputChange,
|
||||
onTextareaClick,
|
||||
onTextareaKeyDown,
|
||||
@@ -154,6 +160,28 @@ export default function ChatComposer({
|
||||
sendByCtrlEnter,
|
||||
}: ChatComposerProps) {
|
||||
const { t } = useTranslation('chat');
|
||||
|
||||
// Voice state is hosted here (not in the mic button) so the main Send button can stop
|
||||
// recording and send the transcript in one tap, the way the mic button drops it in the box.
|
||||
const voiceAvailable = useVoiceAvailable();
|
||||
const [voiceError, setVoiceError] = useState<string | null>(null);
|
||||
const voiceErrorTimer = useRef<ReturnType<typeof setTimeout> | null>(null);
|
||||
const handleVoiceError = useCallback((msg: string) => {
|
||||
setVoiceError(msg);
|
||||
if (voiceErrorTimer.current) clearTimeout(voiceErrorTimer.current);
|
||||
voiceErrorTimer.current = setTimeout(() => setVoiceError(null), 4000);
|
||||
}, []);
|
||||
useEffect(() => () => {
|
||||
if (voiceErrorTimer.current) clearTimeout(voiceErrorTimer.current);
|
||||
}, []);
|
||||
const noopTranscript = useCallback(() => {}, []);
|
||||
const { state: voiceState, toggle: voiceToggle, stop: voiceStop } = useVoiceInput(
|
||||
onVoiceTranscript ?? noopTranscript,
|
||||
handleVoiceError,
|
||||
);
|
||||
const isRecording = voiceState === 'recording';
|
||||
const isTranscribing = voiceState === 'transcribing';
|
||||
|
||||
const textareaRect = textareaRef.current?.getBoundingClientRect();
|
||||
const commandMenuPosition = {
|
||||
top: textareaRect ? Math.max(16, textareaRect.top - 316) : 0,
|
||||
@@ -309,6 +337,10 @@ export default function ChatComposer({
|
||||
<ImageIcon />
|
||||
</PromptInputButton>
|
||||
|
||||
{onVoiceTranscript && voiceAvailable && (
|
||||
<VoiceInputButton state={voiceState} onToggle={voiceToggle} errorMsg={voiceError} />
|
||||
)}
|
||||
|
||||
<button
|
||||
type="button"
|
||||
onClick={onModeSwitch}
|
||||
@@ -387,10 +419,21 @@ export default function ChatComposer({
|
||||
{sendByCtrlEnter ? t('input.hintText.ctrlEnter') : t('input.hintText.enter')}
|
||||
</div>
|
||||
<PromptInputSubmit
|
||||
onClick={isLoading ? onAbortSession : undefined}
|
||||
disabled={!isLoading && !input.trim()}
|
||||
onClick={
|
||||
isLoading
|
||||
? onAbortSession
|
||||
: isRecording
|
||||
? (e: MouseEvent<HTMLButtonElement>) => {
|
||||
e.preventDefault();
|
||||
voiceStop({ send: true });
|
||||
}
|
||||
: undefined
|
||||
}
|
||||
disabled={isLoading ? false : isRecording ? false : isTranscribing ? true : !input.trim()}
|
||||
className="h-10 w-10 sm:h-10 sm:w-10"
|
||||
/>
|
||||
>
|
||||
{isTranscribing ? <Loader2 className="h-4 w-4 animate-spin" /> : undefined}
|
||||
</PromptInputSubmit>
|
||||
</div>
|
||||
</PromptInputFooter>
|
||||
</PromptInput>
|
||||
|
||||
@@ -15,6 +15,7 @@ import { Reasoning, ReasoningTrigger, ReasoningContent } from '../../../../share
|
||||
|
||||
import { Markdown } from './Markdown';
|
||||
import MessageCopyControl from './MessageCopyControl';
|
||||
import MessageSpeakControl from './MessageSpeakControl';
|
||||
|
||||
type DiffLine = {
|
||||
type: string;
|
||||
@@ -415,6 +416,9 @@ const MessageComponent = memo(({ message, prevMessage, createDiff, onFileOpen, a
|
||||
{shouldShowAssistantCopyControl && (
|
||||
<MessageCopyControl content={assistantCopyContent} messageType="assistant" />
|
||||
)}
|
||||
{shouldShowAssistantCopyControl && (
|
||||
<MessageSpeakControl content={assistantCopyContent} />
|
||||
)}
|
||||
{!isGrouped && <span>{formattedTime}</span>}
|
||||
</div>
|
||||
)}
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
import { Volume2, Loader2, Square } from 'lucide-react';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import { useTts } from '../../hooks/useTts';
|
||||
import { useVoiceAvailable } from '../../hooks/useVoiceAvailable';
|
||||
|
||||
// Tap-to-speak button beside the copy control on assistant messages.
|
||||
// Renders nothing unless the optional voice feature is enabled.
|
||||
const MessageSpeakControl = ({ content }: { content: string }) => {
|
||||
const { t } = useTranslation('chat');
|
||||
const available = useVoiceAvailable();
|
||||
const { state, toggle, error } = useTts(() => content);
|
||||
|
||||
if (!available) return null;
|
||||
|
||||
const title =
|
||||
state === 'playing' ? t('voice.stopSpeaking') : state === 'loading' ? t('voice.loading') : t('voice.speak');
|
||||
|
||||
return (
|
||||
<span className="relative inline-flex">
|
||||
{error && (
|
||||
<span className="absolute bottom-full left-1/2 z-10 mb-1 max-w-[240px] -translate-x-1/2 whitespace-normal rounded bg-red-600 px-2 py-1 text-center text-xs text-white shadow-lg">
|
||||
{error}
|
||||
</span>
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
onClick={toggle}
|
||||
title={title}
|
||||
aria-label={title}
|
||||
className="inline-flex items-center gap-1 rounded px-1 py-0.5 text-gray-400 transition-colors hover:text-gray-600 dark:text-gray-500 dark:hover:text-gray-300"
|
||||
>
|
||||
{state === 'playing' ? (
|
||||
<Square className="h-3.5 w-3.5" />
|
||||
) : state === 'loading' ? (
|
||||
<Loader2 className="h-3.5 w-3.5 animate-spin" />
|
||||
) : (
|
||||
<Volume2 className="h-3.5 w-3.5" />
|
||||
)}
|
||||
</button>
|
||||
</span>
|
||||
);
|
||||
};
|
||||
|
||||
export default MessageSpeakControl;
|
||||
46
src/components/chat/view/subcomponents/VoiceInputButton.tsx
Normal file
46
src/components/chat/view/subcomponents/VoiceInputButton.tsx
Normal file
@@ -0,0 +1,46 @@
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import { Mic, Square, Loader2 } from 'lucide-react';
|
||||
|
||||
import { PromptInputButton } from '../../../../shared/view/ui';
|
||||
import type { VoiceInputState } from '../../hooks/useVoiceInput';
|
||||
|
||||
type Props = {
|
||||
state: VoiceInputState;
|
||||
onToggle: () => void;
|
||||
errorMsg?: string | null;
|
||||
};
|
||||
|
||||
// Push-to-talk mic button (presentational). Recording state and the stop-and-send action
|
||||
// are owned by the composer so the main Send button can drive them too. This button just
|
||||
// starts recording and, while recording, stops and drops the transcript into the input box.
|
||||
export default function VoiceInputButton({ state, onToggle, errorMsg }: Props) {
|
||||
const { t } = useTranslation('chat');
|
||||
|
||||
const icon =
|
||||
state === 'recording' ? (
|
||||
<Square className="text-red-500" />
|
||||
) : state === 'transcribing' ? (
|
||||
<Loader2 className="animate-spin" />
|
||||
) : (
|
||||
<Mic />
|
||||
);
|
||||
|
||||
return (
|
||||
<span className="relative inline-flex">
|
||||
{errorMsg && (
|
||||
<span className="absolute bottom-full left-1/2 mb-1 -translate-x-1/2 whitespace-nowrap rounded bg-red-600 px-2 py-1 text-xs text-white shadow-lg">
|
||||
{errorMsg}
|
||||
</span>
|
||||
)}
|
||||
<PromptInputButton
|
||||
tooltip={{ content: state === 'recording' ? t('voice.stopRecording') : t('voice.input') }}
|
||||
onClick={(e: { preventDefault: () => void }) => {
|
||||
e.preventDefault();
|
||||
onToggle();
|
||||
}}
|
||||
>
|
||||
{icon}
|
||||
</PromptInputButton>
|
||||
</span>
|
||||
);
|
||||
}
|
||||
@@ -4,6 +4,7 @@ import {
|
||||
Eye,
|
||||
Languages,
|
||||
Maximize2,
|
||||
Mic,
|
||||
} from 'lucide-react';
|
||||
import type { PreferenceToggleItem } from './types';
|
||||
|
||||
@@ -54,4 +55,9 @@ export const INPUT_SETTING_TOGGLES: PreferenceToggleItem[] = [
|
||||
labelKey: 'quickSettings.sendByCtrlEnter',
|
||||
icon: Languages,
|
||||
},
|
||||
{
|
||||
key: 'voiceEnabled',
|
||||
labelKey: 'quickSettings.voiceEnabled',
|
||||
icon: Mic,
|
||||
},
|
||||
];
|
||||
|
||||
@@ -6,7 +6,8 @@ export type PreferenceToggleKey =
|
||||
| 'showRawParameters'
|
||||
| 'showThinking'
|
||||
| 'autoScrollToBottom'
|
||||
| 'sendByCtrlEnter';
|
||||
| 'sendByCtrlEnter'
|
||||
| 'voiceEnabled';
|
||||
|
||||
export type QuickSettingsPreferences = Record<PreferenceToggleKey, boolean>;
|
||||
|
||||
|
||||
@@ -28,6 +28,9 @@ export default function QuickSettingsContent({
|
||||
onPreferenceChange,
|
||||
}: QuickSettingsContentProps) {
|
||||
const { t } = useTranslation('settings');
|
||||
const inputSettingToggles = preferences.voiceEnabled
|
||||
? INPUT_SETTING_TOGGLES
|
||||
: INPUT_SETTING_TOGGLES.filter(({ key }) => key !== 'voiceEnabled');
|
||||
|
||||
const renderToggleRows = (items: PreferenceToggleItem[]) => (
|
||||
items.map(({ key, labelKey, icon }) => (
|
||||
@@ -67,7 +70,7 @@ export default function QuickSettingsContent({
|
||||
</QuickSettingsSection>
|
||||
|
||||
<QuickSettingsSection title={t('quickSettings.sections.inputSettings')}>
|
||||
{renderToggleRows(INPUT_SETTING_TOGGLES)}
|
||||
{renderToggleRows(inputSettingToggles)}
|
||||
<p className="ml-3 text-xs text-gray-500 dark:text-gray-400">
|
||||
{t('quickSettings.sendByCtrlEnterDescription')}
|
||||
</p>
|
||||
|
||||
@@ -27,12 +27,14 @@ export default function QuickSettingsPanelView() {
|
||||
showThinking: preferences.showThinking,
|
||||
autoScrollToBottom: preferences.autoScrollToBottom,
|
||||
sendByCtrlEnter: preferences.sendByCtrlEnter,
|
||||
voiceEnabled: preferences.voiceEnabled,
|
||||
}), [
|
||||
preferences.autoExpandTools,
|
||||
preferences.autoScrollToBottom,
|
||||
preferences.sendByCtrlEnter,
|
||||
preferences.showRawParameters,
|
||||
preferences.showThinking,
|
||||
preferences.voiceEnabled,
|
||||
]);
|
||||
|
||||
const handlePreferenceChange = useCallback(
|
||||
|
||||
@@ -3,7 +3,7 @@ import type { Dispatch, SetStateAction } from 'react';
|
||||
import type { LLMProvider } from '../../../types/app';
|
||||
import type { ProviderAuthStatus } from '../../provider-auth/types';
|
||||
|
||||
export type SettingsMainTab = 'agents' | 'appearance' | 'git' | 'api' | 'tasks' | 'browser' | 'notifications' | 'plugins' | 'about';
|
||||
export type SettingsMainTab = 'agents' | 'appearance' | 'git' | 'api' | 'voice' | 'tasks' | 'browser' | 'notifications' | 'plugins' | 'about';
|
||||
export type AgentProvider = LLMProvider;
|
||||
export type AgentCategory = 'account' | 'permissions' | 'mcp' | 'skills';
|
||||
export type ProjectSortOrder = 'name' | 'date';
|
||||
|
||||
@@ -7,6 +7,7 @@ import SettingsSidebar from '../view/SettingsSidebar';
|
||||
import AgentsSettingsTab from '../view/tabs/agents-settings/AgentsSettingsTab';
|
||||
import AppearanceSettingsTab from '../view/tabs/AppearanceSettingsTab';
|
||||
import CredentialsSettingsTab from '../view/tabs/api-settings/CredentialsSettingsTab';
|
||||
import VoiceSettingsTab from '../view/tabs/VoiceSettingsTab';
|
||||
import GitSettingsTab from '../view/tabs/git-settings/GitSettingsTab';
|
||||
import BrowserUseSettingsTab from '../view/tabs/browser-use-settings/BrowserUseSettingsTab';
|
||||
import NotificationsSettingsTab from '../view/tabs/NotificationsSettingsTab';
|
||||
@@ -157,6 +158,8 @@ function Settings({ isOpen, onClose, projects = [], initialTab = 'agents' }: Set
|
||||
|
||||
{activeTab === 'api' && <CredentialsSettingsTab />}
|
||||
|
||||
{activeTab === 'voice' && <VoiceSettingsTab />}
|
||||
|
||||
{activeTab === 'plugins' && <PluginSettingsTab />}
|
||||
|
||||
{activeTab === 'about' && <AboutTab />}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { Bell, Bot, GitBranch, Info, Key, ListChecks, MonitorPlay, Palette, Puzzle } from 'lucide-react';
|
||||
import { Bell, Bot, GitBranch, Info, Key, ListChecks, Mic, MonitorPlay, Palette, Puzzle } from 'lucide-react';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
|
||||
import { cn } from '../../../lib/utils';
|
||||
import { PillBar, Pill } from '../../../shared/view/ui';
|
||||
import type { SettingsMainTab } from '../types/types';
|
||||
@@ -20,6 +21,7 @@ const NAV_ITEMS: NavItem[] = [
|
||||
{ id: 'appearance', labelKey: 'mainTabs.appearance', icon: Palette },
|
||||
{ id: 'git', labelKey: 'mainTabs.git', icon: GitBranch },
|
||||
{ id: 'api', labelKey: 'mainTabs.apiTokens', icon: Key },
|
||||
{ id: 'voice', labelKey: 'mainTabs.voice', icon: Mic },
|
||||
{ id: 'tasks', labelKey: 'mainTabs.tasks', icon: ListChecks },
|
||||
{ id: 'browser', labelKey: 'mainTabs.browser', icon: MonitorPlay },
|
||||
{ id: 'plugins', labelKey: 'mainTabs.plugins', icon: Puzzle },
|
||||
|
||||
91
src/components/settings/view/tabs/VoiceSettingsTab.tsx
Normal file
91
src/components/settings/view/tabs/VoiceSettingsTab.tsx
Normal file
@@ -0,0 +1,91 @@
|
||||
import type { InputHTMLAttributes } from 'react';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import SettingsSection from '../SettingsSection';
|
||||
import SettingsToggle from '../SettingsToggle';
|
||||
import { useUiPreferences } from '../../../../hooks/useUiPreferences';
|
||||
import { useVoiceConfig } from '../../../../hooks/useVoiceConfig';
|
||||
|
||||
const inputClass =
|
||||
'w-full rounded-md border border-border bg-background px-3 py-2 text-sm text-foreground placeholder:text-muted-foreground focus:outline-none focus:ring-2 focus:ring-ring';
|
||||
|
||||
function Field({ label, ...props }: { label: string } & InputHTMLAttributes<HTMLInputElement>) {
|
||||
return (
|
||||
<label className="block space-y-1">
|
||||
<span className="text-sm font-medium text-foreground">{label}</span>
|
||||
<input className={inputClass} {...props} />
|
||||
</label>
|
||||
);
|
||||
}
|
||||
|
||||
export default function VoiceSettingsTab() {
|
||||
const { t } = useTranslation('settings');
|
||||
const { preferences, setPreference } = useUiPreferences();
|
||||
const { config, update } = useVoiceConfig();
|
||||
const voiceEnabled = preferences.voiceEnabled;
|
||||
|
||||
return (
|
||||
<div className="space-y-8">
|
||||
<SettingsSection title={t('voiceSettings.title')} description={t('voiceSettings.description')}>
|
||||
<div className="flex items-center justify-between rounded-lg border border-border p-3">
|
||||
<div className="pr-3">
|
||||
<div className="text-sm font-medium text-foreground">{t('voiceSettings.enable')}</div>
|
||||
<div className="text-xs text-muted-foreground">{t('voiceSettings.enableDescription')}</div>
|
||||
</div>
|
||||
<SettingsToggle
|
||||
checked={voiceEnabled}
|
||||
onChange={(v) => setPreference('voiceEnabled', v)}
|
||||
ariaLabel={t('voiceSettings.enable')}
|
||||
/>
|
||||
</div>
|
||||
</SettingsSection>
|
||||
|
||||
{voiceEnabled && (
|
||||
<SettingsSection title={t('voiceSettings.backendTitle')} description={t('voiceSettings.backendDescription')}>
|
||||
<div className="space-y-4">
|
||||
<Field
|
||||
label={t('voiceSettings.baseUrl')}
|
||||
placeholder="https://api.openai.com/v1"
|
||||
value={config.baseUrl}
|
||||
onChange={(e) => update({ baseUrl: e.target.value })}
|
||||
/>
|
||||
<Field
|
||||
label={t('voiceSettings.apiKey')}
|
||||
type="password"
|
||||
autoComplete="off"
|
||||
placeholder="sk-…"
|
||||
value={config.apiKey}
|
||||
onChange={(e) => update({ apiKey: e.target.value })}
|
||||
/>
|
||||
<div className="grid grid-cols-1 gap-4 sm:grid-cols-4">
|
||||
<Field
|
||||
label={t('voiceSettings.sttModel')}
|
||||
placeholder="whisper-1"
|
||||
value={config.sttModel}
|
||||
onChange={(e) => update({ sttModel: e.target.value })}
|
||||
/>
|
||||
<Field
|
||||
label={t('voiceSettings.ttsModel')}
|
||||
placeholder="tts-1"
|
||||
value={config.ttsModel}
|
||||
onChange={(e) => update({ ttsModel: e.target.value })}
|
||||
/>
|
||||
<Field
|
||||
label={t('voiceSettings.voice')}
|
||||
placeholder="alloy"
|
||||
value={config.ttsVoice}
|
||||
onChange={(e) => update({ ttsVoice: e.target.value })}
|
||||
/>
|
||||
<Field
|
||||
label={t('voiceSettings.format')}
|
||||
placeholder="mp3"
|
||||
value={config.ttsFormat}
|
||||
onChange={(e) => update({ ttsFormat: e.target.value })}
|
||||
/>
|
||||
</div>
|
||||
<p className="text-xs text-muted-foreground">{t('voiceSettings.note')}</p>
|
||||
</div>
|
||||
</SettingsSection>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
Reference in New Issue
Block a user