← Build log

Streaming Transcription: How to Show Live Text as Whisper Produces It (Without Breaking Your Final Result)

May 2026
SoloBuilderRustWhisperStreamingUXTauriLocalLLM

The specific bridge from whisper-rs’s segment callback to a Tauri event stream to a React hook — with the exact pattern that keeps streaming previews and final results perfectly in sync.

Published: May 2026 Tags: #SoloBuilder #Rust #Whisper #Streaming #UX #Tauri #LocalLLM

The “So What?” - Why You Should Care

Bottom Line Value: A specific pattern for turning whisper-rs’s segment callback into a live-streaming UX. The exact callback wiring on the Rust side, Tauri event emission, frontend hook for accumulation, and the rule that prevents streaming-vs-final mismatch.

What You’ll Gain:

  • The C-callback bridge from whisper-rs into Rust (this is the part the docs gloss over)
  • The Tauri event emission pattern for high-frequency updates
  • A React hook (useMemo based) that derives live text from incoming segments
  • The single rule that prevents “live preview != final result” bugs
  • ROI: ~2 days to ship. The perceived-performance win is enormous — users see text appearing within 1 second instead of waiting 5–10 seconds for the result

Who This Helps: Solo builders with batch-result transcription (or any batch LLM) that feels slow because users wait for the whole result. Anyone who needs to add streaming text UX to an existing in-process inference path.

Why Bother: Perceived performance is real performance. A 5-second transcription that shows nothing until the end feels slow. A 5-second transcription that streams text as it’s produced feels instant. The work is the same; the UX is dramatically different.

TLDR: The Move in 5 Steps

  1. Set up a segment callback in whisper-rs — set_segment_callback_safe_lossy (1 hour)
  2. Emit a Tauri event per segment — transcription-segment with the segment data (1 hour)
  3. Add a frontend listener — accumulate segments into a useState array (1 hour)
  4. Derive live text from segments — useMemo joining segments in order (30 min)
  5. Keep final result authoritative — don’t replace it with streaming preview text (the rule that prevents bugs) (30 min)

Result: Live streaming text during inference. Final result still authoritative. ~2 days of focused work.

The Moment You’re Probably In

You have in-process transcription working (see post #138). It’s fast. It produces clean results. But the UX is:

  1. User stops recording
  2. App shows “Processing…” spinner
  3. App stares at the spinner for 5–10 seconds
  4. Result appears all at once

Users tell you the app feels slow. You know it’s fast — the actual work is happening — but they perceive a multi-second blackout where nothing is visible.

Meanwhile, every consumer LLM app shows tokens streaming. ChatGPT, Claude, Otter — they all show progressive text. It’s table stakes now.

You’ve thought about adding streaming, but:

  • whisper.cpp’s docs talk about a “new_segment_callback” but the Rust binding is confusing
  • You’re not sure how to ferry callback data across the IPC boundary to your frontend
  • You’re worried about race conditions between streaming previews and the final result

The fix is a 3-tier pipeline: whisper-rs callback → Tauri event → React hook. Here’s how.

Step 1: The whisper-rs Segment Callback (1 hour)

whisper-rs exposes whisper.cpp’s segment callback through FullParams::set_segment_callback_safe_lossy. The “lossy” part means: if the channel is full, drop events rather than blocking inference. That’s the right choice — UI updates dropping is fine; inference stalling is not.

use whisper_rs::{FullParams, SamplingStrategy, SegmentCallbackData};

pub fn transcribe_with_streaming<F>(
    model: &WhisperModel,
    samples: &[f32],
    language: Option<&str>,
    mut on_segment: F,
) -> Result<TranscriptionResult, TranscriptionError>
where
    F: FnMut(TranscriptionSegment) + Send + 'static,
{
    let mut state = model.context.create_state()?;

    let mut params = FullParams::new(SamplingStrategy::Greedy { best_of: 1 });
    if let Some(lang) = language {
        params.set_language(Some(lang));
    }
    params.set_print_progress(false);
    params.set_print_special(false);
    params.set_print_realtime(false);

    // The segment callback fires once per produced segment, in order
    params.set_segment_callback_safe_lossy(move |data: SegmentCallbackData| {
        on_segment(TranscriptionSegment {
            text: data.text.clone(),
            start: data.start_timestamp as f64 / 100.0, // centiseconds → seconds
            end: data.end_timestamp as f64 / 100.0,
            index: data.segment as usize,
        });
    });

    state.full(params, samples)?;

    // ... then build the final TranscriptionResult from the completed state ...
    let num_segments = state.full_n_segments()?;
    let mut segments = Vec::with_capacity(num_segments as usize);
    let mut full_text = String::new();
    for i in 0..num_segments {
        let text = state.full_get_segment_text(i)?;
        let t0 = state.full_get_segment_t0(i)? as f64 / 100.0;
        let t1 = state.full_get_segment_t1(i)? as f64 / 100.0;
        segments.push(TranscriptionSegment { text: text.clone(), start: t0, end: t1, index: i as usize });
        full_text.push_str(&text);
    }
    Ok(TranscriptionResult { text: full_text, segments })
}

Three things to know:

  1. The callback fires in segment order, with each new segment as whisper.cpp finds boundaries.
  2. The closure must be Send + 'static because whisper-rs runs inference on internal threads.
  3. The lossy variant is preferred over the blocking one — UI updates dropping is fine; inference stalling is not.

Step 2: Emit Tauri Events (1 hour)

The Rust callback fires inside the transcription engine. Your IPC code (Tauri command) needs to ferry those events out to the frontend.

use tauri::Emitter;

#[tauri::command]
pub async fn stop_recording(
    state: tauri::State<'_, Arc<AppState>>,
    app: tauri::AppHandle,
) -> Result<RecordingResult, String> {
    let samples = state.recording.lock().unwrap().take_samples();
    let model = transcription::model::get_or_load_model(&state.model_path)?;

    // Wrap app handle for the callback
    let app_for_callback = app.clone();
    let result = transcription::engine::transcribe_with_streaming(
        model,
        &samples,
        Some("en"),
        move |segment| {
            if let Err(e) = app_for_callback.emit("transcription-segment", &segment) {
                tracing::warn!(error = %e, "failed to emit segment");
            }
        },
    ).map_err(|e| e.to_string())?;

    // Final result still goes via the regular command return — see Step 5
    Ok(RecordingResult::from(result))
}

The events flow to the frontend as JSON payloads. The frontend can listen via Tauri’s listen() API.

One non-obvious thing: tauri::Emitter::emit can fail (if the app handle is disposed). Don’t .unwrap(). Log and continue.

Step 3: Frontend Listener (1 hour)

In the frontend, listen for the event and accumulate segments into local state.

import { useState, useEffect } from 'react';
import { listen, UnlistenFn } from '@tauri-apps/api/event';

export interface TranscriptionSegment {
  text: string;
  start: number;
  end: number;
  index: number;
}

export function useStreamingSegments() {
  const [segments, setSegments] = useState<TranscriptionSegment[]>([]);

  useEffect(() => {
    let unlisten: UnlistenFn | undefined;
    listen<TranscriptionSegment>('transcription-segment', (event) => {
      setSegments(prev => [...prev, event.payload]);
    }).then(fn => { unlisten = fn; });

    return () => {
      if (unlisten) unlisten();
    };
  }, []);

  // Reset segments when a new recording starts (caller's job to invoke this)
  const reset = () => setSegments([]);

  return { segments, reset };
}

Call reset() when the user starts a new recording. Otherwise segments accumulate across sessions.

Step 4: Derive Live Text (30 minutes)

The segments are an array. Live text is the join. Use useMemo to derive once per update:

import { useMemo } from 'react';

export function useLiveText(segments: TranscriptionSegment[]): string {
  return useMemo(() => {
    return segments
      .slice()                        // copy to avoid mutating
      .sort((a, b) => a.index - b.index)
      .map(s => s.text)
      .join('');
  }, [segments]);
}

Why useMemo: high-frequency updates can otherwise re-compute on every render. With the memo, it only re-derives when segments actually changes (which is when a new segment arrives).

Render:

function ProcessingView() {
  const { segments } = useStreamingSegments();
  const liveText = useLiveText(segments);

  return (
    <div className="processing-view">
      <div className="processing-indicator">Transcribing...</div>
      <div className="live-text">{liveText}</div>
    </div>
  );
}

The user now sees text appearing as whisper finds segments. The actual transcription takes the same wall-clock time, but the perceived experience changes completely.

Step 5: The Rule That Prevents Bugs

The final result is authoritative. The streaming preview is not.

When the IPC command returns its result (the RecordingResult), that result replaces the streaming preview in the UI. The streaming preview was best-effort — it might have been throttled, segments might have been dropped (lossy), or your formatter might want a different version of the text.

function App() {
  const [finalResult, setFinalResult] = useState<RecordingResult | null>(null);
  const { segments, reset } = useStreamingSegments();
  const liveText = useLiveText(segments);

  const handleStopRecording = async () => {
    const result = await invoke<RecordingResult>('stop_recording');
    setFinalResult(result);  // final result becomes authoritative
  };

  // Use finalResult.text if available, otherwise liveText preview
  const displayText = finalResult?.text ?? liveText;
  // ...
}

This rule prevents two classes of bugs:

  1. Drift: streaming text and final text don’t match. Always trust the final result.
  2. Race conditions: if the user starts a new recording before the previous result is set, the live text resets but the final result might arrive late. Handle the race by checking session IDs or simply by trusting the final result over preview text.

What This Doesn’t Fix

  • Streaming doesn’t make transcription faster. Same wall-clock time. It changes perception, not performance.
  • The lossy callback can drop segments under heavy load. That’s fine for UI preview — the final result has all of them.
  • Some users prefer the final result only. Make streaming optional in settings if you want to be polite.
  • Formatting passes don’t run on streaming text in our setup. Streaming shows raw transcription; final result runs the full formatter. If formatting matters in the preview, run a lightweight formatter on the streaming text.
  • CSS transitions on rapidly-updating text can flicker. Use a stable rendering target (no fade-ins per segment) and let the text accumulate without animation.

Try It This Week

Saturday morning (3 hours):

  • Wire up the whisper-rs segment callback
  • Emit one Tauri event per segment
  • Verify in the frontend devtools that events arrive

Saturday afternoon (2 hours):

  • Build the useStreamingSegments and useLiveText hooks
  • Render live text in your processing view
  • Test with a 60-second recording — verify text appears progressively

Sunday morning (1 hour):

  • Add the final-result-overrides-preview rule
  • Reset segments on startRecording
  • Test the start/stop/start race condition

Sunday afternoon (1 hour):

  • Polish: subtle animation, scroll-to-bottom on long text, “show final result” replacement
  • Test with edge cases: very short recordings (<1s), very long recordings (>5min)

Monday:

  • Ship and watch your “this feels slow” support emails drop to near zero

That’s ~1.5 days of focused work for a transformative perceived-performance win.

The Takeaway

Streaming text isn’t a feature; it’s a UX baseline in 2026. Users expect it. Apps without it feel slow even when they’re not.

The path from batch transcription to streaming is mechanical:

  • whisper-rs callback captures segments as they’re produced
  • Tauri event emission ferries them across IPC
  • Frontend hook accumulates them into state
  • useMemo derivation turns the array into live text
  • Final result rule prevents preview/final drift

Two days of focused work. Same actual performance. Dramatically better perceived performance.

The pattern generalizes to any in-process LLM with a token or chunk callback. The bridge is always: callback → typed event → frontend accumulator → derived display. If you’re shipping a local LLM in 2026, this is the UX bar.

The streaming experience is one of the easiest “looks expensive, isn’t” wins in your codebase. Take the win.

Coming up next: post #143 — The parity closure doc. Once features are working and shipping, the next problem is that your phase plan no longer matches reality. Here’s the doc pattern that separates “what’s actually done” from “what we wish were done” — and why honest tracking is its own discipline.