← Build log

🧠 From Voice Recorder to Voice Agent: A Real-Time Case Study in AI-Powered Product Evaluation

📍 Starting With the Right Question

This morning, I opened with a deceptively simple question:

“Is there a place for VoiceCatcher on the iPhone?”

VoiceCatcher is a desktop-first, privacy-first voice transcription app built on whisper.cpp. It works beautifully as a local recorder and summarizer — but that trust gap still exists when you speak and have to wait to see your words. That’s where the thread started. What followed was a fast, iterative evaluation of product gaps, opportunities, and technical feasibility — powered by a tight feedback loop between human judgment and AI support.

In this post, I want to walk you through how we:

  • Identified critical user trust gaps
  • Explored competitive mobile positioning
  • Assessed real-time transcription experience needs
  • Validated whether an AI agent adds real value or just bloat
  • Designed modular architecture and prompts to prototype safely

This is the kind of outcome-driven, AI-partnered work solo builders and technical PMs need to master.

⚡ Gap #1: No Real-Time Transcription = No Trust

We started by acknowledging a core UX gap: users can’t see their words appear as they speak.

It’s a common problem in batch transcription systems (like Whisper), but a fatal flaw if trust is the cornerstone of your app.

We explored:

  • 🧪 Using micro-buffers with whisper.cpp
  • ⚡ Using real-time engines like VOSK for live previews
  • 🤝 Hybrid solution: real-time display via VOSK + high-accuracy post-processing with Whisper

This insight alone was product-defining: live transcription is not a bonus — it’s a trust multiplier.

🕵️‍♂️ Evaluating the Mobile Opportunity

The next thread was a strategic one:

“Is there room for VoiceCatcher on iOS if dictation is already built in?”

We did a rapid market landscape analysis:

  • iOS already handles speech-to-text, but only via cloud or Siri
  • No iOS app today offers: offline, local, structured agentic voice commands
  • Whisper-based tools exist, but none ship with bundled privacy-first LLM agents

Key outcomes:

  • ✅ Desktop is still the right place to win first
  • 🧪 A companion iOS app for quick-capture + sync is worth exploring later
  • 🚫 Replacing Apple’s native dictation is not a winning strategy

🧠 Is an AI Agent a Gimmick or a Game-Changer?

We asked the right follow-up:

“Would an AI agent that could turn your last 3 voice notes into bullets or summaries feel magical… or gimmicky?”

After sketching the UX, we realized:

  • If it’s embedded naturally, not branded as ‘chat’, and feels proactive — it’s magic
  • If it requires remembering commands or speaking like a robot — it’s a feature graveyard

We scoped core actions:

  • summarize
  • bullet_list
  • tag
  • noop

We planned for:

  • Structured JSON output via llama.cpp
  • Minimal UI that suggests actions based on context
  • Full modular isolation so it never breaks the core app

🔧 Modular AI Agent Architecture

We knew we didn’t want to break the working app — so we defined:

Agent Folder Layout:

/agent
  /prompts
  agent_router.ts
  agent_test.ts
  run_agent.ts
  README.md

LLM Tech Stack:

  • llama.cpp + phi-2 or gemma (GGUF quantized)
  • JSON-based intent parsing
  • Fallback to noop if confidence is low

UX Strategy:

  • After transcription: show 2–3 action suggestions
  • Let user click or say “Summarize that”
  • Let power users automate or disable

🤝 Working With AI to Build This Fast

We partnered with Claude Sonnet to:

  • Generate prompt engineering templates
  • Build modular, testable TypeScript stubs
  • Define folder layout and feature flags
  • Ensure commercial-safe model selection (MIT / Apache 2.0)

Then we made it Cursor-ready:

  • ✅ Prompts saved as Markdown for iterative work
  • ✅ Feature flags gated behind ENABLE_AGENT=true
  • ✅ UX and logic fully decoupled from core Whisper pipeline

🎯 Outcome-Driven Framework for Solo Builders

If you’re a solo dev or engineer trying to level up as a product builder, take this approach:

1. Start with a core user pain or trust gap

→ e.g. “I can’t see what I’m saying in real time”

2. Evaluate your competition + constraints

→ e.g. “Apple handles dictation, but not privately or agentically”

3. Validate tech feasibility without committing

→ e.g. VOSK + Whisper hybrid, modular agent architecture

4. Use AI tools as collaborators, not generators

→ Prompt AI to create clean scaffolds, not your vision

5. Always protect your working system

→ Build as plugins, wrap in feature flags, test before integration

🚀 What’s Next

With this architecture and thinking in place:

  • We’re shipping an agentic layer to Whisper, powered by local LLMs
  • We’ll optionally include real-time VOSK previews
  • We’ve saved all prompts as a reusable system for future solo projects

This was a powerful reminder: you don’t need a 5-person product team. You need:

  • A clear outcome
  • A trustable system
  • And the ability to partner with AI to move faster, safer

✅ This markdown file is one of our internal development blog posts (#32) and is stored in our blog/ folder in the project.