Published: January 11, 2025
Sometimes the most important product decisions happen at 1 AM when your beautiful transcription software is spitting out [BLANK_AUDIO] [BLANK_AUDIO] [BLANK_AUDIO] instead of, you know, actual words. That’s when you realize you’ve got a choice: slap a band-aid on the problem or dig into the messy technical reality that will make or break your product’s core value proposition.
Last night, staring at logs showing 62% silence in a 3-minute recording, I made the call to invest serious engineering time into what most founders would dismiss as “technical debt.” Here’s why that decision might be the difference between building something that works and building something that actually works.
Update: This post chronicles our complete journey from FFmpeg investigation through licensing discoveries to our final Core Audio solution, including the systematic prompt-driven approach that got us there.
TLDR: The 40-Hour Investment That Saves 400 Hours
- Problem: Removed SOX preprocessing (GPL licensing), audio quality crashed
- Wrong path: Investigated FFmpeg → licensing nightmare, bundling complexity
- Right path: Enhance existing Swift CLI with native Core Audio preprocessing
- Investment: ~40 hours of systematic engineering work
- Payoff: Zero licensing issues, zero external dependencies, production-ready audio quality
- Bottom line: Sometimes the best solution is enhancing what you already have
The Problem: When Audio Quality Determines Product Success
Our VoiceCatcher app had a problem that would kill the product if left unfixed. After removing SOX preprocessing due to GPL compliance requirements, our transcription quality nosedived:
- 62% silence in a 3-minute recording
- Multiple 20-second blank segments
- [BLANK_AUDIO] tokens instead of clean speech
- User trust evaporating with each failed transcription
The contextual paragraph formatter was working perfectly—it was correctly rejecting the blank tokens as invalid sentence boundaries. The issue wasn’t our algorithm; it was the garbage audio we were feeding it.
The Architecture Decision: FFmpeg vs. Core Audio
Phase 1: The FFmpeg Investigation
Initially, FFmpeg seemed like the obvious choice:
- 95% of SOX capabilities for audio preprocessing
- Commercial LGPL licensing (seemed viable)
- Industry standard for audio processing
But when we dug into the real requirements for a consumer DMG application, the picture changed dramatically:
FFmpeg Licensing Reality Check:
- LGPL compliance requires source code distribution
- Dynamic linking only (no static bundling)
- Attribution requirements in app and website
- GPL components must be disabled (
--disable-gpl --disable-nonfree) - Binary size impact: 50-100MB+ for full feature set
The Killer: FFmpegKit (the go-to Swift wrapper) was discontinued in 2024, leaving us with unmaintained dependencies for a commercial product.
Security Gotcha: When I was exploring quick fixes, Cursor actually recommended downloading a precompiled SOX binary from some random GitHub release page. That’s exactly the kind of shortcut that gets companies compromised—downloading random binaries from questionable sources and treating them as “dependencies.” My experience immediately raised red flags: if you’re building commercial software, you never download precompiled binaries from unofficial sources. This near-miss reinforced why a systematic approach was essential.
Phase 2: The Core Audio Revelation
Then we realized something obvious: our Swift CLI already uses Core Audio through AVFoundation. Why build external dependencies when we could enhance what’s already working?
Existing Foundation:
- ✅ 16kHz mono linear PCM output (perfect for Whisper)
- ✅ Zero external dependencies (pure Swift + AVFoundation)
- ✅ Commercial licensing (Apple’s frameworks)
- ✅ Production tested in our Electron app
- ✅ File + stdout support working flawlessly
The “Aha” Moment: Instead of replacing our audio system, we could enhance it with preprocessing filters using the same Core Audio framework it already uses.
The Systematic Approach: Solutions Architecture with Checks and Balances
Here’s where this gets interesting from a process perspective. I’m using ChatGPT as my solutions architect to craft prompts and double-check technical recommendations, creating a checks-and-balances system that’s been crucial for avoiding costly wrong turns.
This isn’t about replacing human judgment—it’s about augmenting decision-making with systematic analysis:
- Initial problem analysis → Human intuition + AI technical research
- Solution exploration → AI helps map licensing, technical, and integration challenges
- Implementation planning → Collaborative prompt crafting for precise requirements
- Risk assessment → Cross-validation of technical decisions
The FFmpeg→Core Audio pivot happened because this process helped us dig deeper into licensing implications and recognize we already had a better foundation.
The Implementation Plan: Two Targeted Prompts
Instead of a massive rewrite, we’re making surgical enhancements with two focused prompts:
Prompt 1: Swift CLI Enhancement
We have a working Swift CLI for audio capture built with AVFoundation. It already:
- Records 16kHz mono linear PCM audio
- Outputs to file or stdout
- Includes audio normalization
- Is built as a standalone CLI app for use in a commercial Electron app
- Has no GPL or system-level dependencies
Now we want to extend this same app with native **audio preprocessing** features to fully replace our old SoX pipeline.
🔧 Please add the following features:
1. **Optional Filters (via CLI):**
- `--highpass [freq]`: Apply high-pass EQ at specified frequency (e.g. 80Hz)
- `--lowpass [freq]`: Apply low-pass EQ (e.g. 8000Hz)
- `--normalize-volume`: Use AVAudioEngine to boost gain appropriately
2. **Chunking Support:**
- `--chunk-size [sec]`: Split audio into time-based chunks (e.g. 10 sec)
- Works with both `--file` and `--stdout`
- For `--file`, name like `audio_chunk_001.wav`
- For `--stdout`, send chunk boundary markers if needed
3. **Silence Detection Support:**
- Optional `--boost-silence-detection` to apply gain control or noise floor adjustment
- Improves compatibility with our Node-side silence shutoff logic
4. **Tests and Help Output:**
- Update CLI help text with all new flags
- Add Swift unit tests using `XCTest` to validate:
- Filters apply correctly
- Chunking works with valid durations
- Invalid args produce expected error messages
📦 This CLI is used in a commercial macOS app and must remain dependency-free. Do not use ffmpeg or SoX.
Do **not rewrite** the app — just enhance it.Prompt 2: Node.js Wrapper Integration
Our Node.js wrapper is already successfully calling our Swift audio-capture CLI and piping output to Whisper.
We've just enhanced the Swift CLI with new flags:
- `--highpass [freq]`
- `--lowpass [freq]`
- `--chunk-size [sec]`
- `--normalize-volume`
- `--boost-silence-detection`
Please update the Node wrapper to:
1. Accept these options as part of the config object
2. Append the correct flags to the Swift CLI spawn command
3. Add TypeScript types for the new options
4. Update Vitest tests to:
- Validate correct flag generation for config
- Confirm correct behavior in file and stdout mode
- Add mock support for testing chunk boundaries or filtering
This wrapper is part of a production Electron app — make sure it's clean, production-safe, and error-tolerant.Why This Approach Works: Technical and Business Alignment
Technical Wins:
- 4 weeks vs. 8-10 weeks implementation time (enhancement vs. rebuild)
- Zero licensing complexity (native macOS frameworks)
- Zero binary bloat (uses system-provided Core Audio)
- Proven reliability (building on working foundation)
- Real-time processing capability for chunking
Business Wins:
- No compliance risk from external audio libraries
- Faster time to market with incremental enhancement
- Lower technical debt than adding new dependencies
- Easier maintenance with familiar Swift codebase
- User trust preserved through reliable audio quality
Product Wins:
- Chunking support eliminates single points of failure for long recordings
- Intelligent silence detection reduces wasted processing time
- Contextual formatter preservation keeps our finely-tuned paragraph algorithm intact
- Backward compatibility ensures existing functionality remains untouched
The Bigger Lesson: Enhancement > Replacement
This project taught me something important about technical decision-making: the best solution isn’t always building something new. Sometimes it’s recognizing what you already have and making it better.
Our Swift CLI was already:
- ✅ Working reliably in production
- ✅ Using the right frameworks (Core Audio/AVFoundation)
- ✅ Producing the right output format (16kHz mono PCM)
- ✅ Commercially compliant with zero licensing issues
The “enhancement mindset” turned a potentially complex FFmpeg integration into a straightforward Core Audio feature addition. Same quality improvement, fraction of the risk.
Implementation Reality: The Next 40 Hours
Here’s what the actual work looks like:
Week 1-2: Swift CLI Enhancement
- Add AVAudioEngine preprocessing pipeline
- Implement CLI flag parsing for new options
- Add chunking logic with intelligent boundary detection
- Write comprehensive XCTest suite for all new features
Week 3: Node.js Integration
- Extend TypeScript interfaces for new audio options
- Update spawn command building logic
- Add Vitest test coverage for flag generation and edge cases
- Integration testing with existing transcription pipeline
Week 4: Testing & Polish
- End-to-end testing with real recordings
- Performance optimization for chunking workflows
- Documentation updates and final production validation
- Zero disruption to existing users during rollout
The Systematic Mindset
What made this decision process work was treating technical architecture like a business decision:
- Analyze true requirements (commercial licensing, zero dependencies)
- Evaluate hidden costs (maintenance, compliance, integration complexity)
- Leverage existing assets (working Swift CLI + Core Audio foundation)
- Plan incremental delivery (enhance, don’t replace)
- Validate with systematic prompts (AI-assisted technical due diligence)
The result? A solution that’s technically sound, commercially viable, and delivers faster than the “obvious” alternative.
Sometimes the best engineering decision is recognizing when you’re already most of the way there.
This is part of our ongoing series about building production-quality voice transcription software. The complete implementation plan and technical details are documented in our Core Audio Enhancement Strategy.