Internals

Transcript flow

GitHub Edit page

Transcript Provider Flow

Goal: keep provider entrypoints thin; keep provider policy explicit.

#Provider entrypoints

  • packages/core/src/content/transcript/providers/youtube.ts

    • YouTube orchestration only. Web captions first. yt-dlp, Android VR direct media, then Apify fallback.

  • packages/core/src/content/transcript/providers/youtube/native-media.ts

    • Android VR direct-audio resolution + shared media transcription only.

  • packages/core/src/content/transcript/providers/podcast.ts

    • Podcast orchestration only. Feed/Spotify/Apple/enclosure/yt-dlp chain.

  • packages/core/src/content/transcript/providers/generic.ts

    • Thin orchestration only. Embedded tracks first. Direct-media / Loom / X media fallback next.

  • packages/core/src/content/transcript/providers/generic-embedded.ts

    • Embedded media detection + caption-track parsing only.

  • packages/core/src/content/transcript/providers/generic-direct-media.ts

    • Direct-media and Loom yt-dlp/transcription fallback only.

  • packages/core/src/content/transcript/providers/generic-twitter.ts

    • X/Twitter cookies + yt-dlp orchestration only.

#Shared policy

  • YouTube transcript and web-player requests share header construction in youtube/api.ts. Android-player headers remain separate, and the identity token remains web-player-only.
  • podcast/feed-flow.ts owns RSS transcript lookup, timestamp selection, and result shaping for direct feeds, Apple, and Spotify. Each source retains feed fetching, fallback decisions, and failure metadata; Apple's iTunes path still records the feed attempt before fetching.
  • podcast/media.ts shares download progress, transcription options, and completion across in-memory and temporary-file sources. Size limits and duration probing remain source-specific; temporary files are removed after success or failure.
  • content/link-preview/content/page-media.ts

    • HTML and Firecrawl share media detection, transcript resolution, source-metric refresh, and embedded article/transcript composition. Each page builder retains its metadata and Markdown policy; metric deadlines still start at builder entry.

  • transcription-capability.ts

    • One place for:

  • resolveTranscriptProviderCapabilities
  • canTranscribe
  • canRunYtDlp
  • missing-provider note/result shaping
  • transcription-start.ts

    • Runtime availability only. Local whisper, ONNX, cloud presence, display hints.

  • Local Whisper readiness checks for a usable model before starting the executable probe. FFmpeg segmentation and transcoding share process completion/error handling; MP3/WAV and lenient recovery retain their own arguments. OpenAI plain and diarized transcripts share HTTP submission but keep separate payload validation.

#Remote fallback

  • packages/core/src/transcription/whisper/cloud-providers.ts

    • Provider order + labels + model-id chain.

  • packages/core/src/transcription/whisper/remote-provider-attempts.ts

    • Per-provider byte/file attempts.

  • packages/core/src/transcription/whisper/remote.ts

    • Order loop only. Fallback notes. OpenAI chunk/delegate policy.

  • packages/core/src/transcription/whisper/diarization.ts

    • Explicit speaker-label requests only. Extracts local video audio once before provider attempts. ElevenLabs Scribe v2 first in auto mode, then OpenAI gpt-4o-transcribe-diarize.

  • YouTube yt-dlp media:

    • Diarization-only downloads audio. Combined slides + diarization downloads separate audio/video streams once and shares the video cache entry with slide extraction.

  • src/speaker-identification/

    • Optional post-processing for generic diarization labels. Timestamp anchors first, exact-transcript remembered mappings second, OpenAI GPT-5.5 context inference last. Named extracts have identity-aware cache keys; raw transcript cache entries remain provider-generic.

#Current order

  • Groq first when GROQ_API_KEY is set
  • local ONNX / whisper.cpp before the remaining cloud providers
  • remaining cloud bytes/file order:
  • AssemblyAI
  • Gemini
  • OpenAI
  • FAL
  • Deepgram
  • speaker diarization (--diarize):
  • ElevenLabs
  • OpenAI

#Rules

  • keep entrypoints thin
  • add provider notes in shared helpers, not scattered strings
  • prefer pure parser helpers before touching orchestration
  • if adding a new provider:
  • register cloud metadata
  • add remote attempt handler
  • widen shared capability helper
  • add focused provider tests before live tests