A local-first Somali speech pipeline: preserve the voice, recover its timing, and train a model that can carry both pronunciation and rhythm.
Research noteBuilding Somali prosody from the signal, not a punctuation guess
The project started with a practical constraint: Somali speech data is valuable only when the transcript, the speaker, and the timing remain connected. The pipeline turns approved long-form recordings into short, trainable clips without treating transcription as a one-way export. Each stage leaves a reviewable artifact behind — cleaned FLAC, source transcript JSON, derived text, metadata, manifests, and training outputs — so mistakes can be corrected without destroying provenance.
The most important research question became pauses. Raw transcripts often flatten a natural delivery into a stream of words, while the recording still contains the breath, hesitation, emphasis, and sentence boundary. ElevenLabs word timestamps make that recoverable. For adjacent words, the gap is computed as next.start minus current.end; conservative thresholds then propose ordinary commas, periods, or ellipses. The audio never changes, and there are no invented pause tags. Instead, normal punctuation becomes an alignment cue the model can learn from repeated audio-text examples.
CosyVoice separates the work into two learnable pieces. The LLM maps Somali text toward speech tokens, where pronunciation, phrasing, and linguistic timing live. The Flow model maps those tokens toward acoustic detail and temporal realization. Training both on the same aligned corpus is deliberate: the goal is not a second dataset of artificial rhythms, but better preservation of the rhythms already spoken in the source material. Independent checkpoints make it possible to compare an LLM-first change, a Flow-first change, and a paired change against the same held-out clips.
The current workflow is designed like an experiment loop rather than a one-shot training run. Dataset revisions are versioned, timestamp-derived punctuation is sampled for human listening review, short natural phrases and held-out passages are compared across checkpoints, and only then are the best model pairings promoted into local inference. That makes improvements in Somali pronunciation measurable without losing sight of the thing that makes a voice feel human: where it waits, breathes, lands, and continues.
8,745Audio-text pairs
98 / 2Train-dev split
24 kHzFinal mono FLAC
LLM + FlowSeparate checkpoints
