One speech synthesis model for Hausa, Zarma and French
We grew one voice-cloning synthesis model from Hausa to Zarma and then French, and measured every language at each step. The three-language model holds all three within tolerances fixed before scoring, with French judged by one recogniser on a repaired test set.
What we found
Accent
Training on Hausa alone carried Hausa letters into Zarma
After training on about 128 hours of Hausa, a recogniser with a language model found Zarma as intelligible as before: 11.5% against 11.8%.
Natural Zarma speech: 1 segment in 888 holds one.
Hausa writes these four hooked letters.
Two recognisers read the same renders back
% character error, folded
Each tile is one render: 140 held-out sentences in 5 voices. The data does not say which renders held a hooked letter, or which letter: their places are arbitrary. Bands: 95% intervals.
A CTC recogniser, which has no language model, heard a difference: 13.7% against 11.7%. 46 of 700 Zarma renders now contained a Hausa hooked letter, against 7 for the base model and 1 in 888 natural Zarma segments.
Replay
Mixing Hausa back in kept it
We then trained on 3.6 hours of Zarma with Hausa in a quarter of samples. Started from the public base, Hausa read back at 3.0%, against 1.8% for version 1.0.
Choose where to start and how much Hausa to keep
About a quarter Hausa
40% Hausa
From the public base
Not trainedThe 40% share was tried from version 1.0 only.From version 1.0
HausaZarma
Hausa read back, % character error (recogniser with a language model)
toleranceversion 1.0
1.8%public base
2.9%1.9%1.52.02.53.03.5No measurable change: +0.11 points (−0.12 to +0.35). This became version 1.1.
Zarma renders with a Hausa hooked letter: 3 of 700 (version 1.0: 46)
Hausa: 334 targets from 20 speakers that are in version 1.0’s training data. The tolerance, fixed before scoring: at most 0.5 points worse than version 1.0. Differences are paired, with 95% intervals; the share bars are approximate.
Started from version 1.0 with Hausa in about a quarter of samples, Hausa read back at 2.1%: inside the tolerance, but a measurable change. With Hausa in 40% of samples, Hausa read back at 1.9%, no measurable change from version 1.0. This became version 1.1. Version 1.1 also improved Zarma by 1.1 to 3.2 points over version 1.0, and the Hausa letters left: 3 renders of 700.
French
French was kept, and now it is practised
One model, three versions: Hausa first, then Zarma with Hausa kept, then French with both kept.
French practised, and all three stay inside their tolerance.
Hausaadded in 1.0, about 128 h
2.3%, against 2.4% for 1.1inside the tolerance
Zarmaadded in 1.1, 3.6 h
10.5%, against 10.4% for 1.1inside the tolerance
Frenchadded in 1.2, 6.5 h
2.4%, against 2.2% for the public baseinside the tolerance
tolerance: at most 0.5 points worsethe version it is checked againsttrainednot trained yet
Read back by the recogniser without a language model, folded character error; bands are 95% intervals where the source has them; window: at most 0.5 points worse. Each version is checked against the one before it in one evaluation run, French against the public base. 1.1 was measured in both runs: Hausa 2.3% and 2.4%, Zarma 10.5% and 10.4%.
Before any French training, version 1.0 read French back at 2.1% and version 1.1 at 2.2%, level with the public base at 2.2%. Version 1.2 practised French in about a fifth of samples and read back at 2.4%, within the 0.5-point tolerance of the base, on the repaired test set. It kept Zarma at 10.5% against 10.4% for version 1.1, and Hausa at 2.3% against 2.4%.
Estimator
Who is in the data depends on the estimator
A fast pitch-based estimator put women at about 14% of the French training hours.
MFMFMFMFMFMFMFMFMFMFMFMFMFMFMFMFMFMFMFMFplaced alike by both estimators and the toolkita woman, as the toolkit placed her
Pitch estimates, not human labels. The places of the 20 are arbitrary; for the 44 others, whom all three placed alike, the data does not give the gender.
Checked against a phonetics toolkit on 64 Hausa speakers, it disagreed on 20, each time calling male a speaker the toolkit placed as female. A more careful pitch tracker agreed on all 64 and found women at 34% of the French hours.
What comes next
- Native listening panels in Hausa, Zarma and French, with decision rules written first.
- New Zarma speakers recorded and consented for synthesis.
- A French training pool balanced by a checked gender estimator, with more Nigerien speech.
- Recordings of laughter, sighs and other expressive speech in Zarma and Hausa.
The paper
One Speech Synthesis Model for Hausa, Zarma and French: Adding Languages One at a Time
Namu AI LabNovember 202614 pagesPDF
Abstract
We grew one voice-cloning speech synthesis model from Hausa to Zarma and then to French, and measured at every step whether the new language cost the earlier ones. Starting from a public massively multilingual model, we fine-tuned it on about 128 hours of read Hausa (version 1.0), then on about 3.6 hours of Zarma from five speakers with Hausa in 40% of the training samples (version 1.1), then added French at about a fifth of the samples (version 1.2). Two recognisers of different architecture read the renders back (one in the French stage); we report folded character error rate (CER) with 95% intervals, and count Hausa hooked letters in Zarma read-backs as a proxy for a Hausa accent. Training on Hausa alone kept the base model's French but, by this proxy, carried Hausa sounds into Zarma: 46 of 700 Zarma renders contained a Hausa hooked letter, against 7 for the base model. Training on Zarma with 40% Hausa lowered Zarma CER by 1.1 to 3.2 points against the Hausa model, removed the hooked letters (3 of 700) and moved Hausa by +0.11 points (interval −0.12 to +0.35); the same Zarma data with Hausa in a quarter of the samples, started from the public base instead, ended 1.2 points behind the Hausa model on Hausa. The three-language model holds French, Zarma and Hausa within tolerances fixed before scoring, with French judged by one recogniser on a test set we repaired after the registered set failed. Several effects outside the model were as large as the effect of training: the choice of voice spread Zarma CER from 9 to 16%, digits left as numerals raised it to about 20%, silence in a reference clip became long pauses in the renders, one truncated reference transcript made every system fail on its voice, and a pitch-based estimator put women at 14% of the French hours where a more careful one found 34%. All results are automatic measurements; no native-speaker panel has rated any output.
Give this research more voices
What comes next needs consented speech from many more speakers. If you speak Hausa or Zarma, you can record short missions from your phone.
