Collecting consented speech in the languages of the Sahel

What we found

1. Gap

   Few speakers, and few kinds of speech

   Of about 1,060 hours of public Zarma audio, about 10.7 hours are both openly licensed and transcribed, roughly 1%.

   All hoursOpen hoursSpeakers

   Those hours come from 7 speakers. The test set we plan has about as many hours, from 100.

   Fulfulde of central-eastern Niger

   3 speakers open, 9.5 h

   110 speakers test design, 10.2 h

   Kinds of open Zarma speech

   • Prompted spontaneous10.7 h
   • Conversationnone open
   • Telephonenone open
   • Childrennone open

   Where the open hours sit in the field has no meaning. Dots count speakers; how the hours divide among them is not in our data.

   That open Zarma comes from seven speakers, and open Fulfulde of central-eastern Niger from three. Our test-set designs plan 100 and 110 speakers. We found no open Zarma conversational or telephone speech, no children's speech, and no public test set with many speakers kept apart from training.

2. Speakers

   Five training voices were memorised

   Fine-tuned on about seven hours from five speakers, a recogniser's training loss fell from 125 to 37 while held-out loss rose from 153 to 221, on scales that may differ.

   LossError on the held-out speaker

   Updates

   5,000

   training speakers

   37at 3,850

   held-out speaker

   221at 3,500

   loss, as the training recipe reports it% folded character error, held-out speaker

   050100150200250202224262830

   the five voices memorised within about a thousand updatestraining speakersheld-out speaker

   best zero-shot recogniser, 25.5%within 0.2 points from 2,000 to 5,000 updateswithin 0.2 points

   01,0002,0003,0004,0005,000updates

   Losses as the recipe reports them; the two may be on different scales, so read their directions only. One held-out speaker and one transcriber: indicative only.

   On the held-out speaker, error fell from 29.0% to 25.7% folded character error rate, then stayed within 0.2 points from 2,000 to 5,000 updates. That is level with the best zero-shot recogniser, at 25.5%. With five speakers, longer training added little, so the programme aims first at the number of speakers.

3. Journey

   One recording, from phone to dataset

   Adults record short spoken missions on the phone or the web, offline if needed. 113 missions are published in four recording languages.

   up to 60 shashup to 60 shash0 bytes

   agreement revisionmission context

   frozen before the microphone opens

   Back

   Step 1 of 8

   Next

   Record Built

   The mission and the agreement revision are frozen before the microphone opens, and the recording points to the consent it was made under. Offline, the phone records only under a consent the server has already acknowledged.

   The web app sends units of at most 60 seconds, retrying from 15 seconds up to 5 minutes. A receipt needs the server to read back the stored bytes. Two evaluators judge each recording without knowing who spoke. Two agreeing judgments decide, a split adds a third, and a rejection always needs two people. In the design, accepted recordings enter dated, frozen versions, and a withdrawal is purged from them within 7 days.

4. Test

   A test set balanced by design

   The planned Zarma test set has 100 adults, 50 women and 50 men, in four age bands, recorded with a facilitator under evaluation-only consent.

   GenderAge bandRegion

   50 women and 50 men.

   a womana mangender not shown

   11.2 h hours by kind of speech

   • Read 2.8 h 20 new sentences each
   • Spontaneous 4.2 h about 2.5 min each
   • Conversation 1.7 h 10 pairs, 10 min each
   • Phone call 1.7 h 50 speakers
   • Voice note 0.8 h 50 speakers

   A design: nothing has been recorded. Each dot is one planned speaker.

   By place of upbringing: Niamey 30, the Dosso region 25, the Tillabéri river valley 20 and Zarmaganda 25. About 11.2 hours in all: read sentences, spontaneous speech, two-channel conversations, ordinary phone calls and voice notes, transcribed twice without automatic drafts.