Who is in the FLEURS Hausa test set

The Hausa test split of FLEURS holds 621 read utterances, about 3.3 hours of audio. Its metadata labels 620 of them female and one male.

FLEURS is a speech benchmark from Google that covers 102 languages (Conneau et al., 2022). Many papers report Hausa results on it. Its Hausa configuration is named ha_ng, for Hausa as spoken in Nigeria. We looked at who is in its test split, because a test set decides what a score can claim.

What the test split contains

We read the split’s metadata file, data/ha_ng/test.tsv in the google/fleurs repository on Hugging Face (CC BY 4.0), on 12 October 2026. It lists 621 utterances, about 3.3 hours of audio in total. The shortest lasts 6.1 seconds and the longest 78.9 seconds. The median is 17.9 seconds.

Each utterance carries a gender label. 620 are labelled female. One is labelled male, utterance 505 in file order, which lasts 12.8 seconds. The metadata gives no speaker identifier, so it cannot tell us how many different people were recorded.

Why it matters

A word error rate on this split describes how a model hears read Hausa from voices labelled female, in a set labelled Nigerian Hausa. That is useful, and it is the limit of what the split can show.

The split cannot support a separate score for men. One utterance is an example, and a rate computed on it would mean nothing. It also says nothing direct about Hausa from Niger. When a paper reports a single Hausa number on FLEURS, these limits travel with the number, whether or not the paper states them.

Its limits

• Read speech. FLEURS speakers read sentences from FLoRes-101, a translation benchmark built from English Wikimedia articles (Conneau et al., 2022). The register is written Hausa read aloud. Spontaneous conversation is absent.
• One variety. The configuration is labelled Nigerian Hausa. Nothing in it is labelled as Hausa from Niger.
• One gender label, almost entirely. 620 of 621 utterances carry the same label, so gender cannot be studied with it.
• Speakers unknown. Without speaker identifiers we cannot check how many voices the 3.3 hours contain, or whether a few speakers dominate.
• Spelling not yet checked. We have not yet checked how consistently its transcripts write the hooked letters ɓ ɗ ƙ ƴ. That check matters for scoring, and it will be part of our re-scoring note.

What we do with it

FLEURS is still worth reporting, because it lets results be compared with published work. When we report on it, we will name the split, the normaliser and its version, and the limits above.

For the claims FLEURS cannot support, we need other test sets. Rumbu plans a speaker-disjoint Hausa test set that states its dialects, genders and phone conditions, frozen before any training data is collected. Until it exists, we will say which questions we cannot yet answer.