Finding 05
Finding 5: Comparing language configurations
In one sentence: One bilingual calibration model with a language flag had the lowest average WCPM error, with small differences between the three configurations.
Sharing a calibration model across languages
READSmart supports English and Filipino. Separate calibration models can fit each language independently; a shared model has more data to learn from and uses a language flag. The dataset has fewer than 100 recordings per language.
Speech model: medium. Recordings: 167 (83 English, 84 Filipino).
All three options were tested on the same held-out recordings.
Reading speed results (words per minute)
| Configuration | English | Filipino | Both | Offset (both) |
|---|---|---|---|---|
| (raw measurement) | 9.447 | 20.561 | 15.037 | -13.60 |
| A: separate model per language | 8.006 | 12.395 | 10.214 | +0.07 |
| B: learn on the other language, re-centre | 8.742 | 12.326 | 10.544 | +0.36 |
| C: one bilingual model + language flag ← primary | 7.862 | 12.198 | 10.043 | +0.15 |
The English, Filipino, and Both columns show mean absolute error in WCPM. Lower is better. The final column shows the mean signed offset across languages.
Small differences between configurations
The lowest and second-lowest mean absolute errors differ by 0.17 WCPM. For context, the estimated teacher-to-teacher disagreement band is around ±9.6 WCPM (Finding 01). The small observed difference gives little reason to prefer the more complex separate-model setup. The single bilingual model is the simpler option to maintain.
Higher error in Filipino
Configuration C has a mean absolute error of 7.862 WCPM in English and 12.198 WCPM in Filipino, about 1.6× higher.
Finding 02 also found higher Filipino transcription error at every model size. None of these calibration configurations removed the scoring-error gap. Deployment evaluation should report each language separately.
Limitations
- The smaller language group has 83 recordings, limiting confidence in small differences between configurations.
- Configuration B learns the relationship from the other language, then corrects the offset locally. More complex transfer methods were not tested.
- Ridge was held fixed across all three so this compares configurations, not algorithms.
Generated by pipeline/language_configs.py.