Findings
Results from the READSmart study, with the sample, method, and limitations for each experiment.
- 01Finding 1: Agreement between teachers
Teachers agreed strongly on words read correctly, while agreement on miscues and reading accuracy was less certain.
- 02Finding 2: Comparing Whisper model sizes
The medium model had the lowest word error rate in this sample; we also measured processing time to compare its hardware cost with the other models.
- 03Finding 3: Calibrating scores against teacher measurements
Calibration reduced the average reading-speed offset against teachers from -13.603 to 0.149 WCPM in student-grouped cross-validation, while individual errors remained substantial.
- 05Finding 5: Comparing language configurations
One bilingual calibration model with a language flag had the lowest average WCPM error, with small differences between the three configurations.
- 06Finding 6: Exploring groups in the recording measurements
Groups formed from automated recording measurements had only a weak relationship with teacher-assigned Phil-IRI levels in this sample.
- 07Finding 7: A spelling difference can look like a reading error
An approved name spelling such as “Jackie” for “Jacky” should not lower the automated reading estimate solely because its spelling differs; the correction fixes that regression, while rescoring all 1,290 saved model results changed none because the study corpus contains different passages.