Project Listen Reflection

STTREND Reflection Paper No:

Joseph E. Beck

Project LISTEN / May 31, 2012

Project LISTEN is an automated reading tutor that teaches reading to children by displaying stories on the computer screen and listening to them read the words aloud. The speaker and his team formed many research questions such as if particular parts of speech (POS) are more difficult than others, the factors that influence question difficulty, and how to improve evaluation of students through the use of Multiple Choice Cloze Questions. They chose Multiple Choice Cloze Questions because of the following advantages: (1) the questions constructed are well-specified and unbiased because of randomization, (2) the questions are automatically administered, scored and recorded, (3) there is longitudinal collection over the school year, and (4) there are a large number of students and questions. Their approach to predict individual question performance was to build a logistic regression model consisting of different factors (POS, common POS, POS confusability, level of difficulty, and student identity) and covariates (question length, deletion location, and number of choices with answer’s POS). They found out that there is a sort of hierarchy among the different parts of speech (noun < verb < adjective < adverb < function words). Also, longer questions and earlier deletion location tend to be harder. Aside from that, they found out that advanced readers have lower scores when the given choices can behave as different parts of speech. However, this does not mean that there is a decline of reading ability in advanced readers. Rather, this is more of due to the syntactic awareness of advanced readers as compared to beginner readers that are oblivious of the possibility of certain words having multiple parts of speech.

I think that the way they assess students is good enough. However, one little gripe I have is the randomization of the generated questions. I believe that the questions should not...