Follow Us




Features Communication and the Japanese Language
A Warning About Japan’s Future? AI Outperforms Students at Reading

Professor Arai Noriko has predicted that artificial intelligence will perform half of all white-collar jobs in 2030. Her project pitting an AI against the University of Tokyo entrance examination got her thinking about how well young students are actually able to read. She started researching a new test to assess reading ability and draw attention to those junior high and high school students who cannot read well enough to understand their textbooks.

Arai Noriko

Arai NorikoProfessor at the National Institute of Informatics and director of the Research Institute of Science for Education. After graduating from Hitotsubashi University, completed a doctorate in mathematics at the University of Illinois. Specializes in mathematical logic. Launched the “Can a Robot Get into the University of Tokyo?” project in 2011. Began research on the Reading Skill Test for assessing reading ability in 2016.

The Limits of AI

“Can a Robot Get into the University of Tokyo?” If you heard the name of this project, you would be forgiven for thinking that the aim was to get a robot to pass the prestigious institution’s entrance examinations. Project leader Arai Noriko of the National Institute of Informatics explains, however, that the goal was not to show what an artificial intelligence could do, but rather what it could not—to clarify the limits of AI compared with humans.

After some six years of research, the AI ultimately failed to pass the test, but posted a deviation score of more than 57, putting it in the top 20% of final-year high school students. This means it would be able to pass the entrance exams for some famous private universities. However, it did not solve questions by understanding the meaning of words. For example, Tōrobo-kun, as the AI is called, wrote essays by searching textbooks and Wikipedia, picking out and arranging sentences, and then polishing the text. The result was better than most students’ essays. How could an AI that is unable to read and understand sentences perform better than a human? In pondering this question, Arai began to wonder about the reading ability of students in seventh through twelfth grade.

Assessing Students’ Reading

Arai developed a Reading Skill Test, which has been taken by more than 40,000 students since April 2016. She says it is very unusual for a noncompulsory survey to receive this level of response.

The test includes six types of questions: identifying what demonstrative pronouns like sore (that) and kore (this) or omitted subjects and objects are referring to (anaphora), identifying the subject and object (syntactic dependency), inferring from sentences based on logic and common sense, judging which specific actions or things would fit a given definition, judging whether two sentences have the same meaning, and identifying which diagram matches a sentence. The questions are based on materials in the junior high school and high school textbooks, as well as dictionaries and newspapers. If students cannot read and understand the questions, they cannot read and understand these other everyday texts.

“Since I started writing an introductory mathematics textbook, I have frequently visited junior high schools. I eat lunch together with the students and talk to them, so I can learn from what point they find it difficult to understand. In San’ya, Tokyo, where there are many cheap lodging houses, I went every week to a meal center for two years and saw where people tripped up. This is all connected with the RST.”

  • [2018.04.05]
Related articles
Also in this series

Related articles

Video highlights

New series

  • From our columnists
  • In the news