Creating Test Contents with Generative AI to Publication | Exam Site

Introduction to Exam Site

We operate Exam Site, a platform that tests knowledge across various fields such as programming, AWS, and the Japanese Language Proficiency Test (JLPT N3). The site is currently in beta, but if you are interested, you can access it from the link below.

Exam Site

Methods of Creating Test Content

Creating all the questions, answers, and their explanations from scratch is extremely time-consuming. Beyond the content of the questions, it’s also necessary to review the wording and grammar of the text. Therefore, we utilize several AI services to facilitate the provision of our tests.
Today, I would like to introduce the process of creating questions for the JLPT N3.

1. Creating Test Questions

There are two methods of creating test questions:

• Using prompts: We create questions based on specific prompts given to ChatGPT.
• Creating from materials: ChatGPT reads content from specific literature or materials and creates questions based on that. For example, this test was created by reading the content of articles from this site.

2. Adding Images

We search for free images that match the questions and incorporate them.
(Since ChatGPT’s DALL-E 3 is available, we might also start using generated images from AI in the future.)

3. Conducting Reviews

Reviews are conducted in cooperation between humans and AI. The role of AI, using tools such as ChatGPT, Bing, and Google Bard, is to check whether the explanations and answers are correct.
Humans mainly check whether the content is appropriate as a question. Common issues include having multiple questions using incorrect options, choices being obviously incorrect, or the last option in a multiple-choice question almost always not being set as the correct answer.
There might be unbalanced problems due to issues in the prompts used in creating test questions. So, we need the combination of human and AI verification.

4. Creating Audio Files

We tried various options like Google’s Text to Speech and Natural Readers, and eventually, we settled on using Narakeet. The voice feels more natural compared to others, and it allows instructions to be given via script files. Because we can instruct via script files, we can switch conversation voices during audio file creation, eliminating the need for audio editing, and making re-output easy in case of minor edits.

Example Script

(voice: Takeshi)
4番：
(voice: Tomoka)
明日の会議は8時半からです。遅れないようにしてください。
(voice: Takeshi)
1.
(voice: Kasumi)
了解しました。
(voice: Takeshi)
2.
(voice: Kasumi)
構わないです。
(voice: Takeshi)
3.
(voice: Kasumi)
どういたしまして。


This is the sound file we created.


This concludes the process of creating a test.
Next, all that remains is to register this on the management screen of Exam Site. Here is the test that has been created in this process.
JLPT N3 問題 – 聴解①

Finally

Exam Site utilizes various technologies to create and publish efficient and diverse test contents. Through these processes, we will continue to provide test questions to everyone.

I would appreciate it if you could follow me on X (Twitter).
Train Japanese | Exam Site