The Test Without a Right Answer The Secrets of Creative Testing (Part 2)

Following Part 1, this installment focuses on the "CR (Creative) Test." Conducted by Dentsu Inc. for employee training for over 30 years, this test has also contributed to discovering creators within the company. The "slightly unusual problems" unique to the CR Test have left a deep impression on many examinees. We explore the relationship between the CR Test and active learning while examining the importance of "questions without clear answers."

Professor Masashi Okuma, along with Dentsu Inc.'s Miyako Ito and Yu Kato, who have been involved with the CR Test for many years, discuss the CR Test and active learning. The session was facilitated by Nadezhda Kirillova, a researcher at the "Active Learning: What About This?" Institute.

From left: Director Brave, Mr. Okuma, Ms. Kirillova (Dentsu Inc.), Ms. Ito, Mr. Kato

 

 

In Japan it's a zero, in Canada it's a hundred

Nadia: Overseas schools sometimes have assignments without a single correct answer. For example, in Canada, there was a class where we made clocks. We got to decide what kind of clock to make ourselves. It could be simple or incredibly complex. The teacher evaluated both the difficulty level—how challenging the project was—and the level of completion—how well it turned out according to plan.

If you try to make something difficult and fail, that's bad. But if you make something simple, no matter how well you complete it, you won't get a high score.

The key is to create something good while balancing your own skill level with your ideas. But since there are no rules about what kind of clock to make, there's no single right answer.

Ōkuma: It's about challenging yourself with difficult techniques and seeing how much you can achieve. It's just like figure skating judging.

Nadia: In the international class at my French school, you rarely got full marks just by giving the standard answer to an assignment. Students didn't try to "find the answer the teacher expected"; they explored how to surprise the teacher, how to make them think, "Oh, they went this way!" They might even draw a picture for an essay assignment.

Ito: There's this "teacher's astonishment point," isn't there? Interesting.

Nadezhda: You start seriously thinking about how to come up with ideas that are different from everyone else's.

I once entered an essay contest in Canada when my English wasn't very good yet. I figured I couldn't beat native English speakers using conventional methods, so I wrote a poem and submitted it. Unbelievably, I won first prize.

If I'd done that in Japan, I'd probably have gotten zero points. Submitting a poem when it says "essay" is basically a foul play in Japan. A hundred points in Canada, zero points in Japan. There's a difference in how evaluators perceive "unconventional approaches."

Ōkuma: Why can't you do that in Japan? I think it's because teachers would get nervous if you did.

Naja: Having my poem evaluated for an essay contest made me realize, "Oh, that's acceptable," and it really boosted my confidence.

Kato: It's huge to be able to affirm that there are various possible answers. For problems like this, it's crucial for teachers to first broaden their own perspective.

Okuma: Hmm. Yes. I think developing the ability to create the future hinges on whether teachers can recognize children's ideas that go beyond their own standards and think, "I never thought of that!" If they can't do that, it won't lead to educational reform.

Naja: After leaving school, life doesn't have a single right answer, does it? People live by doing their best with what they find for themselves. That's why I think it's vital to increase those kinds of experiences from childhood.

Evaluation is done by advertising professionals over time

Ōkuma: How exactly do you evaluate the answers within Dentsu Inc.? I'd really love to hear about that.

Ito: Reading all the answers from hundreds of people is a huge task. The actual evaluations are done by our active copywriters and creative directors.

Ōkuma: What kind of answers tend to get higher evaluations?

Ito: Well, answers that make the evaluators genuinely think, "Wow, that got me!" tend to score high. Things that make you go, "I never thought of that approach" or "That's how they approached it." Also, answers that offer discoveries like, "So this is how young people think nowadays?" are good too.

Good answers are universally recognized, so they get decided quickly. For answers that spark debate, we discuss them. If someone passionately argues, "This entrant is great!" we might say, "Well then, why don't you try nurturing them at your agency?" (laughs).

The later answers in the 100-Idea Challenge are the most interesting.

Naja: I've had chances to see young people's answers too. When grading, they seem to value the breadth of ideas generated. They judge that people with broad thinking who come up with various answers have potential. Do schools have systems that evaluate that?

Okuma: From what I know, school teachers have their own yardsticks and evaluate students based on how closely they align with those standards.

Kato: First, the teachers need to change, right?

Okuma: Yeah, that's true, but what can we do?

Kato: Even if students say something outrageous, teachers need to properly acknowledge and evaluate it. To handle the diverse range of answers students might give, teachers need to broaden their tolerance.

Ito: We do a training exercise called "100 Ideas in a Row" for new employees. For example, we give them a problem like, "Think of as many ways to use a paper clip as possible." First, they try to come up with as many as they can in 3 minutes. Then they do it in 5 minutes, and the person who came up with the most gets to present. Doing this shifts the focus from the quality of the ideas to the sheer breadth of what they can think of.

Naja: When brainstorming 100 ideas, it's interesting how the ones written around the 80th idea—when they've basically given up and just jotted down anything—can be surprisingly good.

Okuma: Hold on a second. This is a big deal. In schools today, even with problems like "rubber bands," they're trying to get students to come up with just one best idea. If they can come up with one, getting them to think of another is already pushing it.

So there's no culture where "thinking of 100 ideas leads to interesting ones later." I realized the big problem is how teachers themselves can develop this "breadth of thinking."

When considering active learning, it's truly essential to encourage "generating many ideas" and "acknowledging diverse values," much like brainstorming. To achieve that, we might need to change the existing culture of schools.

Naja: For us, the biggest appeal of the CR test is that you can give various answers as test responses and have those values acknowledged.

Ito: While most school education focuses on increasing knowledge and developing logical thinking—essentially training the left brain—the power of the right brain is also crucial. It's said that great creativity happens when your thinking makes a sudden leap. It would be wonderful if we could nurture that aspect too.

Naja: At the Active Learning Institute, we're trialing something called "Weird Homework" at universities and companies. The content is almost identical to the CR Test, but we get reactions like, "Wow, it's okay to think such strange things!" The CR Test has this ability to spark excitement, whether in adults or children.

Okuma: If the know-how behind the CR Test's exciting approach to thinking spreads, it could bring a fresh breeze to school education.

Naja: I hope both children and teachers realize this soon—the joy of thinking, "It's okay to think such strange things!" It would be great to start by letting school teachers know about this fun and its potential.