A new experience where music finds us

This third installment, "The Future of Communication Opened by Biosignals," features neurowear's Yasuhiro Tsuchiya and Kana Nakanishi discussing their music project.

 

I feel like my concentration on music is declining

──When did you join neurowear, Mr. Tsuchiya?

Tsuchiya: I joined neurowear in April 2012. I've always loved music, and the music project started from the idea that music and brainwaves seemed like a good match. At neurowear, there are two product lines: the Nakanaka line and the Tsuchiya line. I'm in charge of the music-related approach.

──Are you the central figure in the music project?

Tsuchiya: I'd say I'm the driving force. We brainstorm together as a team, so ideas come from everyone. The first thing we made was the NEURO TURNTABLE, which we developed with our partner, Qosmo.

──Is the brainwave sensor used the same as in necomimi?

Nakano: Yes, it's the same one that measures concentration and relaxation.

Tsuchiya: The difference lies in what we replace concentration and relaxation with. For necomimi, we translated it into robotic movements. For NEURO TURNTABLE, we explored other ways to create interesting interactions, leading us to alter the music playback speed. It's a music player that stops playing if you're not concentrating on the music.

──Where did that idea come from?

Tsuchiya: I carry around lots of MP3s on my iPod and iPhone, but I often skip tracks while listening. I never listen to an album from start to finish. To be more precise, I rarely listen to a single song all the way through. It feels like my ability to concentrate on music has decreased precisely because I can carry so many songs with me now.

──Yeah, that's definitely a trend.

Tsuchiya: Before MP3s, you'd put a record or CD on the player, press play, and listen to an album with about 12 songs from start to finish, right? But that's completely disappeared. When I thought about how to regain the act of concentrating on listening to music, I realized that if there was a music player that structurally forced you to concentrate to listen, well, you'd have no choice but to concentrate (laughs). We created the NEURO TURNTABLE thinking it could renew the meaning of "listening to music is fundamentally an act of concentrating on the music." We initially developed it as an application, but since we had the chance, we linked it to an actual record player and created something where the pitch rises and falls in sync with brainwaves.

(“NEURO TURNTABLE” demo movie　 http://www.youtube.com/watch?v=Z0fOJCCiglQ )

──How did you proceed with the work alongside your partner, Qosmo?

Tsuchiya: Qosmo is a company that handles many projects combining music and technology. When I explained what data we could capture with the brainwave sensors, they responded with ideas like, "Well, we could do this," or "We could implement it on iOS." For NEURO TURNTABLE, they even made a prototype right away.

──Are you involved in other music projects as well?

Tsuchiya: Mr. Tokui from Qosmo, who also authored the book 'iPhone × Music: What the iPhone Predicts for the Future of Music', took an interest in neurowear's initiatives. Actually, besides NEURO TURNTABLE, we've also collaborated with Qosmo on projects like Brain Disco, ZEN TUNES, and mico.

──Brain Disco is a club event, right?

Tsuchiya: This is an event Qosmo hosts. It started from a casual conversation about how interesting it would be to hold a DJ contest measuring the brainwaves of the audience.

──What were the results?

Tsuchiya: Ironically, people were most focused when there was no music playing. When the music suddenly stopped, everyone's concentration spiked (laughs). So it felt quite unpredictable. The DJs had to figure out how to boost focus, like abruptly stopping a track and cutting in another song, creating a different kind of call-and-response dynamic than a typical dance floor.

──So the DJs figured out tricks to boost the audience's concentration?

Tsuchiya: Yeah. It's less about a trick and more about surprising people to boost focus. So they'd sprinkle in little gimmicks here and there, making it feel like a real-life Beatmania.

 

If only there were a way to directly search for non-verbal, sensory things

──What about ZEN TUNES?

Tsuchiya: It's an application that creates music playlists based on brainwaves.

("ZEN TUNES" demo movie　http://www.youtube.com/watch?v=fIrRhd2mRZY )

Nakano: ZEN TUNES was a project serving as a test case, perhaps even a precursor to mico.

Tsuchiya: Later, when we started feeling a bit stuck with the "concentration and relaxation" concept, we met Professor Mitsukura from Keio University. Applying Professor Mitsukura's new logic to the ZEN TUNES concept led to mico—headphones that generate music tailored to your brain state.

Nakano: Separate from the algorithm we discussed last time, Professor Mitsukura's lab also researches music. We thought, "This could help us realize ZEN TUNES in a more interesting way," and that's how it started.

──What exactly does this music research involve?

Tsuchiya: Broadly speaking, it's research into what characteristics appear in brainwaves when listening to different genres of music. Based on that research, we analyzed brainwave patterns when not listening to anything. We thought that if there was similarity to a music pattern, we could figure out music that fits that person.

──So you can figure out what music suits each person?

Nakano: More than just the person, it's about finding music that matches their state at that moment.

──So it plays music that matches the brain's state. Does it select from the songs you already own?

Tsuchiya: We unveiled mico at the SXSW (South by Southwest) music convention trade show in Austin, Texas, USA, this past March. At that time, it selected from a sample library of 100 songs we prepared. The latest version we're developing now is planned to work by analyzing the user's own music library.

Nakano: Ultimately, we envision it being integrated as an add-on to subscription-based music streaming services. In Japan, that would be equivalent to unlimited music streaming services.

Tsuchiya: Unlimited music streaming services are fantastic for music connoisseurs, but they present a challenge for those less knowledgeable—they often don't know what to listen to. That's why I think it would be great if, when someone wants to listen to music, it could select something that matches their mood. Current web search has limitations because you can only search for things you can verbalize. When you think, "What was that fluffy, crunchy song again?", it would be great to have a way to search directly for those non-verbal, sensory feelings. mico isn't there yet, but eventually, I think we'll be able to search directly for those non-verbal sensations using biometric information.

Nakano: It feels weird that just to listen to music , you have to start by studying genres for the search. When you're in the mood for something "popping and thumping," it'd be great to search directly with that sensation.

Tsuchiya: Exactly. For example, someone who doesn't know the word "metal" but wants to listen to metal might think, "What kind of music is dark and heavy, with gorillas running around, and sometimes horses too?" But it takes time to find the song they actually want to hear. Music can feel restrictive if you don't know the words, but people who don't know the words might actually listen more freely, unbound by genres. So, wouldn't it be cool if we could create opportunities to discover new music without needing to know the genres? That's the core idea behind mico.

──What's next for mico?

Tsuchiya: First, we'll expand the sample size beyond the initial 100 songs and make it easier to analyze in the future. Beyond that, we want to improve the recommendation accuracy to make it more satisfying.

Nakano: mico's core concept is providing new encounters with music. The name itself stands for "Music Inspiration from your subCOnsciousness," embodying the idea that "it would be great if music just came to you."

Tsuchiya: Like an oracle.

Nakano: It started with the idea of "wouldn't it be great if divine revelations came?"—so that's where the shrine maiden concept came from. Think of it as a shaman or something like that.

Tsuchiya: We're promoting the term "Music Serendipity." We want to create a new experience where, instead of active searching, the machine senses and finds the music for us.

("mico" instruction movie　http://www.youtube.com/watch?v=JyiXQgj_Nfk )

 

What I want to do musically is like scratch

Nakanaka: Oh, and mico also has a "feeling leak" feature.

──What's that? (laughs)

Nakano: Instead of sound leakage, having "emotion leakage" feels like something you'd expect from futuristic headphones.

Tsuchiya: There's a screen near the ear, and it analyzes brainwaves. When you feel stressed, an angry face appears, and when your concentration increases, an exclamation mark pops up. It's a feature Nakanishi-san really championed.

("mico" first demo with Julie Watai　http://www.youtube.com/watch?v=TGlUNL7z990 )

Nakano: Actually, that feature takes up about half the headphone's size... (laughs).

Tsuchiya: We added the screen to clearly show it was measuring brainwaves, thinking about standing out at SXSW. mico was created as a showcase to demonstrate how fun brainwave-sensing headphones could be, hence the large size, but we can make them more standard headphone size. Our partner, tsug, beautifully finalized the hardware design.

 

──Moving in a direction that lets your feelings show...

Tsuchiya: That's Nakano-san, right? (laughs)

Nakano: It's kind of fun, isn't it, when someone's thoughts spill out? People wearing headphones on the train seem exclusive, but imagine a girl wearing mico checking her email, and suddenly an angry emoji pops up—that's super cute. You start imagining, "Did she get an annoying message from her boyfriend?" or whatever (laughs). It makes you feel a sense of solidarity with the situation.

Tsuchiya: So it's about everything spilling out.

──What direction will the music project take from here?

Tsuchiya: Many people said that while it's great for mico to play music matching your current mood, it would also be nice to use music to control your feelings—like having it comfort you when you're sad or calm you down when you're angry. By analyzing brainwaves, we could define music's effects. If we could figure out that listening to this music tends to calm you down, it would be interesting to think of it like a music supplement.

Nakano: I'd love to see colorful mico headphones made for different effects.

──So if we have ideas that fit new technologies or analysis methods, we'll create them, or rather, we'll explore them.

Tsuchiya: Beyond just brainwaves, we could infer emotions from things like heart rate or sweat levels and use that to recommend music. For example, a cliché but practical idea: an app for runners that generates beats matching their heart rate. By matching the music's pace, they could achieve their best running form.

──So you're proposing new ideas.

Tsuchiya: What I want to do with music is similar to scratch DJing. Records weren't meant to be scratched, but scratching them sounded cool—that's the kind of new movement I want to create. I want to explore how unexpected uses of technology can lead to fascinating possibilities. It's not limited to music projects. If companies have technological seeds, I think they could consult us with "We have this technology," and we could offer proposals from a fresh perspective.