Could VR be a key to healing this condition? The untold story behind developing VR-based remote therapy for phantom limb pain.

Phantom limb pain is a difficult-to-treat condition experienced by many people who have lost limbs due to accidents or illness. They feel as if the missing or paralyzed limb still exists (phantom limb) and experience pain in that phantom limb. The nature of the pain varies from person to person, and research into its causes and treatments has not progressed significantly.

Kazunori Inomata, representative of KIDS and himself a sufferer of phantom limb pain, was developing a phantom limb pain therapy system utilizing VR technology. In the summer of 2019, he met Atsushi Okada, who was part of the Open Innovation Lab (hereafter, InnoLab ) at Dentsu Inc. International Information Services (ISID), a company researching VR-based remote communication. Combining their expertise, they launched the "Phantom Limb Pain VR Remote Therapy System." We spoke with Inomata and Okada about the background of its development, specific initiatives, and the future potential of VR technology.
 

The Phantom Limb Pain VR Remote Therapy System is a system developed to alleviate phantom limb pain. It allows patients to connect with a therapist in a VR space from a remote location to receive therapy. The system recognizes the patient's movements via a head-mounted display (HMD) and reflects joint information in the VR space in real time. Within the VR space, the patient can see their own limbs as if they were present, enabling communication with the remote therapist as if they were face-to-face. It operates independently without connecting to networks or other devices (standalone type).

 

Because we are individuals who have suffered from phantom limb pain, we can achieve this. Creating groundbreaking mechanisms with cutting-edge technology.

From left in photo: Atsushi Okada (ISID), Kazunori Inomata (KIDS Representative / Vice Chairman, NPO Mission ARM Japan)

――Please tell us about the background leading to the development of the "Phantom Limb Pain VR Remote Therapy System."

Inomata: Since suffering severe injuries in an accident at age 17, I've been plagued by pain I couldn't express. At the time of the accident, I consulted multiple doctors, including my primary physician. However, the term "phantom limb pain" wasn't widely recognized then, so my pain was simply deemed a sequela. Years later, I happened to meet another doctor who suggested, "Could this be phantom limb pain?" Since I had always thought it was just a sequela, I never sought further consultation. By then, many years had passed since the accident. But I felt that if this pain had a name, there must be others suffering from it too.

I first learned about "mirror therapy," considered the only treatment for phantom limb pain, in 2006. Mirror therapy is a rehabilitation technique where the healthy limbs are reflected in a mirror, training the brain to perceive the missing limbs as present and moving. I consulted my primary doctor and tried it, but I didn't experience significant relief. However, I sensed the possibility that modifying the approach might reduce the pain.

At the time, I worked in manufacturing and routinely utilized CG technologies, including VR. I thought that applying this technology to recreate "mirror therapy" might yield better results. I believed it could help not only myself but also others suffering similarly. Furthermore, having survived life-threatening injuries, I felt a deep gratitude towards medicine and wanted to give back. That's why, in 2015, I founded KIDS to develop a phantom limb pain therapy system utilizing VR technology.

――When did the two of you meet?

Okada: KIDS and Inolab met in the summer of 2019. An employee at ISID's Hiroshima branch knew Mr. Inomata, and through him, Inolab was approached for consultation.

We thought that collaborating Mr. Inomata's VR phantom limb pain therapy system with Inolab's technology could lead to an even more groundbreaking new system. So, we had Mr. Inomata bring a demo unit of the VR therapy system, and the Inolab members experienced it. That was the start – discussing what the challenges were and what ISID might be able to contribute.

Kazunori Inomata, Representative of KIDS and Vice Chairman of NPO Mission ARM Japan (who is himself a phantom limb pain sufferer, as well as a developer and therapist)

From solitary training to therapy that many can enjoy like a game

――Please tell us about the challenges you faced during development.

Inomata: We wanted to make therapy enjoyable, not a lonely, painful exercise. Existing VR therapy systems had me, the therapist, right beside the patient, but the VR space was for the patient alone. Also, since pain can strike unexpectedly, there was strong demand for therapy anytime, anywhere. Furthermore, people wanting therapy weren't just in Tokyo; there were many across the country. To make it accessible to them, we needed to simplify the system—limiting some features—and create a way to connect with therapists in distant locations, as well as increase the number of therapists available.

Okada: Back then, HMDs were much bulkier devices than they are now. They came with sensors and cables, making them difficult for phantom limb pain sufferers to wear. A PC was also essential to run the application, and the processing speed significantly affected movement in VR, often making it jerky – a major challenge.
We felt it was essential to make the system more compact and simple so people could receive therapy easily and without stress.

Inomata: Exactly. We wanted to overcome these hurdles and create a system where therapy could be accessed easily, even remotely. We also envisioned a therapeutic approach with a game-like quality—where people could chat casually while playing with both hands, and before they knew it, they'd feel better.

Dentsu Inc. International Information Services (ISID) Open Innovation Lab, Atsushi Okada (engaged in R&D on xR, telepresence, ambient computing, and remote collaboration)

We aimed for development that didn't leave behind the feelings of those suffering from phantom limb pain

――How was the development carried out?

Okada: At the time, the Innovation Lab was conducting R&D on VR applications with Professor Koyama's lab at the University of Tokyo Graduate School. Coincidentally, Mr. Inomata was also discussing VR therapy systems with Professor Koyama. This connection led to the start of a joint research project between KIDS, the University of Tokyo, and the Innovation Lab.

Inomata: We involved many patients in the development process, advancing through repeated trial and error verification. Actually, until then, there had been no precedent for developing a therapy method that incorporated the perspectives of both the "affected individuals" experiencing phantom limb pain and the "developers" of the treatment. But pain is purely subjective; even from the standpoint of a pain specialist physician, it's impossible to understand the patient's feelings 100%. That's why I felt it was absolutely essential not to leave the affected individuals' feelings behind during development.

Okada: Similarly, I wanted to understand patients suffering from phantom limb pain. Back then, I knew nothing about it, but by listening to Mr. Inomata and patients share their experiences and trying out the demo unit, I gained a deeper understanding. The more I learned, the more I felt the profound significance of contributing to society through IT, and a determination grew within me that this couldn't be done half-heartedly.

Inomata: For patients, seeing the developers' faces offers a huge advantage. It fosters a sense of co-creation. Many patients started saying, "I want to be cured myself, but if it helps many other patients suffering like me, I'll cooperate however I can."

A hand recreated on the VR system. Through repeated verification with the patient, we achieved a natural position and smooth finger movements.

Through repeated proof-of-concept experiments, we refined the hand's position and movements to make them more realistic.

――Could you tell us about the proof-of-concept experiment that became a turning point?

Okada: There were two major ones. First, the experiment in December 2019 where both the patient and therapist Inomata entered the VR space simultaneously. This was a valuable opportunity to gain a real understanding of the suffering the patient experiences daily and how they experience VR therapy.
To alleviate pain, the key is whether the patient can perceive the VR world as their own and become immersed in it.
The key points were measuring the spatial relationship between the head and the position of the healthy arm/hand using sensors, estimating the position of the missing arm as a natural movement, and determining how smoothly the fingers should move. We made improvements while incorporating the patient's feedback.

Inomata: From a therapist's perspective, there were times when I simply couldn't convey the movements I wanted the patient to perform effectively with words alone. Being able to communicate these through gestures was a major step forward.

Okada: The second milestone was the actual remote proof-of-concept experiment we conducted in August 2020. Video smoothness improved, and accuracy definitely increased. Furthermore, we refined the device so that therapy could be received anytime, anywhere, requiring only Wi-Fi and an HMD.

Inomata: The heavy HMD was a struggle just to put on with one hand, but it became lighter and easier to wear.

Furthermore, enabling multiple patients to enter the VR space simultaneously allowed them to engage in "concurrent training" while conversing with each other. This led to feedback such as, "It's more enjoyable than training alone," and "We hope this system can help save even more patients."

Moving forward, we aim to leverage xR technology to create systems that contribute to society.

――The "Phantom Limb Pain VR Remote Therapy System" seems to be creating a new space for communication between patients. What further expectations do you have for xR technology going forward?

Inomata: Exactly. Patients can converse and share pain and suffering they couldn't previously discuss with others. Opportunities for those directly affected to connect are rare, aren't they? During the pandemic, gathering was difficult, and some people are shy and find it hard to attend such events alone. I feel VR spaces offer the advantage of allowing even such individuals to participate without hesitation. It would be wonderful to create a system that gradually heals emotional pain in this way.

I believe xR technology won't just stop at visualization; it could directly contribute to treatment and healing. For example, successful experiences in recovering from brain function disorders boost motivation and lift spirits. To help people share their struggles and pain while fostering a positive outlook, we really need to spread this system quickly...

Okada: Let's definitely do it!

Inomata: Exactly. The ISID team genuinely listened to and considered the reality of phantom limb pain and the patients' feelings. That's the primary reason I wanted to team up with ISID. I'm also grateful they're thinking deeply about the challenging task of developing scenarios to sustain ongoing activities.

Okada: Thank you. The system's greatest feature is enabling people in distant locations to feel as if they're right next to each other. The highly detailed visual representation of hand movements means we see potential not just for phantom limb pain treatment, but also for transferring manual skills.

For example, in manufacturing, we envision using VR for skill transfer and training in tasks requiring manual dexterity. When transferring technology to workers at overseas factories, it might enable a certain level of skill transfer without needing engineers to travel there or bringing workers to Japan. Furthermore, when facing constraints like tight deadlines, it might be possible to conduct extensive training beforehand within the VR space.
In consumer applications, we envision teaching knife skills for cooking. It might even become possible to master the techniques of renowned chefs. We will continue advancing this technology into other fields, so please stay tuned.

Dentsu Inc. International Information Services (ISID)'s Open Innovation Lab ( InnoLab ) is a technology boutique that supports the entire process from "problem discovery" to "social implementation." Collaborating with domestic and international companies, ventures, and universities, it combines technological seeds with societal needs to solve various social challenges. It pioneers the implementation of solutions for future cities and societies worldwide.