What can the modular AI service "KEY RING" do? The possibilities expanded by AI-powered image processing

Data Artist Inc. focuses on providing digital solutions and solving marketing challenges, primarily through AI. The company has developed "KEY RING," a service that combines AI modules to build solutions. Its key feature is the diverse range of applications enabled by combining AI modules, allowing even non-AI experts to explore AI-powered initiatives.

In our previous article covering "KEY RING, " Ken Yamada of Data Artist introduced the service's development background, overview, and key features. In this second installment, Hatsumi Suzuki of Data Artist presents case studies primarily utilizing "image processing modules."

While AI is now used across many fields, many people likely feel, "I'm interested, but I don't know how to use it" or "It seems difficult to apply it to my company's business." If that sounds like you, we encourage you to read this article and consider the potential for AI adoption within your own company.

"KEY RING" includes five "image processing modules"

"KEY RING" consists of 18 modules in total. Among these, five are categorized as "image processing modules." Fundamentally, these modules enable AI to analyze objects captured in video footage. The specific analyses possible include the following five:

① Facial Expression/Emotion Recognition Module: Analyzes a person's emotions through facial expression analysis

② Gender and Age Estimation Module: Estimates the subject's gender and age group

③ Object Detection Module: Determines what the captured object is

④ Motion Tracking Module: Tracks the movement of objects

⑤ Image Region Segmentation Module: Breaks down the image pixel-by-pixel to identify what is present in each area

Here, we present four examples of how we actually applied these modules to develop solutions.

Four Examples Utilizing the "Image Processing Module"

Case 1: Marathon Event "Face Pass" Participation (Combining the "Facial Expression/Emotion Recognition Module" and "Object Detection Module")

Previously, we had installed cameras at event venues to track the faces of people entering. Without identifying individuals, we analyzed audience emotions (whether they were enjoying themselves or when feelings of enjoyment peaked) and where they tended to linger within the venue. Building on this experience, we launched the challenge: "Let's make face recognition entry a reality."

For a citizen marathon in a certain city, we created an app for participants and had them register their photos in advance. We then conducted a proof-of-concept experiment to see if they could enter solely via facial recognition on the day of the event.Participants registered their name, entry number, and photo within the app beforehand. On the day of the event, tablet terminals were set up at the venue reception. Simply showing their face to the terminal completed their entry. When implemented, the authentication accuracy was 100%. The verification time was also very smooth, taking only about 2 seconds. We believe this type of application has the potential to expand significantly in the future.

Of course, there were some challenges. For example, poor communication conditions could prevent recognition or cause delays. If the entry point is underground, communication might not be smooth. However, if this system operates reliably, it could reduce the need for registration staff, alleviate congestion, aid infection control, and most importantly, reduce participant stress from having to wait long lines for registration on the event day.

Case 2: Sushi Topping Identification ("Object Detection Module")

This project originated from a request by a conveyor belt sushi restaurant seeking to verify whether sushi toppings were being served correctly according to customer orders. We conducted a proof-of-concept experiment utilizing our object detection module.

Specifically, the challenge was whether AI could accurately distinguish 13 types of sushi toppings: horse mackerel, sardine, young yellowtail, lean tuna, medium fatty tuna, fatty tuna, yellowtail, salmon, seared salmon, raw shrimp, boiled shrimp, salmon roe, and sea urchin.When we actually tested it, ikura and uni were easily distinguishable. However, the shiny fish like horse mackerel, sardine, and gizzard shad are visually similar and difficult to identify. Even so, we achieved an accuracy rate of over 80% for these three shiny fish types.

You might wonder, "Is this experiment really meaningful?" (laughs), but the key point is that developing this approach opens up the possibility of "using AI for error prevention and quality control." By incorporating AI image analysis, it might be possible to check and reject items, for example, to see if they match the order or if the quality is poor. If that happens, a single camera could handle everything from error prevention to quality control, potentially expanding the scenarios where it can be used.

Case 3: AI ELEVEN (Combination of "Motion Tracking Module" and "Image Region Segmentation Module")

AI ELEVEN is a unique AI that predicts soccer match outcomes in real time. It detects player and ball movements from live match footage and analyzes them using multiple AI systems. Viewers can instantly grasp the state of play. It has already undergone field tests at events like the EAFF E-1 Football Championship, an international tournament hosted by the East Asian Football Federation (EAFF), and in South Korea's K League.

This system uses image analysis to instantly capture everything during a match: ball possession (the percentage of time a team holds the ball), player movements, which areas of the field play occurs, how much each team is attacking, and what events happen how many minutes after kickoff. The AI then instantly determines the predicted outcome. This allows viewers not only to enjoy the soccer match but also to experience a new way of watching, where the predicted outcome evolves in real-time alongside the match's progression.

While methods like sports betting exist for enjoying "soccer outcome predictions," these typically involve predicting the winner beforehand based on player condition, past head-to-head records, and recent match results. AI ELEVEN, on the other hand, "instantly predicts the outcome based on the situation unfolding right before your eyes during the match." This represents a new way to enjoy the game, unique to AI.

Case 4: Hair Color Simulator ("Image Region Decomposition Module")

This project, developed in collaboration with a hair care and toiletries company, created an AI-powered "Hair Color Simulator."

Users take a photo of their face with their smartphone and can freely change their hair color on the screen. Many people have likely experienced trying a hair color only to find the result didn't match their vision. This simulator allows users to preview how their appearance will change with a particular hair color beforehand, reducing that gap.

Furthermore, this system can simulate not only hair color but also gray hair coverage. While "taking a photo with your smartphone and changing hair color on screen" might sound simple, hair varies greatly in shape from person to person and is very fine, making it extremely technically challenging to accurately capture its contours.With gray hair, the boundaries between hair and scalp can be difficult to discern. If the simulator's accuracy is low, the result might look like an unnatural color is simply placed on top of the head, making it hard to visualize the actual effect. Accurately identifying hair areas and applying color to them requires highly advanced AI technology.

This time, we introduced four use cases utilizing our "image processing module." Applying these examples could enable various scenarios, such as "enhancing security with facial recognition access control," "analyzing customer movements to optimize store displays," or "providing remote customer service combined with simulators." We believe there are elements here that can contribute to the business growth of everyone reading this article.

We, too, continue to challenge ourselves and evolve daily. While real-world events have decreased due to the pandemic, we have developed systems like one that evaluates event attendees based on their expressions and behavior, rather than surveys. Moving forward, we will continue to develop and provide solutions that address various challenges, leveraging "KEY RING."

We've seen how AI can deliver various benefits, such as easing congestion, improving quality control, and providing new forms of entertainment. While this article focused on specific examples using image processing modules, "KEY RING" comprises a total of 18 AI modules. In the Transformation SHOWCASE, we will continue to introduce AI solution cases built using this "KEY RING." We encourage you to read those articles as well.