Detecting Signs of Social Issues through AI and Big Data Analysis

Public Affairs (PA) refers to the activities through which companies and organizations engage in dialogue with stakeholders to understand their surrounding environment and create a better environment. This series will introduce the background behind the growing demand for Public Affairs in future companies, the elements required for influencing public opinion and policy, and the "Innovation PA Model" developed by the Corporate Public Relations Strategy Institute.

In Part 1, Ken Nagahama of the Corporate Public Relations Strategy Institute explains the social innovation issues extracted from social media and the awareness and information needs of Diet members regarding these issues. This analysis is based on the results of a joint study by Dentsu Inc. Public Relations, the Sakata-Mori Laboratory at the University of Tokyo Graduate School, and Hotlink.

Public Affairs Activities Required for Corporate Innovation

Today, new markets for products and services like AI, drones, and autonomous driving continue to emerge. To turn these areas into new revenue streams, companies must engage in efforts to shape their own business environment, including influencing regulatory frameworks.

Engaging in rule-making, including legal frameworks, requires efforts distinct from conventional marketing activities. Public affairs (PA) activities are needed to target diverse stakeholders—including national and local legislators, domestic and international regulators (such as bureaucrats), and civic groups—to gain their understanding.

It is particularly crucial to clearly explain not only the advantages of one's own products and services but also how shaping new rules can solve specific societal challenges.

The targets for rule formation encompass both hard law (statutes) and soft law (guidelines, technical standards, etc., related to the interpretation of statutes). However, in either case, a dual approach is required: direct engagement with policymakers and communication to gain public understanding.

The Ministry of Economy, Trade and Industry (METI) has established a "Rule Formation Strategy Office" within its Trade Policy Bureau, sending the message that "in today's world where rules change in response to global social challenges, a management perspective is necessary" (*1).

※1: March 2014, Trade Policy Bureau , Ministry of Economy, Trade and Industry, "Toward Rule Formation as Corporate Strategy," Page 5

Estimating Social Innovation Challenges through AI (Artificial Intelligence) × Big Data Analysis

This research defines "social innovation challenges" as "issues whose resolution would significantly transform and improve society, the economy, and daily life." It conducted AI analysis targeting social challenges across broader domains.

Two AI systems were developed: the "Social Innovation Issue Estimation AI" and the "Community Estimation AI." First, keywords related to social issues were extracted from the government's growth strategy "Future Investment Strategy 2017" (*2). The "Social Innovation Issue Estimation AI" was then trained on Twitter data containing these keywords. It was instructed to identify tweets likely related to social issues from one month of Twitter data in July 2017. Furthermore, we used the "Community Estimation AI" to clarify the communities within which these tweets were being posted (Figure 1).

The 18 social issues extracted from the AI analysis were classified into four groups based on two criteria on Twitter: "① High volume of tweets" and "② Concentration of tweets within specific communities (low topic entropy<※3>)" (Figure 2).

For example, social issues with high tweet volume and broad distribution (HH) include "Improving childcare environments and resolving the waiting list problem." Tweets about "daycare centers" overflowed with the concerns of working women, such as difficulty getting in, high fees, and anxiety about finding jobs without daycare options. Tweets about "children on waiting lists" highlighted issues like differing standards for measuring waiting list numbers across municipalities. This reveals that many voices from citizens demanding solutions exist online for already prominent social issues like childcare facilities and waiting lists.

Social issues (LH) with fewer tweets but wider distribution included "improving QOL/well-being." Examining these tweets reveals discussions about the QOL (Quality of Life) of elderly people receiving care, alongside tweets concerning the QOL of doctors and nurses in clinical settings, single mothers, men struggling to find their place in society, hairdressers, and even dogs. While not major topics, this shows a variety of social issues exist.

※2: Future Investment Strategy 2017: A growth strategy aiming to realize "Society 5.0" by incorporating innovations from the Fourth Industrial Revolution (IoT, big data, artificial intelligence <AI>, robotics, sharing economy) into all industries and social life to solve various social issues. Approved by the Cabinet on June 9, 2017.

※3: Topic Entropy: Topic entropy measures the diversity of communities to which users spreading information within a topic belong, based on information quantity concepts. Low topic entropy indicates the topic was spread by users from only a few communities, meaning even if the number of shares is high, it likely reached only a limited user base. Conversely, high topic entropy indicates the topic was widely disseminated to members of diverse communities.

Issues with few tweets but wide distribution are less likely to be recognized by Diet members

Regarding awareness of these 18 social innovation issues, we sent questionnaires to approximately 700 national Diet members from both the House of Representatives and the House of Councillors. We then calculated the average score for each issue based on the responses received from 64 Diet members. This revealed which characteristics of social issues make them more likely to be recognized by Diet members.

So, what kind of social issues are most likely to be recognized by Diet members? Analysis of the survey results showed that social issues discussed within specific communities tended to have higher recognition among Diet members. Conversely, issues that are broadly distributed across society and have fewer tweets tended to be less recognized by Diet members (Figure 3) (*4).

*4: For Figure 3, we compared the average scores across the four categories (each MP × each issue) to the overall average score (82.8 points) across all MPs nationwide × all 18 issues. We confirmed statistically significant differences (at the 5% level) using a z-test.

Furthermore, Diet members were asked whether they wanted to obtain information useful for solving each social innovation issue.

※5: As with Figure 3, Figure 4 also compared the scores to the overall average of 81.5 points and confirmed statistically significant differences (at the 5% level) using z-tests.

The results showed that potential social innovation issues with wide dispersion and few tweets (LH in the figure above) not only struggle to reach lawmakers but also have low perceived need for information necessary to solve them (Figure 4).

To achieve rule-making that fosters innovation, stakeholders like Diet members must recognize what social issues corporate products and services solve. As a prerequisite, it is crucial they first understand what social issues exist.

However, this survey found that widely dispersed social issues with few tweets are less likely to be recognized by Diet members. The voices of potential users who could benefit from products and services created through innovation may be widely dispersed, raising concerns that they may not reach policymakers.

Mechanisms for extracting emerging social innovation challenges

MPs' awareness of social issues is influenced by sources other than Twitter and the nature of the issues themselves, so further analysis is needed to establish a causal relationship with their distribution on Twitter. However, at the very least, the tendency for MPs to be less aware of social issues where those in need are dispersed is a point worthy of attention when planning future solutions.

When asked about the necessity of using AI to extract social innovation issues, approximately 90% of the Diet members who responded to this survey answered "necessary" (Figure 5). While it is possible that the respondents included many Diet members already interested in innovation, this indicates the existence of a certain number of Diet members interested in creating new mechanisms using AI.

To solve social issues through innovation, it is essential to clarify the challenges existing in society and provide information to Diet members driving solutions, thereby promoting accurate understanding and discussion of these issues.

These survey results indicate the need for a new mechanism (Figure 6) to extract emerging social innovation challenges from social media platforms like Twitter and deliver them to policymakers, including Diet members. Furthermore, activities by companies and other entities to communicate these extracted social challenges to policymakers are also considered important.

Next time, based on the survey results, we will clarify what kind of information should be disseminated from which sources regarding social issues to make them more easily recognized by policy makers (members of parliament).

[Survey Overview]

• Research Entities: Sakata-Mori Laboratory, Department of Technology Management and Strategy, Graduate School of Engineering, The University of Tokyo; Corporate Communications Strategy Institute (within Dentsu Inc. Public Relations); Hotlink

① AI for Estimating Social Innovation Issues and Big Data Analysis

• Survey Overview: Based on "Future Investment Strategy 2017" and Twitter data, two AI systems were developed: "AI for Estimating Social Innovation Issues" and "AI for Estimating Communities on Social Media." Furthermore, 18 issues were classified into 4 groups based on "① Volume of Tweets" and "② Concentration of Tweets within Specific Communities" on Twitter.

• Analysis Target: Data from over tens of millions of randomly sampled tweets from July 2017

② Diet Member Survey

• Survey Method: With the cooperation of the Diet Members' League for the Promotion of Digital Society, questionnaires were sent to all Diet members (both House of Representatives and House of Councillors). Responses were obtained via the questionnaire or online.

• Survey Period: December 9, 2017 – April 8, 2018

• Responses: Total of 64 (43 House of Representatives members, 21 House of Councillors members)