AI and Creativity - The Library of Combinations and AI

Once upon a time, back when Latin was still widely used, there was a colossal library. It was truly enormous, said to contain not only every book ever written, but also every book that might ever be written. In other words, it was a library encompassing every possible combination of the Latin alphabet.

If every possible combination of books existed here, then somewhere within this vast library, there should be "Harry Potter and the Philosopher's Stone," the Bible, and even this very manuscript itself (if it had been written in Latin). However, most books would likely look something like this:

"aatlewpukwhzep,uvtpqstymfhmpfbrteleaiswxuxxkouufdcg.opidjwn, prgjigdo,yldyhptnmrvvcejsa.xxchmxoihesl xsdckue bqsrbe.."

The setting described so far is based on the short story "The Library of Babel" by Jorge Luis Borges, one of Argentina's most renowned writers. The protagonist librarian in this story is born within the library and is destined to spend his entire life wandering its halls, searching for a meaningful book.

Created by modifying " Trinity College Library - Dublin " © Jorge Láscar (Licensed under CC BY 2.0 )　

 If you were this librarian, how would you search for books? What are the odds of randomly walking through the library and stumbling upon an interesting book? You can imagine this by calculating the library's collection size.

Adopting the setting of "The Library of Babel," assuming each book consists of about 400 pages, uses the 25-letter Latin alphabet, has 80 lines per page, and 40 characters per line, the number of books is calculated as: . Since the total number of atoms in the universe is
, the number of books far exceeds this.

If this librarian were to discover an unpublished new Harry Potter novel or an interesting science fiction novel that no one has ever read within this library, he or she would undoubtedly be incredibly creative.

To create something new with AI

Yes, "creativity" is often thought of as creating something from nothing, but it can be approached as a problem of exploring combinations. By combining known finite elements (the alphabet), an infinite yet finite number of variations (books) can be created. Not just text, but music too, through sequences of notes and combinations of instruments; and graphics or photos, by exhaustively covering combinations of pixel colors, can (approximately) cover every possible expression.

As the social implementation of artificial intelligence (AI) advances today, the potential for applying AI to creative domains is gaining attention. Creating a creative AI is essentially no different than creating a librarian for this "library of combinations." The goal is simply to select the best from the possible combinations. Kevin Kelly, founding editor of Wired magazine, quotes evolutionary biologist Richard Dawkins—known for works like The Selfish Gene—in Out of Control: The New Biology of Machines, Social Systems and The Economic World (p. 239):

Richard Dawkins echoes this when he asserts that "effective searching procedures become, when the search-space is sufficiently large, indistinguishable from true creativity."

In other words, "When the search space is sufficiently large, effective searching procedures become indistinguishable from true creativity."

The key point here is the "evaluation" part. Even if a mechanism for rapidly scanning books were developed, how would AI judge the quality or interest of those books?

For most AI, this evaluation relies on similarity to patterns found in human-written texts from the past. If it detects "pat," the next character is more likely to be "n" rather than "m" or "g"; if it detects "written by humans," the next words are likely to be "sentence," "book," or "novel." Thus, by gathering past masterpieces of writing and music as training data and having the AI learn the patterns within them, it becomes possible (in principle) to generate writing reminiscent of Natsume Sōseki or music reminiscent of Bach. While it might be able to produce short sentences, generating long texts with a narrative structure and story arc, like a novel, is an area where AI struggles.

But what if we take it a step further and want to write a novel in a completely new style or plot that no one has ever read, or create music that no one has ever heard? Can we evaluate something that doesn't yet exist in the world? Several years ago, the computer Go program "AlphaGo" caused a major sensation by defeating the world's top Go players with unconventional moves that defied conventional wisdom. However, unlike Go, where the rules and criteria for victory are clear-cut, the world of expression lacks straightforward rules for determining quality.

Therefore, efforts are beginning to shift away from directly evaluating the quality of novels or music. Instead, they replace this task with another—one where AI excels—to ultimately create something new that isn't merely imitating humans.

For example, New Zealand artist and researcher Tom White applies this approach to generating paintings (I'm also looking forward to his talk at Creative Aliens, an event on AI and creativity happening October 10th and 11th). He reframes the evaluation of paintings as an image recognition problem, something AI excels at.

He randomly places circles and lines on a canvas, then feeds it through image recognition systems to measure how well it is recognized as a specific object (like a fan). He then updates the positions and sizes of elements to make the painting look more like a fan. Repeating this process produces abstract paintings that seem to compress the essence of our image of a fan (top left in the figure below). It's also interesting to consider what the other paintings might represent. But what's most remarkable is that the AI generates paintings with a unique style, not imitating any past artist's work.

Tom White "Perception Engines"
Paintings generated using image recognition mechanisms. Starting from the top left: the result of gradually altering the image to cause the AI to misidentify it as a fan, basketball, iron, ski mask, forklift, ruler, sewing machine, and bra.

The same principle applies to the AI DJ Project we've been developing at Qosmo, where I serve as CEO. Rather than learning from a human DJ's past playlists, the AI DJ selects tracks using a model that calculates the similarity of a song's mood based on a model that estimates music genres and the types of instruments used. This allows it to fulfill the DJ's fundamental role of maintaining the flow of music while enabling a free-thinking approach to track selection, unbound by preconceptions.

 

AI DJ Project - Photo by Yasuhiro Tani
Courtesy of Yamaguchi Center for Arts and Media [YCAM]
Performance of the AI DJ Project at YCAM (2017)

 AI DJ Project at Google I/O  2019 Keynote Preshow - Courtesy of Google, Inc.