Understanding the depths of feeling and culture
from communication through computers
What ways have been used to gain an understanding of different cultures? The most orthodox way was to read books about foreign lands. One could also watch movies, listen to music, or experience different cultures directly through overseas study and travel. And so on. Today, however, the world is always interconnected by the internet. Would it be possible to gain an understanding of the deeper levels of cultures using the powerful technology of computers? This is the concept of Cultural Computing being explored by Professor Naoko Tosa. The phrase “Cultural Computing” is itself a new expression coined by Professor Tosa.
The use of computers for purposes other than as machines for computational processing and data processing. This can be a means for understanding the hidden nature of human emotions and feelings, and for understanding cultural aspects unique to individual races. Professor Tosa is pioneering this newly opening field.
Professor Tosa’s career began with “video art.” As a consequence the research content is outstandingly visual, and worthy of being called “artwork.” An initial work, “Neuro Baby,” was inspired by the neural network model of the human brain. An adorable baby appeared on the computer display. The baby reacted in response to an uttered expression. For instance, the word “cute” might elicit delight, sadness, displeasure or other emotion, depending on the tone and intonation of the speaker’s voice. The reaction was in response not to the meaning of the word, but to the tone and nuance. “Cute” uttered with malice elicited sadness, whereas a heartfelt “cute” resulted in pleasure. Probing for the feelings hidden behind words and putting them into visual form is art from a completely new perspective. It is thought this technology can eventually be applied to robotic speech, to enable conversation with expression of feeling.
The human brain is a collection of nerve cells called neurons. Each neuron has a dendrite (input terminal) and an axon (output terminal). Together they form a complex network structure for mutual exchange of information. The neural network is an approach for reproducing this brain mechanism with computers for use in problem-solving.
Computing the “Unconscious”
Moving deeper into research, Professor Tosa’s interest turned toward seizing the “unconscious.” The outcome was Unconscious Flow. For example, a couple (man and woman) facing each other each have a tiny device for measuring heart rate attached to the tip of a finger. One holds a red pallet in the hand; the other holds a yellow pallet. A camera constantly measures the distance between the two pallets. The screen shows a mermaid and a merman, representing the woman and the man. As the hands of the people move, the heart rates and the distance between the two pallets change. The screen merman and mermaid move in response. When there is a high degree of empathy between the people, the mermaid and merman draw closer together and begin to dance. When the degree of empathy is low, they move apart. In the extreme case, the mermaid and merman might begin to fight. No words are exchanged between the people. One person simply moves the hand in response to the other person. However, the invisible and unconscious world hidden behind the movement is put into visual form through the movements of the mermaid and merman. Although the merman and mermaid are interested in each other, embarrassment causes them to feign indifference. Visualization reflects the subtle ‘weave’ of the heart.
Computing the “Zen World”
From the application of “feelings” and “unconsciousness” as information not previously considered in the world of computers, research developed into exploration of the depths of racial culture. This was an ambitious attempt to explore the world of Zen using computers. The trigger for such research was a retrospective exhibition on the work of Sesshu, held in Kyoto in April 2002. Professor Tosa was strongly attracted by the landscape paintings, and began thinking about whether the Zen world perspective could be comprehended using computers. The outcome was a program for creating computer-generated Zen landscapes, titled ZENetic Computer. Research was conducted jointly with Seigo Matsuoka, an editorial engineer. The display screen shows the Chinese characters for mountain, people, house and river. Clicking on ‘mountain’ results in appearance of a number of black and white illustrations of mountains. The user selects one or more appealing mountains and drags them onto a place of choosing on the screen. ‘House’ is next. An original landscape can thus be created. The subsequent flow is as follows. (1) The elements of the landscape are selected and the picture is complete. (2) A haiku appropriate for the landscape icon can be created. (3) When approaching the features of the landscape picture, related Zen Question & Answer dialogues appear. (4) Depending on the interaction with the Zen dialogues, a kimono design style matching the personality of the user appears. The design style is drawn from kimono designs by Miyazaki Yuzensai (17th century). Finally, the Jugyuzu (Ten Cow Pictures) appears. A series of ten drawings of about a cow, Jugyuzu is a Zen technique for approaching enlightenment.
Since presentation to the public, its popularity has continued to grow. In exhibitions overseas, children as well as adults derive much enjoyment from creating landscape pictures. Even Zen priests enjoy the process.
Makoto Nagao, former President of Kyoto University and now Librarian of the National Diet Library, had high praise for Professor Tosa’s research: “The starting point was issues of art and technology. This progressed to issues of culture and technology. Serious study is now focused on the interaction of artists and appreciators of art in a world formed from fusion of culture and technology.”
The “artwork” of Professor Tosa is outstandingly visual. As well as the research outlined here, Professor Tosa has other fascinating creations, such as “Interactive Manzai (Comic Dialogue)” whereby users can create manzai through dialogue with a “Repartee Computer.” An encounter with these artworks on Professor Tosa’s website is highly recommended.
Jugyuzu (Ten Cow Pictures)
A method for guiding people toward enlightenment. The cow represents “mind.” It begins with a picture of a boy searching for a cow, and progresses through a series of nine levels. The 10th level depicts the return to society and the spreading of enlightenment by mingling with people.
(Shigeru Tomono December 4, 2009)