1. Microsoft Agents

We can use Microsoft agents to get a feel for how scenarios work. Assume that we assign the following task to the Microsoft agent named Merlin: Let a user who wants to learn more about the traditional Japanese kimono visit the kimono Web site and freely click Web pages. Each time the user visits a new page, the agent summarizes its content. If the user does not react for a while, the agent moves to the next subject.

Fig. 1 shows this Qscenario and its outcome. The action !gesture can perform any of the 60 different gestures that Microsoft agents support. Although computing professionals can work with Qeasily, scenario writers may not be familiar with the Scheme syntax. Further, since Q is a general- purpose scenario description language, it grants too much freedom for describing scenarios for specific domains. We thus introduced interaction pattern cards (IPCs) to capture the interaction patterns in each domain. Fig. 2 shows an IPC equivalent of the Q scenario shown in Fig. 1. Scenario writers can use Excel to fill in the card. The IPC translator then generates a Qscenario according to the card's contents and predefined semantics. Note that IPC is not merely an Excel interface to Q. Rather, it provides a pattern language, and so it should be carefully designed by analyzing the interactions in each domain.

We have used Q and Microsoft agents to develop a multicharacter interface for information retrieval in which domain-specific search agents cooperate to satisfy users' queries.3 Previous research often used blackboard systems to integrate the results from multiple agents. However, given that computing professionals develop search agents independently, attempts to integrate these results are often unsatisfactory.

Thus, we have taken a totally different approach. Instead of integrating the results at the back end, our interface displays the integration process to users as a dialogue involving multiple characters, each of which represents a different search agent. Users can observe the collaboration process among the agents and join the conversation if necessary. This multicharacter interface increases user satisfaction by integrating the results socially at the front end.

2. Crisis Management Simulation

We have started using virtual cities to conduct crisis management simulations involving humans and agents. This pilot application links computer scientists, architects and social psychologists. The roles of agents in this simulation include pedestrians, security guards and so on. Realistic evacuation simulations can be created by having pedestrian agents act as humans running around trying to escape. Such simulations will contribute to discover typical human mistakes and to make correct decisions in real crises.

(a) 2D Simulation

(b) 3D Simulation

Fig.3 Evacuation Simulation (by T. Kawasoe and K. Minami)

Fig.1 shows 2D and 3D simulations of how humans behave when a crisis occurs in a small room. We compared the results obtained from the ongoing simulation to previous findings, and confirmed that multi-agent simulation guided by Q scenarios are sufficiently realistic. Based on this experiment, we are planning to conduct crisis management simulations in subway and railway stations in virtual Kyoto. The simulation will take place with 20 or more people (avatars) connected via the Internet, and hundreds of agents controlled by Q scenarios. The cover story is as follows:

In 200X, an evacuation simulation involving Kyoto subway and railway stations is conducted in 3D virtual Kyoto with hundreds of socially engaged agents. Twenty or more citizens participate via the Internet. The data is collected and analyzed for planning real evacuations.

In 200Y, a disaster occurs and a fire starts in the stations. The situation in the physical space is captured by omni-directional cameras and sent to the control center through wireless networks. The movement of people as a group is grasped and displayed on a central screen. Based on the results of previous evacuation simulations, appropriate directions are sent to the peoples' mobile terminals, and agents start to guide them.

The evacuation simulation brings up the following issues. What can we learn from the simulation results in virtual spaces? Even if the simulation indicates heavy casualties, it is not clear what this means. People may think the virtual space simulation is just like a video game. We thus started scientific research from two different aspects. The first aspect is to understand how humans behave differently in the physical space from the virtual space. People are given the same task in both spaces (for instance, leave via a given exit) and all fixations recorded by eye cameras are analyzed. We are interested in how difficult it is to move in virtual space without peripheral vision, which is inherent in physical space. The second aspect is to understand the difference between human-human and human-agent interaction. We trust software when it is efficient, while we trust humans not because they are efficient. Therefore, human reactions may differ according to whether the directions were given by humans or by software officers, but different in what way? These issues must be analyzed carefully in controlled experiments before conducting large-scale simulations.

3. Social Psychological Study of Agents

To understand the nature of agents, several social psychological experiments have been performed. However, the effects of introducing agents in human communities have not been investigated. We are currently conducting experiments to see how agents can support human communication and influence human relations. Agents can act as go-betweens among people who have different social roles such as inhabitants and visitors, the young and the aged, and so on.

In our experiment on cross-cultural communication between Japanese and American students, the agents influenced not only the impressions of agents but also the impressions of conversation partners and the stereotypes of nationalities. For example, if the agent encouraged students to discuss political problems, the Japanese students became as talkative as the Americans. Fig.2 shows a screen of FreeWalk in these experiments. The dog-shaped agent is supporting human-human conversation. An application designer (social psychologist) needs to repeatedly carry out experiments in various settings in order to study the nature of the socially engaged agent. In these experiments, the user interface for communicating with the agent is very simple. The agent does not use voice?it presents questions to the users in a text-balloon above its head. (We thought text was far less intrusive than audio.) The users indicate `yes' or `no' using the mouse to click on their answer.

The scenario in this case is fairly complex as follows: At first, the agent selects one topic from its repertoire of safe (or unsafe) topics, out of those that have not yet been used. It then randomly chooses one of the two participants as the target for the first question (Fig.4 (1)). Let's call this person A. When A answers (Fig.4 (2)), the agent replies to A's answer (Fig.4 (3)). Based on A's answer, the agent then chooses a follow-up question and turns to B to pose the question (Fig.4 (4)). When B answers, the agent makes a general comment to guide the participants into addressing this topic. This general comment is selected based upon the previous answers from the participants, so that it makes sense given their replies. After making this comment, the agent departs. If at any time a user does not respond to the agent's question, the agent will wait for a while, and then return to idling mode, without trying to continue its line of questioning.

Person A is asked the first question (1), and responds (2), then the agent comments (3). Next person B is asked a question. Note that the agent faces the person it is addressing (4).
Fig.4 Example Scenario in Social Psychological Experiments

In February 2002, we ran a joint class for social psychological experiments between KyotoUniversity and StanfordUniversity using FreeWalk via the Internet. In this class, several groups of Japanese and American students planned and carried out experiments to understand the nature of human behavior in a virtual space. Fig.3 displays how American and Japanese students used FreeWalk for cooperatively designing social psychological experiments. Students decided to work on topics including "the difference between American and Japanese work styles in virtual spaces" and "the difference between American and Japanese impressions of computer characters that look American or Japanese." Q was used to describe the scenarios of students' experiments.

Fig.3 Discussions between Stanford and Kyoto Students (by H. Ito)