Digital Life

How Can Robots Get Our Attention?

Atlanta, GA (March 8, 2011) — Getting someone’s attention can be easy with a loud noise or a shout, but what if the situation calls for a little more tact? How can a robot use subtle cues to attract a human’s notice and tell when it has captured it? In a preliminary study, researchers at the Georgia Institute of Technology have found that they can program a robot to understand when it gains a human’s attention and when it falls short. The research is being presented today at the Human-Robot Interaction conference in Lausanne, Switzerland.

“The primary focus was trying to give Simon, our robot, the ability to understand when a human being seems to be reacting appropriately, or in some sense is interested now in a response with respect to Simon and to be able to do it using a visual medium, a camera,” said Aaron Bobick, professor and chair of the School of Interactive Computing in Georgia Tech’s College of Computing.

Using the socially expressive robot Simon, from Assistant Professor Andrea Thomaz’s Socially Intelligent Machines lab, researchers wanted to see if they could tell when he had successfully attracted the attention of a human who was busily engaged in a task and when he had not. 

“Simon would make some form of a gesture, or some form of an action when the user was present, and the computer vision task was to try to determine whether or not you had captured the attention of the human being,” said Bobick.

With close to 80 percent accuracy Simon was able to tell, using only his cameras as a guide, whether someone was paying attention to him or ignoring him.

“We would like to bring robots into the human world. That means they have to engage with human beings, and human beings have an expectation of being engaged in a way similar to the way other human beings would engage with them,” said Bobick.

“Other human beings understand turn-taking. They understand that if I make some indication, they’ll turn and face someone when they want to engage with them and they won’t when they don’t want to engage with them. In order for these robots to work with us effectively, they have to obey these same kinds of social conventions, which means they have to perceive the same thing humans perceive in determining how to abide by those conventions,” he added.

Researchers plan to go further with their investigations into how Simon can read communication cues by studying whether he can tell by a person’s gaze whether they are paying attention or using elements of language or other actions.

“Previously people would have pre-defined notions of what the user should do in a particular context and they would look for those,” said Bobick. “That only works when the person behaves exactly as expected. Our approach, which I think is the most novel element, is to use the user’s current behavior as the baseline and observe what changes.”

The research team for this study consisted of Bobick, Thomaz, doctoral student Jinhan Lee and undergraduate student Jeffrey Kiser.

For more information contact:

David Terraso

Communications and Marketing

404-385-2966

Photos

Click on an image below to see the full photo

  • How can robots get our attention? Simon photo

Media

Click on an image to launch a media file.

  • How Can Robots Get Our Attention? VideoVideo Clip

Faculty

  • Amy Bruckman

    Amy Bruckman

    Associate Professor
    School of Interactive Computing, College of Computing

    Areas of Expertise:
    Educational Technology, Social Networking/Online Communities, Wikipedia, Twitter, Facebook, Internet Research Ethics, Human Computer Interaction, Human Computer Interaction for Kids

  • Carl DiSalvo

    Carl DiSalvo

    Assistant Professor
    School of Literature, Communication and Culture, Ivan Allen College of Liberal Arts

    Areas of Expertise:
    Participatory Design, Critical Design, Design Studies, Robotics and Sensing in Art and Community Settings

  • Keith Edwards

    Keith Edwards

    Associate Professor
    School of Interactive Computing, College of Computing

    Areas of Expertise:
    Social Impacts of Technology, Home Network Security, Home Networking, Human-Computer Interaction

  • Irfan Essa

    Irfan Essa

    Professor
    School of Interactive Computing, College of Computing
    School of Electrical and Computer Engineering, College of Engineering

    Areas of Expertise:
    Computational Video, Computational Photography, Computational Journalism, Computational Media, Computational Perception

  • Beki Grinter

    Beki Grinter

    Associate Professor
    School of Interactive Computing, College of Computing

    Areas of Expertise:
    Societal Impacts of Technology, Human-Computer Interaction, Computer Supported Cooperative Work

  • Renu Kulkarni

    Renu Kulkarni

    Executive Director, FutureMedia

    Areas of Expertise:
    Convergence of digital, social, mobile and multimedia industries, Strategic Alliances, Industry Partnerships, Open Innovation Practices

  • Blair MacIntyre

    Blair MacIntyre

    Associate Professor
    School of Interactive Computing, College of Computing
    School of Literature Communication and Culture, Ivan Allen College of Liberal Arts

    Areas of Expertise:
    Augmented Reality, Virtual Reality, Mobile Games, Social Games, Augmented Reality Games, Video Game Design, Video Game Architecture

  • Ali Mazalek

    Ali Mazalek

    Assistant Professor
    School of Literature, Communication and Culture, Ivan Allen College of Liberal Arts

    Areas of Expertise:
    Tangible Interfaces, Experimental Media, Media Arts, Interaction Design, Emerging Technologies

  • Janet Murray

    Janet H. Murray

    Ivan Allen College Dean's Professor
    School of Literature, Communication and Culture, Ivan Allen College of Liberal Arts

    Areas of Expertise:
    Game Design, Interactive Narrative, Interactive Television, Media Convergence, Information Design, Digital Media and Education

  • Elizabeth Mynatt

    Elizabeth Mynatt

    Director, GVU Center
    Professor, School of Interactive Computing
    Associate Dean for Strategic Planning and Initiatives
    College of Computing

    Areas of Expertise:
    Human-Computer Interaction, Human-Centered Computing, Health Informatics, Ubiquitous Computing, Assistive Technologies

  • Ashwin Ram

    Ashwin Ram

    Associate Professor
    School of Interactive Computing, College of Computing

    Areas of Expertise:
    Artificial Intelligence (AI) (Case-Based Reasoning, Natural Language, & Game/Entertainment AI), Human-Centered Computing - Cognitive Science, Healthcare Informatics

  • Bruce Walker

    Bruce Walker

    Associate Professor
    School of Psychology, College of Sciences School of Interactive Computing, College of Computing

    Areas of Expertise:
    Interactive Music, Mobile Music, Human-Computer Interaction, Auditory Perception, Psychology