People of ACM - Abigail Sellen

February 7, 2017

What is one area of computing, in particular, where you think we could do a better job of incorporating a more human-centered approach?

Most areas of computing could do with a more human-centered approach, but I would single out AI systems. The result of recent advancements in machine learning, such as deep learning and model-based approaches, means that the systems we are building are becoming more complex. While this may be great in terms of what these systems can achieve technically, from a human perspective it can mean that they are ever more difficult for users to understand. AI systems are increasingly acting on our behalf and taking decisions for us. To do this, they make use of data which is largely invisible to us, and algorithms which are just as opaque.

All of this is increasingly problematic. When we don’t understand the systems we interact with, not only are they more difficult to use, but we stop trusting them. So, in addition to making our systems more intelligent, we have to make them more intelligible too. This is a matter of good, human-centered design, and working closely with the technical teams who build them. More than this, I am a firm believer that we need to build AI systems to augment human capabilities rather than to compete with them. If we are to build these partnerships, playing to the strengths of both people and machines, a human-centered approach is more important than ever.

Traditionally, humans have interacted with computer devices via windows, icons, menus, and point-and-click (WIMP) actions. Recently, new forms of interaction, including gesturing and speech recognition, are becoming more prevalent. Within these new forms of interaction, is there a particular area of recent research or innovation that you are especially excited about?

There are many places where new forms of interaction will enrich our capabilities, but we have to think carefully about where they are appropriate. For example, speech interfaces make a lot of sense while driving, but not when you’re sitting on a crowded train. While WIMP interfaces are both effective and familiar, speech input can make sense when you need to formulate a complex query, refer to things in the digital world that you can’t see or point to, or perform searches over vast quantities of data.

Gesture, too, has its place, and when used appropriately can transform our interaction with technology. We did some work with vascular surgeons who, when scrubbed up and in the operating theatre, have to avoid contact with non-sterile computers. We developed a computer vision system that allowed them to use in-air gestures to navigate through data without the need for touch. What’s most exciting now are the new possibilities that open up when you combine input modalities. Speech and gesture often go together quite naturally in our everyday interactions, such as pointing at something and talking about it. The idea of combining speech and gesture goes back decades now, but these input techniques have yet to become mainstream. When they do, I have no doubt that when designed carefully, they will allow us to be much more expressive in our interactions.

In The Myth of the Paperless Office, you noted that, contrary to popular assumptions, the use of email in an organization actually increases paper consumption by 40%. In the book, you introduced the idea of affordances, that is, the activities that an object such as paper allows or “affords.” You argued that we should abandon the unrealistic ideal of a paperless office and focus on how we might design our electronic devices with more affordances. Will you expand on this?

In that book, my co-author Richard Harper and I weren’t so much arguing that electronic devices need “more” affordances so much as that all technologies, whether they be “low-tech” (such as paper) or “high-tech” can be understood in terms of how their properties support human activity. This means we can look at how our technology shapes interaction from the details of the low-level operations we carry out, to the workflows they enable, to the social and cultural aspects of work that they engender. Paper can be understood this way: it supports rich, free-form markup, laying information out in space and flexible navigation. This is one reason people often print out documents they need to understand deeply. Physically getting to grips with paper often helps one mentally “get to grips” with its contents.

Paper also shapes our social processes, such as laying a document on someone’s chair to say “read this” in a way that not only that person can see, but everyone else in the same workspace can see. In highly collaborative situations too, using paper and pen is more visible to one’s colleagues than, say, using a laptop, which can make the nature of ongoing activities more obscure to others. Peripheral awareness of the activities of others is important in how we coordinate with each other. The argument here isn’t that paper is good and computer technology is bad, or vice versa. It is rather that we can learn from how paper is used how to design digital technologies better. In the book, we don’t argue that the idea of the paperless office is bad either, rather, that getting rid of paper for the sake of it doesn’t always lead to success.

Your background is in cognitive psychology, and you have worked with several interdisciplinary teams. What advice would you give students in the social sciences or liberal arts about the benefits of getting more involved in the computing field?

Many of the most important breakthroughs in computing in the years to come will be at the intersection of computing and completely different disciplines. Whether you come from a different scientific background, such as biology or physics, or from the arts and humanities, the bringing together of different perspectives drives transformational change in computing. A case in point is machine learning, which now underpins most of our work in AI but which has its roots in cognitive psychology and models of human memory. Working in interdisciplinary teams is hard work—it takes time just to reach a common language sometimes—but the payoffs can be enormous.

My advice to students, whether they are studying psychology, philosophy, anthropology, fashion, architecture, medicine, or virtually any field, is to not rule out a career in computing. We need all kinds of expertise, and to embrace all kinds of disciplines, because computing permeates every aspect of our lives.