Welcome back, in this lecture I'm delighted to have the opportunity to

talk with Ben Shneiderman, one of the pioneers in the field

of human-computer interaction and user interface design.

And we're going to be talking about the ideas of direct manipulation and

of user control.

Ben is a professor at the University of Maryland where he founded the university's

Human-Computer Interaction Lab that has become famous worldwide as a center for

research in human-computer interaction.

He's written a number of books including three of them that I've put up for

you that are particularly notable.

The Designing the User Interface book, Strategies for

Effective Human-Computer Interaction,

which recently came out in its sixth edition and

is one of the leading textbooks in the field on user interface design.

Perhaps my personal favorite though is Leonardo's Laptop: Human Needs and

the New Computing Technologies, which is more a book about how

do we take computing and use it to enable people to be the kind of creative and

world changing intellects that we want in the modern era.

And a recent one,

The New ABC's of Research: Achieving Breakthrough Collaborations, for

those thinking about how do we do serious innovative work in

partnerships this is very much a must read.

So, Ben, welcome to our class.

So let's jump straight in and why don't we take, in your own words,

what the idea behind direct manipulation is in user interface design and

what it's meant to the field.

>> Thank you, I'm very pleased to see how over many years direct manipulation

has become an influential concept in the design of technologies, it's

very much part of the Microsoft and the Apple user interface guidelines documents.

The idea came to me as I was looking at the kind of applications that

triggered enthusiasm from the users, as opposed to the kind of frustrated

acceptance that many technology software products generated.

And there were things like video games of course,

air traffic control, some signs of simulations of scientific experiments.

These all had the kind of fundamental property that it was a visual interface,

that the objects of interest were visually presented on the screen.

And then, the user could actually manipulate them, directly manipulate them.

They would touch them with a touchscreen, or drag with a mouse, and

they could move the objects from one place to another.

They could open valves in a simulation, they could move players in a video game,

they could select aircraft in an air traffic control environment.

And those kind of exciting environments generate a sense of real

enthusiasm among the users rather than reluctant acceptance.

And so I began to look carefully at what it was that generated that kind of

enthusiasm, and I wrote this down and began to formulate these principles.

They were rapid, they were very responsive, and

an arrow on slower technologies meant machines would

take several seconds to respond to UNIX commands and

other command line features, so the rapid reaction was important.

They were incrementally, you could drag a little bit of an icon into a trash can or

drag it back, you could resize a window,

all those incremental changes were very effective.

So rapid, incremental, and reversible, if you did something you could also undo

something, you could reverse the action and that gave the sense of safety and

a willingness to explore and be creative and so people were trying things out.

Naturally, these blossomed out with the applications such as word processors that,

at the time it was a great step forward to call them WYSIWYG,

what you see is what you get.

And putting a cursor in front of a letter and

typing the backspace key would eliminate it,

it was a remarkable change from the command line interfaces of the past.

Similarly, painting programs became popular and

we're talking now the early 80s, and

the beginning arrival of new technologies with graphic user interfaces.

And my contribution was to recognize the common features

across these different applications and to describe them in a way that future

designers could take these ideas and apply them to their own work.

And so the characterization of the psychological aspects,

the perceptual, the cognitive aspects was really the contribution,

and that triggered a great cascade of

innovative ideas from research labs as well as from companies.

And we contributed things like the Home Finder application

that allowed you to see a map of the Washington, D.C. area.

You could move a marker for the place you worked, and another marker for

the place your wife worked, and you could draw a circle around them.

And you could look for the places that were

in a good region that you could commutes to work easily.

And that kind of interface which allowed you to have sliders for

the number of bedrooms and for the cost, turned out to be fun and

effective for those kind of applications.

We built a variety of them and that inspired many others to keep going.

[INAUDIBLE] then the touchscreen ideas that became so

prevalent on mobile devices.

We saw quite early on that that direct manipulation concept would

be very effective if you could touch the screen directly with your finger,

more direct than even a mouse or a light pen.

And so In the late 90s we developed the idea of a touchscreen keyboard.

First we had a 9 inch wide one and then we worked our way down a 3 inch wide one,

so those small keyboards that you have on so many of the smartphones were derived

from that idea which seemed remarkable at the time, of these 3 inch wide keyboards.

Now to get that kind of precision out of a keyboard required some new thinking.

Most keyboards were land on,

when you touched the screen it would activate, our design change was liftoff.

You put your finger on the screen and then you can rapidly, incrementally, and

reversibly drag your finger back and forth and it activated when you lifted off.

So there still are many discoveries and

inventions to be made about improving the notion of direct manipulation.

And to me I see great satisfaction that contemporary interfaces build on these

principles and go even further with mobile devices, solid user interfaces,

tangible user Interfaces, and those ideas continue to grow and spread.

>> Well let's in fact take this into that long context.

I'm sure some of our learners remember, but

many of them I hope are too young to remember that the early computing that

probably both of us did was as far from rapid as you could imagine.

You would put that deck of punch cards in and come back perhaps the next day and

get a response, and it was very much about data processing.

If you were trying to run through a set of data and understand it,

you would program up all of the things you wanted to compute on it, run a batch, and

then get back a printout, which frankly made data exploration rather challenging.

And one of the great innovations that drove the PCs, the Apple 2 in particular,

was this creation of spreadsheets, of VisiCalc, of its competitors.

And the idea that you could very quickly put something there and see what happened,

and if you didn't like it, change it.

I think there was always this other vision, maybe starting in science

fiction but becoming reality, of systems where you wouldn't have to

bother using the computer at all, you would just tell it what to do.

And that vision still exists for some things but

it feels like direct manipulation is in this middle space where

people need to engage with what they're doing because it requires their

thoughtfulness, their creativity, their ability to understand.

And one of those areas that's been remarkably successful in your work and

others has been visual data exploration.

You've developed a system known

as Spotfire and that was specifically about how do we

very rapidly and interactively explore large sets of data.

Could you say a little bit about how that's changed the way that

people go out and think about analytics and visual exploration?

>> Yes [COUGH] yes, we've come a long way Joe, the days of batch processing and

decks of Fortran cards are long past.

And that moved on quickly to command line interfaces

where we could type commands and maybe wait 10 to 15 seconds and

get a response, and that was a big step forward.

But I think the graphic user interfaces that emerged in the 1980s in

personal computers really made the difference you're talking about,

they enable personal exploration of data.

Our work on the direct manipulation, as you correctly point out,

led to the commercial success story of Spotfire.

I was pleased to develop initial designs and

our paper at the Conference in 1994 led to the formation

of the company in 1997, and so that was a big step forward.

Many other visual Interaction companies have come forward,

Tableau certainly is the big one these days, most widely used, terrific product.

And there are lots of others that are growing in that space, and

what that enables is the kind of creative exploration you describe.

It's quite amazing to realize that even with datasets of million of items,

we can have sub-second response time for selecting a region, for

dragging an item, for moving a slider.

And those allow for those creative exploration moments to

do the what if exploration, to see what's happening.

You're right in also pointing out an important step was VisiCalc,

although it was tabular it was numeric, and data, and textual.

It was not a graphic interface, but it had many of the properties of

direct manipulation because you'd click on a cell and

you'd change a value, and boom, the whole spreadsheet would update.

And that was another thrilling step forward in the process of

arriving at the modern interactive data exploration interfaces,

so Spotfire was an important step forward.

And another aspect of my creative process was the idea of tree maps,

and you'd see the whole hard drive on the screen at once.

Rather than typing the UNIX command of change directory, and

list of files, [LAUGH] and move around one step at a time.

To be able to see the entire hard drive was the goal of that project.

And to me there's great satisfaction that there are hundreds of commercial

applications and thousands of websites that use the tree map

idea to show the entire economy, to show all the jobs in the US, show medical care.

And these promote the kind of insights and understanding that I think is very

important in our big data age when we are the beneficiaries of

wonderful amounts of data of many kinds that enable us to

better understand the complex world around us and support.

So the direct manipulation concept influenced me in thinking about data

in many ways.

The Spotfire technology was part of that but

then the tree map ideas was triggered by my desire to see the entire

hard drive of computer on one screen without any scrolling.

And so that led me to the strategy of showing the hierarchy of the hard drive

and that concept of tree map has spread very widely.

There are hundreds of commercial versions, thousands of websites that show

the whole US economy, that show all the jobs, that show healthcare,

that show many social, economic, and politics that enable

people to understand better what's going in the complex world around us.

And it enables them to make better decisions, that's what's really important.

[INAUDIBLE] concepts are not just for big business and

big government, they also work for personal needs.

The quantified-self movement is building on the huge number of

sensors that are available on our smart phones and the way we capture our lives.

And we can better understand the health determinants to enable us to make

better decisions about our lives, that's what really satisfying and important.

>> So I have one more question for you because many of the folks who

are taking these courses are thinking very much about mobile devices,

they're thinking about developing apps.

And in some ways when we think of computing power,

when we think of screen size, processing apps are a big step backwards,

that we have small computers and we have to make them usable.

What's your take on how effective apps have been at employing

direct manipulation principles?

And is there any piece of advice or two you might offer for

people thinking about designing their own apps?

>> In general I think the million plus apps available through Android and

iPhone stores are amazing.

They generally demonstrate good visual design,

the user interface guide encourages this direct manipulation principles.

And standardized colors, and shapes, and positions of buttons

have been pretty effective in getting good things out there.

And you can take a look at recent success stories like Pokemon Go.

What a great app, and what a great story, and what a great direct manipulation

interface to let you move around in the world and throw the balls at the Pokey

characters, and those show the vibrancy of effective design.

Yes, I've seen lots of apps that have poor design where the touchable buttons vary

in size, and shape, and color, and they lead to some confusion.

But by and large those apps are pretty good, now the dilemma is we have so

many apps that we have to keep track of them.

And so I think we're going to see a coalition of app makers and

larger platforms of apps that let you have some consistency so

that all of your healthcare applications might have a common strategy.

And so you could share your data from your blood pressure machine,

from your thermometer, from your scale, from

all those devices that give us the information about how our body is doing.

>> Well wonderful, well thank you so much for joining us with

a message that is not only informative, but uplifting about

the fact that we've really succeeded in this field in making a difference and

have a bright future to a large extent because we've thought about and

now put into practice ways of putting the user in control.

>> Let me add to that, that I'm very proud that our field of human-computer

interaction has really devoted itself to the notions of societal transformation.

We've had powerful effects and

increasingly we understand that our role, for example in the United Nations

sustainability development goals can be very influential.

We can make the world a better place, we have made the world a better place, and

we will continue to do so.

>> Well, that's a wonderful way to end.

This has been an interview with Ben Shneiderman on direct manipulation and

user control.

We'll see you next time.

Direct Manipulation & User Control (Interview with Ben Shneiderman)

Prototyping and Design

与讲师见面