Where's the magic of human-computer interaction? It lies beyond windows, scroll bars, menus, double-click, drag-and-drop, keyboard shortcuts, context-sensitive help, dialog boxes, or any of the other fancy features that cover our screens. These are all mere shadows.

The wellspring of power is direct manipulation — the metaphor that we're working with physical objects, real things themselves, when we're on a computer. Of course, we're not. Our actions are translated by the user interface into low-level commands: fetch those bytes, put them over there, set that flag, multiply these numbers. Thousands of atomic operations add up to files copied, displays refreshed, words moved, and sounds generated — mid-level activities. Thousands of those, in turn, give us e-mail, web pages, music, and animation. So we press buttons, grab documents, drop them in the trash, copy/paste photographs, and do all the other top-level actions that serve our needs as people. We (most of us, anyway!) don't get our jollies by flipping bits. Direct manipulation is an illusion that clothes a deeper reality.

But in an important sense, we use the same illusion in real life. When we see a pin and pick it up, at a low level electromagnetic waves (photons) interact with free electrons in the pin, propagate through space, are absorbed by molecules, and cause further rearrangements of atoms. At a mid-level, cells in the retina absorb light, then trigger a cascade of neural firings which lead ultimately to conversion of chemical energy to motion. High-level, we recognize an object, decide to act, bend over, and grasp it. We ignore the curved-space gravitational forces that keep the pin on the ground; the solid-state physics that makes its atoms cohere; the biochemistry that fuels our bodies; the mechanics of bones and joints, tendons and muscles; and the psychology, philosophy, and mathematics of thinking and choosing.

So direct manipulation computer interfaces aren't such a conjurer's sleight-of-hand trick after all!

Wednesday, October 13, 1999 at 07:49:48 (EDT) = 1999-10-13

TopicScience - TopicProgramming

(correlates: ManyWorlds, PersonalCosmology, Comments on 2008-10-08 - Butter and Eggs, ...)