Perception  and Interaction

Dr. C. George Boeree
Shippensburg University

Perception

Perception -- seeing, hearing, touching, smelling, tasting, feeling the positions of joints and the tension of muscles, balance, temperature, pain... -- begins with the stimulation of sensory neurons.  Each sense involves highly evolved cells which are sensitive to a particular stimulus:  Pain receptors respond to certain chemicals produced when tissues are damaged.  Touch receptors involve cells with hairs which, when bent, cause signals to travel down the cell's axon.  Balance, movement, and even hearing involve similar hair cells.  Temperature sensitive neurons have hairs that expand and contract in response to heat and cold.  Taste and smell receptors respond to environmental molecules in the same way that other neurons respond to neurotransmitters.  And the neurons of the retina respond to the presence of light or the specific frequency ranges of light we perceive as color.

But perception is more than just passive reception of information.  Perception is an active process:  Touch, for example, requires movement -- something that nowadays we call "scanning." Touch includes information about you (e.g. your muscles, joints) as well as about what you are touching.  We can say the same about hearing. We should really call it listening!  The sound itself is intrinsically moving, of course -- it is constantly changing. If it didn’t, we would stop hearing it!

And the same is true about vision. Vision involves constant movement -- of our eyes, head, and body, or of the things we see or all of the above. The outer parts of our retina are particularly sensitive to motion -- so when something comes into our field of vision, our attention is drawn to it. Even the fact that we have two eyes (binocular vision) is a kind of movement -- the two views are slightly different! If we kept our eyes and the scene we are looking at perfectly still, everything would all become white!

We should also keep in mind that perception is not something done with the eyes or the ears or any specific sense organ. It is a multi-sensory, full bodied thing, totally involving: "A one-year-old child standing on the floor of a room will fall down if the walls are silently and suddenly moved forward a few inches, although nothing touches him." (Neisser, p. 116, referring to Lee and Aronson, 1974)

The "raw material" of perception

The perceptual process has a lot to work with from the very beginning: We are not in the position of having to make sense of a mosaic of meaningless dots of light or disconnected sounds or smells.  Even as far back as the 1890's, William James pointed out that we not only perceive things but relationships such as "and" and "or" as well.

Take your hands and hold them up in front of you, separated by a few inches.  You see your hands, of course, and it isn't hard to imagine that perceiving them is a matter of certain patterns of light followed by similar patterns of neural firings.  But notice that you are also perceiving your hands as "next to each other," while in fact that perception is not directly presented to you as a stimulus in the way your hands themselves are.

Much of what we experience comes "pre-packaged," ready for our consumption.  Nature provides "edges" -- changes in light patterns, transitions of sounds -- for us to use to pick objects out from their environment. For example, we see things as standing out from their background, something called the figure-ground phenomenon, introduced by the Danish phenomenologist Edgar Rubin (1886-1951).

Rubin demonstrated the phenomenon by creating his classic example of an ambiguous figure-ground situation:

Basically, we perceive one aspect of an event as the figure and the other as the ground. In Rubin's figure, there is no true figure and ground.  It is a drawing that pretends to be an object.  We are forced by the ambiguity of it to use the shifting attention we give the vase or the faces to see one thing or the other.

Depth is a major example of something we experience directly, without the need for anticipatory interpretation. Traditionally, it has been assumed that we construct depth from such clues as perspective and relative size, as well as the slightly different images we get with binocular vision.  But we only need to use such clues when we are looking at pictures that are faking depth!  Actually, we see true depth because it is there to see.

Again, it is the fact that vision involves movement that shows us the truth of the matter.  For example, things that are closer to us change position more quickly than things that are farther away, and distant objects form the backgrounds for the closer ones.  Remember from childhood how the moon seems to follow you as you drive along, while telephone poles whip by at a million miles an hour?  Even binocular vision -- the disparate images of each eye -- is a kind of movement.

Eleanor Gibson won her place in the history of psychology books with her visual cliff experiment. She built a special table: One half had plexiglass with a checkerboard pattern stuck beneath it. The other half also had plexiglass, but the checkerboard pattern was a couple of feet below, on the floor. In between was a board. Infants were then placed on the board, and their mothers were asked to coax them to crawl over one side or the other. Guess which side they didn't want to try? Apparently, babies are quite capable of seeing depth with very little, if any, experience with "cliffs."

Gestalts

The Gestalt psychologists (Wertheimer, Koffka, and Kohler) discovered many other ways in which what we perceive is already organized.

Gestalt psychology is based on the observation that we often experience things that are not a part of our simple sensations. The original observation was Wertheimer’s, when he noted that we perceive motion where there is nothing more than a rapid sequence of individual sensory events. This is what he saw in a toy stroboscope he bought at the Frankfurt train station, and what he saw in his laboratory when he experimented with lights flashing in rapid succession (like the Christmas lights that appear to course around the tree, or the fancy neon signs in Los Vegas that seem to move). The effect is called the phi phenomenon, and it is actually the basic principle of motion pictures and television!

If we see what is not there, what is it that we are seeing? You could call it an illusion, but it's not an hallucination. Wetheimer explained that you are seeing an effect of the whole event, not contained in the sum of the parts. We see a coursing string of lights, even though only one light lights at a time, because the whole event contains relationships among the individual lights that we experience as well.

We are built to experience the structured whole as well as the individual sensations.  And not only do we have the ability to do so, we have a strong tendency to do so. We even add structure to events which do not have gestalt structural qualities.

In perception, there are many organizing principles called gestalt laws. The most general is called the law of pragnanz. Pragnanz is German for pregnant, but in the sense of pregnant with meaning, rather than pregnant with child. This law says that we are innately driven to experience things in as good a gestalt as possible. “Good” can mean many things here, such regularity, orderliness, simplicity, symmetry, and so on, which then refer to specific gestalt laws.

For example, a set of dots outlining the shape of a star is likely to be perceived as a star, not as a set of dots. We tend to complete the figure, make it the way it “should” be, finish it. Like we somehow manage to see this as a "B"...

The law of closure says that, if something is missing in an otherwise complete figure, we will tend to add it. A triangle, for example, with a small part of its edge missing, will still be seen as a triangle. We will “close” the gap.

The law of similarity says that we will tend to group similar items together, to see them as forming a gestalt, within a larger form. Here is a simple typographic example:

It is just natural for us to see the o’s as a line within a field of x’s.

Another law is the law of proximity. Things that are close together as seen as belonging together. For example...

You are much more likely to see three lines of close-together *’s than 14 vertical collections of 3 *’s each.

Next, there’s the law of symmetry. Take a look at this example:

Despite the pressure of proximity to group the brackets nearest each other together, symmetry overwhelms our perception and makes us see them as pairs of symmetrical brackets.

Another law is the law of continuity. When we can see a line, for example, as continuing through another line, rather than stopping and starting, we will do so, as in this example, which we see as composed of two lines, not as a combination of two angles...:

The gestalt psychologists also pointed out that, when we see a duck and an elephant, there is little in their natures that would cause us to perceive them as belonging together.  But if they are both walking together in the same direction across a field, their common path is immediately perceived as a connection between them.

But the gestalt principles are by no means restricted to perception -- that’s just where they were first noticed. Take, for example, memory. That too seems to work by these laws. If you see an irregular saw-tooth figure, it is likely that your memory will straighten it out for you a bit. Or, if you experience something that doesn’t quite make sense to you, you will tend to remember it as having meaning that may not have been there. A good example is dreams: Watch yourself the next time you tell someone a dream and see if you don’t notice yourself modifying the dream a little to force it to make sense!

Gestalt theory is well known for its concept of insight learning. People tend to misunderstand what is being suggested here: The Gestalt psychologists are not so much talking about flashes of intuition, but rather solving a problem by means of the recognition of a gestalt or organizing principle.

The most famous example of insight learning involved a chimp named Sultan. He was presented with many different practical problems (most involving getting a hard-to-reach banana). When, for example, he had been allowed to play with sticks that could be put together like a fishing pole, he appeared to consider in a very human fashion the situation of the out-of-reach banana thoughtfully -- and then rather suddenly jump up, assemble the poles, and reach the banana.

A similar example involved a five year old girl, presented with a geometry problem way over her head: How do you figure the area of a parallelogram? She considered, then excitedly asked for a pair of scissors. She cut off a triangle from one end, and moved it around to the other side, turning the parallelogram into a simple rectangle. Wertheimer called this productive thinking.

The perception of utility

Many psychologists, including phenomenologists and Gestalt psychologists, talk about the direct perception of the use of objects.  Some things "afford various possibilities for action, carry implications about what has happened or will happen, belong coherently to a larger context, possess an identity that transcends their simple physical properties." (Neisser, p. 71)

Rubin called it utility determination: "We see immediately that a hammer is meant for hammering, a pencil for writing, a pipe for smoking." (From, p. 15)  Koffka called this demand character: "Each thing says what it is...a fruit says 'Eat me;' water says 'Drink me;' thunder says 'Fear me...'" (Koffka, p. 7)

J.J. Gibson calls them affordances: "The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill....  the 'values' and 'meanings' of things in the environment can be directly perceived." (Gibson, p. 127)

"An elongated object of moderate size and weight affords wielding. If used to hit or strike, it is a club or hammer. If used by a chimpanzee behind bars to pull in a banana beyond its reach, it is a sort of rake. In either case, it is an extension of the arm. A rigid staff also affords leverage and that use is a lever. A pointed elongated object affords piercing -- if large it is a spear, if small a needle or awl." (Gibson p. 128)

And likewise for objects that afford cutting (knives) or throwing (balls) or binding (rope) or "trace making" (a pen, brush, pencil...).

"The different places of a habitat may have different affordances. Some are places where food is usually found and others where it is not. There are places of danger, such as the brink of a cliff and the regions where predators lurk. There are places of refuge from predators. Among these is the place where mate and young are, the home, which is usually a partial enclosure. Animals are skilled at what the psychologist calles place-learning. They can find their way to significant places." (Gibson, p. 136)

"The medium, substances, surfaces, places, and other animals have affordances for a given animal. They offer benefit or injury, life or death. This is why they need to be perceived." (Gibson, p. 143)

"The different substances of the environment have different affordances for nutrition and for manufacture. The different objects of the environment have different affordances for manipulation. The other animals afford, above all, a rich and complex set of interactions, sexual, predatory, nurturing, fighting, playing, cooperating, and communicating. What other persons afford, comprises the whole realm of social significance for human beings." (Gibson, p. 128)

Person perception

As the last quote suggests, this idea of directly perceiving the meaning of things applies to ourselves as well as other animals! Philosophers sometimes talk about the "problem of the other:" How is it that we know that another person is in fact another person, like us, conscious, capable of thought and feeling? Do we notice that there are similarities to how we ourselves behave, and somehow reason our way to that conclusion? Or is it that we just see their person-hood? I believe the latter.

The phenomenological psychologist Franz From had people look at a variety of movies and describe what they saw. He discovered that "When we have to describe a behavior sequence, we generally do so by indicating a perception of some psychological state in the behaving person." (From, p. 7)

"...when we perceive human behavior as action...implicit in the percieved material sequence there is a certain sens. By this, I mean that we are perceiving the behavior as being governed by a mental factor." (From, p. 69)  This mental factor is also called intention, purpose, or meaning.

We can see "sens" in the behavior of animals, even insects: I can't tell you how impressed I've been with praying mantises and garden spiders. They really look at you, follow your movements, respond with great care... even though their brains are as small as a grain of rice!

This even applies to things that aren't really alive at all -- i.e. we can be quite mistaken about "sense!"  Fritz Heider and Marianne Simmel did an experiment involving a film of triangles moving about in "purposeful" ways: People saw the triangles as having intentions! Rubin refered to other people, animals, and even apparently purposeful triangle as psychoid entities.

When we observe people, the absence of meaning is actually the special case! From tells this story:

"One afternoon when Professor Rubin and I had already put on our overcoats, ready to go home from the laboratory, Rubin said: 'See here, From.' At the same moment he sat down at is desk and looked straight ahead while he made short abrupt horizontal movements right and left in the air in front of him with his right hand, keeping the index finger and the thumb closely together. I just managed to think something like 'What on earth has happened to Rubin,' when he got hold of a pencil and a piece of paper, drew a system of small arrows and pushed the paper across to me, saing: 'Here is the code to the safety lock on my bicycle. Would you mind riding the bike home for me?' The earlier perception of something completely incomprehensible was immediately replaced, and the purpose of his behavior, i.e., to note down the code which he 'had in his fingers,' became quite apparent...." (From, p. 13)

Anticipation

Perception in the broadest sense is a matter of interaction between the world and the self.  At its simplest, the world gives us events; we in turn give those events meaning by interpreting and acting upon them.

There are some obvious details here: we have sensations (input from the world, stimuli) and actions (output to the world, responses). There was a time when psychologists thought this was enough. Now we know better, and we add two more details, which are called anticipation and adaptation.

Anticipation is a little difficult to explain. We have a certain knowledge of the world, a "model" of it. This model includes everything from little details like which shoe you put on first to complex things like how you feel about yourself and your life. We use this model to anticipate -- expect, predict -- what will happen in the next moment or in the next ten years.

If I close my eyes, I expect that when I open them you will still be there, the room will still be there, I will still be there, and so on. If all you of you were to disappear on me I would be seriously surprised.  We also anticipate on a more long term basis: We have expectations about what college will and won't do for us, about love being forever, and the sun rising, and so on.

If I keep my eyes closed and focus on the expectation, rather than on you and the world "out there," I can imagine you. We can understand images and thoughts as anticipations temporarily detached from the stream of events!  "Images are not pictures in the head, but plans for obtaining information from potential environments....  When you have an image of a unicorn at your elbow -- while quite certain that unicorns are purely mythical animals -- you are making ready to pick up the visual information that the unicorn would provide, despite being fully aware that your preparations are in vain." (Neisser pp. 131-132)

Thinking, says Neisser, is also a matter of imagery: "The ability to divide, detach, and manipulate our own anticipations is immensely important. It is, I believe, the fundamental operation in all so-called higher mental processes." (Neisser, p. 133) He goes even further by suggesting that perception, imagery, learning, memory, behavior... are all of a piece, which he refers to as cognition: "Cognition is the activity of knowing; the acquisition, organization, and use of knowledge." (Neisser, p. 1)

Anticipation is a major factor in much of perception. For example, when we are looking at some rather poor handwriting, we can still make it out. But we may interpret the very same blob of ink in very different ways depending on the context -- that is, depending on what we anticipate should be there!

Take a look at the next drawing.  An infant is likely to react to this by sticking pieces in his or her mouth.  A young child may see them as little people or "finger-clickers."  An adult who does not play chess may see them as chess pieces on a board.  When asked what  the two pieces in the foreground are, they might say they are castles.  A beginning chess player would call them rooks, and might add that the white bishop can take the black queen (or vice versa).  They "see" the moves of the pieces, the rules of the game.  A good chess player might note that it is checkmate in one or two moves.  None of these is wrong;  they are simply different meanings applied to the same events.

You might ask:  What is the event really?  But what do you mean by that?  “Really” to whom?  Somebody must always do the seeing, give the meaning.  A physical scientist looking at the pieces and noting their chemical compositions is still giving his or her meaning to the event.

Note, of course, that the "board" is 6 by 6 instead of 8 by 8, that there is no black king, which means there is no game going on, and that in fact this is a drawing -- a set of lines -- and not a set of three-dimensional objects at all.  All of this is an indication of how much our interpretations add to what is "really" there.

Notice how much this adds to the complexity of person perception:  In order to understand and predict and control people's experiences and behaviors, we have to understand the meanings they apply to reality.  No easy trick!

Anticipation is particularly significant in understanding language: from moment to moment, we anticipate which sounds are likely to come next, which grammatical constructions, which meaningful combinations... We can make sense even of a fuzzy, somewhat jumbled conversation.

Anticipation also helps us to understand how we manage to pay attention to some things and not others. How is it we can be listening to a friend in a noisy bar and manage to somehow "filter out" all the other conversations and yet "let in" our friend's voice? We don’t perceive everything that stimulates our senses. How do we 'filter out" the unimportant (less meaningful) stuff? We don't: We just don’t select it! We select things by means of anticipation. We hear the conversation that we are busily involved in, the one we are anticipating moment to moment. The rest is just noise. Likewise with the other senses: We see what we are looking for, we don’t see what we are not looking for.

There are, of course, a few exceptions: certain built-in attention-getters, e.g. loud noises, flashes of light, painful stimuli, sudden movements. These involve inborn responses!

Adaptation

Adaptation is also more difficult to explain. Sometimes, we don't anticipate well. For example, you think you see a friend coming at you and you prepare to give a hearty "hi!" but just as you raise your arm to wave and begin to open your mouth, you realize it's not your friend at all but a complete stranger. (If possible, you convert the raised arm into a back-scratch, and the open mouth into a yawn. If it's too late and you've already said hi, just pretend you know them. This will drive them crazy.)

Whenever you make mistakes, you need to figure out what went wrong, what to do about it, how to make sense of it. As you do, you are improving your understanding of the world and your relation to it; you are improving your "model." This is adaptation. In our example, you may now have a model of the world that includes look-alikes, embarrassing mistakes, and a tendency to hold-off a little in the future before being so exuberant with your hello's. Adaptation is learning, and we will discuss it in depth later.

This additional layer to interaction of anticipation and adaptation is crucial: It means that our behaviors and experiences are not just a function of some common reality. We, ourselves, our understandings of reality, are inevitably and intrinsically a part of our behaviors and experiences. Without "self," reality would be meaningless.

References

From, Franz (1971). Perception of Other People. (Erik Kvan and Brendan Maher, Trans.) New York: Columbia University Press.

Gibson, James J. (1979). The Ecological Approach to Visual Perception. Boston : Houghton Mifflin

Gibson, James J. (1966). The Senses Considered as Perceptual Systems. Boston: Houghton Mifflin

Heider, Fritz and Simmel, Marianne (1944). An Experimental Study of Apparent Behavior. American Journal of Psychology, 57, p. 245

James, William (1890).  The Principles of Psychology.  New York: Holt.

Koffka, Kurt (1936). Principles of Gestalt Psychology. New York.

Lee, D.N. and Aronson, E. (1974). Visual proprioceptive control of standing in human infants. Perception and Psychophysics, 15, 529-532.

Neisser, Ulric (1976). Cognition and Reality. San Francisco: W. H. Freeman.

© Copyright 2002 C. George Boeree