If you spend time watching movies
or TV, you have probably know that you see a moving image on the screen, but
that the sense of motion is created by your brain from a series of static
images. Typical movies, for example,
flash 24 frames per second. Somehow, the
brain takes the changes from one frame to the next and gives you the illusion
of fluid movement.
How does that happen? Take a moment to try to explain it to
yourself.
This question is just one of many
that is explored in a great book that came out in 2015 called Flicker
by cognitive neuroscientist Jeff Zacks.
The book itself explores a variety of topics ranging from low-level
aspects of the way the visual system understands the images on the screen all
the way to high-level topics like the reason why movies are so good at creating
emotion.
So, why do movies move?
You might think that what is
happening is that each image persists a little on the retina (the cells at the
back of your eye that respond to light) and that changes in the image are
detected there. Or perhaps, when the
image is first processed in the brain, it recognizes small discrepancies from
one frame to the next.
Neither of those possibilities is
quite right.
The images from the screen enter
the eye and hit the retina. From there,
they are passed into the brain and ultimately make their way to the occipital
lobe in the back of the brain where most visual processing is done. Initially, the brain looks for simple visual
features in the image like the presence of edges, because edges usually signal
the boundaries of objects.
The interesting thing is, the
brain divides up the task of understanding the image in multiple ways, with
different brain areas searching for different features in the image. The unified sense of vision we have arises
because the brain ultimately puts all of those independent properties back
together.
Early in the processing of images
a particular area of the brain called area MT (shown in the figure) looks for
blobs that have changed position. When
MT sees a blob in a location that has changed it position a bit, it gives a
signal suggesting that there was motion.
Sustained activity of MT indicating motion in a particular direction
gives people the experience that an object moved. This brain area doesn’t really care much
about the blobs themselves. The blobs
could actually change shape or color from one frame to the next, it is just
looking for motion.
So, the motion in movies comes
from activity in the brain area MT. As
Zacks points out, though, this can sometimes cause problems. In particular, when a movie is put together,
it is usually constructed from a set of scenes that are spliced (or cut)
together. If the editor is not careful,
when one scene is cut to the next, some of the objects may appear to jump from
one location on the screen to another.
This jarring sense of movement is called a jump cut. Filmmakers try not
to create these jump cuts, and texts on film making give suggestions for how to
avoid them.
These jump cuts are caused by the
same process that causes the sense of motion in scenes. When one scene is cut to another, if area MT detects
the motion of a blob, it will send a signal that an object in one location
actually moved to another. That can feel
weird, because brain areas that
calculate the size, color, and shape of the object may not see strong
similarities in the objects from one scene to the next, so you can get a
feeling of motion without have a clear sense of what moved.
One of the reasons why Flicker is an interesting read is that
Zacks explores the ways that movies exploit the structure of the brain to give
us an immersive experience. If you can
pull yourself away from the screen for a few hours, it is well-worth the time
to check it out.
