Visual Processing

How does your brain enable you to see? Recordings from the brains of animals have allowed neuroscientists to piece together an account of how the brain processes isolated attributes of an image such as colour, shape, movement. The latest results show how we may soon get more direct information about how these processes work in humans.

You probably know that, just like a camera, the eye forms an image of whatever lies in front of it, on the retina, a light-sensitive outgrowth of the brain that lines the back of the eyeball. But the similarity with cameras, which are designed to record and playback faithful images, ends almost immediately. The brain is not concerned with faithful representation but with meaning. It processes images to extract significant information.

Selection is crucial for information processing. In the brain selection operates at every level. When you scan the list of share prices in the FT you select the the ones that are relevant to your financial survival. Your retina selects information needed to answer the biological survival questions – “Can I eat this? Can it eat me? Can I mate with it? Can I catch it? Will it catch me?”

For this reason colour which indicates the ripeness of fruit and, in some species, the availability of mates; rapid changes in brightness which can be processed to reveal movement; and localised differences in brightness, which allow us to separate an object from its background and analyse its shape and location; are selected and emphasised by the retina. Gradual variations in brightness, which tell us little of survival value, are discarded.

The art of information processing is to bring together what are initially separate pieces of data so that the combination tells us something new. A supermarket might combine a customer address list with information about purchasing histories to send a promotional mailshot for a new cat food to rich people who have bought cat food in the past.

The brain uses the same approach. It works in stages to extract increasingly complex information. It starts with an instantaneous moment by moment record of how much light is caught by individual receptors, each of which catches the light from a tiny patch of the retinal image. Local bright spots and dark spots can be extracted from the image by subtracting from each receptor the average signal collected by the receptors that surround it.

Colour vision exploits the fact that there are three kinds of receptor which differ in the efficiency with which they catch different colours of light (red, green and blue). The individual receptors do not signal the colour of light: high activity in a red-sensitive receptor occurs both when the light that falls on it is bright, and when it is red. In order to preserve colour and discard brightness information the retina subtracts from the red-sensitive receptor the signal of neighbouring green-sensitive receptors. This produces a neurone that responds to red, and does not confuse colour with brightness. The same logic is used in the retina to produce neurones that signal green, blue and yellow.

We know that both these processes happen in the retina because the neurones that transmit from the eye into the brain are activated by patches of colour – especially red, green, blue or yellow – or by local differences in brightness – a dark spot surrounded by a bright background or vice versa. At later stages of visual processing, neurones select more complex shapes. These include recognisable features of visual patterns, such as lines, edges and corners. Thus the different neurones carry information which could be combined to produce recognisable representations of everyday objects. Other neurones select different directions of motion, more specific colours, or objects at particular distances.

This detailed information about visual processing comes from recording the electrical activity of neurones in the brains of animals. It is an article of faith for modern neuroscientists that the electrical signals in neurones generate what we see, and that our neurones have similar responses to those of animals. I once caused a colleague to believe that the laboratory lights were flickering because I accidentally passed a weak electric current through his eyeball. Evidently the change in activity I had induced in the neurones in his retina was similar to that which occurs when lights flicker.

It may soon be possible to get much clearer information about visual processing in human retina. Last week Austin Roorda and David Williams of the Center for Visual Science at Rochester, NY USA published the first ever pictures of the different types of receptor in the central part of the living human retina. These showed that the pattern and the relative numbers of the different colour-sensitive receptors differs from eye to eye. By comparing how eyes with known receptor distributions perform in different visual tasks it will be possible to infer how the individual receptor signals are processed. The eyes will become the window of the brain!