Visual search: what can we learn from biology?

Written by Rhiannon White | Edited by Sahng-Ah Yoo

What can you do more quickly and easily than even the most advanced computer software? Visual search is the task of looking for a target from amongst a background of distractor items (e.g., Where’s Wally being a classic case) and until recently, this task was more difficult for computers to do than for you or I. But advances in visual search algorithms, pioneered by researchers such as Dr Anil Bharath, have since led to the design of technology that can master these tasks.

The secret behind visual search
Dr Anil is the founder of the company Cortexica: a leading developer of computer algorithms used commercially for visual search. Together the team have managed to create technology capable of locating search targets despite cluttered backgrounds, poor quality lighting and occlusion by other objects. The success of Cortexica comes from years of research, and a novel insight: the software is based on biological algorithms that are implemented in the visual system of humans and other animals.

Cortexica uses deep learning algorithms, which uses multiple processing layers that each transform and extract features from the data in a different way. In contrast to previous computer networks, which spread their processing units laterally, Cortexica hardware features a stack of neuron-like units that resemble the layered structures of human visual cortex. In both cases, this involves a cascade of lower- to higher-level operations: so that increasingly complex features, from changes in local light intensity to higher order figures, can be identified. Cortexica was therefore unusual in its use of parallel search mechanisms: like in the biological visual system, processing stages occurred simultaneously, allowing a more efficient search to be carried out.

The computer programme also takes inspiration from neural receptive fields. These are a way of expressing what types of visual input are most likely to elicit a firing response in a particular neuron; for example, different neurons fire maximally in response to a stimulus that is in a particular region of space, or that is of a particular orientation. [1] Within a particular layer, different neurons have different receptive fields, so that the pattern of activation of different neurons can be used to infer properties about the visual input. In a similar way, Cortexica uses filters known as spatial kernels applied across the image, which function as artificial receptive fields.

How can visual search programmes be used?
Aside from the engineering aspects, Dr Anil reports that a key stage of the development process was in applying the software to real world tasks. Among its many uses, the team have used the technique for searching documents in the Nation Archives, trailing the internet for copyright violations, and identifying particular items or brands in a retail setting.

In particular, Cortexica software has been of great importance to the fashion industry. For example, by identifying different products that look similar, recommendations can be made to buyers on the basis of their purchase history. Its ability to mimic biological visual systems means that unlike conventional computerised visual search techniques, the results are often closer to what people expect. It is by identifying visual properties under changing light levels and from different angles that Dr Anil and his team have been able to create a programme that not only mimics, but can extend and enhance human visual search.


[1] Hubel, D. H. & Wiesel, T. N. (1968). Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology. 195, 215–243.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s