Swedish entrepreneurs capture the world’s attention with eye-tracking technology

Eye-tracking software has been under development for at least two decades, and one of the leading companies in the domain is a startup success case from Sweden’s Stockholm area. Tobii, which now employs more than 600 people around the world, builds devices that provide attention computing services, of which eye tracking is a subset. 

Eye-tracking technologies were first used for scientific research purposes. Studies conducted in a controlled environment tracked eye movements to draw conclusions about where subjects were directing their attention. 

Unaware of all the work that had already gone into eye tracking for use in scientific research, three Swedish entrepreneurs came up with their own camera-based solution using infrared illumination to paint a pattern on the user’s eyes, which the camera then recognises. Software analysed the data to determine the direction of the person’s gaze.

The three entrepreneurs established Tobii 20 years ago on the premise that their technology could be used to enable people to control a computer with their eye movements. 

Now, two decades later, the system is much more sophisticated – and the software uses some of the latest computer vision and machine learning techniques to determine much more than just where someone is looking. The techniques are very similar, whether used as a screen-based or a wearable eye-tracking system. 

The market has changed in the meantime. Enterprises now use eye-tracking technologies for a growing number of applications, ranging from measuring user reactions to new products to detecting drowsy car drivers. Requirements have also changed. 

In the early days, when eye tracking was used only in scientific settings, it was easy to control the environment. Researchers would have the subject put their head in a stand to keep it still, and they could also control the lighting. If a subject was particularly difficult to track, the researchers could just pick somebody else for the study. But with business applications, the objective is to attract users and to operate in as many environments as possible.  

“The challenge with eye tracking isn’t so much getting it to work as it is getting it to work for all people in all types of environment,” says Anand Srivatsa, CEO of Tobii. “For the last 20 years, much of what the company has been doing is looking at how we can take eye-tracking technology and make it robust over a large population. We want to cover people with different ethnicities where the shape of the eye is different, and the colour of the iris is different. We want to cover people with eye conditions, such as lazy eye.

“Not only are people different, but the environment might include bright light, or there might be light in front of the user. The system has to correct itself for reflections, and it has to work for users wearing glasses.”

For example, enterprises devising new packaging may want to know if their bottle of drink is attractive. One way to do that is to ask a question, but as soon as you ask a question, the subject uses his or her “active mind”, which is usually biased. For example, a man might deny liking the colour pink, when that is really what caught his eye. 

To overcome these biases, commercial market research is now moving towards measuring people’s responses rather than asking them for explicit information. Eye-tracking technology is perfectly adapted to this need. 

The first area of commercial success for Tobii was in helping companies understand user preferences. Then the company entered another phase, where it marketed medical-grade devices for communication assistance. The technology could be used to give people a voice – someone with amyotrophic lateral sclerosis (ALS), like the late Stephen Hawking, could use their eyes to access the keyboard.  

“Now, for the last five or six years, our technology has become mature enough to be considered for mass-market deployment,” says Srivatsa. “Today we see a lot of opportunity in gaming and in extended realities – virtual reality (VR) and augmented reality (AR). We also see opportunities in automotive for driver monitoring systems, healthcare, education and training. At the same time, we continue to market products and services for consumer research and behaviour.”

The company has now moved beyond simple gaze detection to measuring other things to gain deeper insights into human attention, says Srivatsa. “We’ve started with this first measure, which is where you look. But there are other biometrics you can measure when you look at images of people’s eyes or their faces, like where their head is pointed, whether you are pointing perpendicularly to the screen. Are you moving your head to the left or the right? We call that head pose.

“Of course, when you have a picture of somebody’s face, you could do facial identification like you do for your iPhone. You can look at how open the eye is. You can tell if the person is about to shut his or her eyes. You can look at things like pupil diameter, which can be an indicator for other types of higher signals.

“When we look at images of people’s faces and their eyes, we are able to measure additional signals, which we call core signals. And then the other thing we’ve done in the last two decades is to combine these core signals into higher-level signals, which we call attention signals.

“In the automotive space, we have attention signals around drowsiness, to help a car determine if you need a coffee break, or whether you are distracted”

“We can tell if you’re looking at something like a fixation, or if your eyes are moving rapidly in what is called a saccade. Or, based on pupil dilation, you can understand things about cognitive load. You can detect if somebody is stressed, and things like that.

“These are all signals that, in scientific research or in other applications, can be quite important to measure and understand. In the automotive space, we have attention signals around drowsiness, to help a car determine if you need a coffee break, or whether you are distracted.”

Tobii built up layers of signals, which brought them from eye tracking to the more general area of “attention computing”, of which eye tracking is an essential component. It is no accident that parents tell their kids to “look at me when I’m speaking to you”: where people fix their gaze says a lot about where they are directing their attention. 

One of the areas for future innovation is AR, where users wear a pair of normal-looking glasses, into which all the technology has to fit snugly. Power management is a big concern, with the eye-tracking software sharing a limited battery life with networking and display and everything else. Miniaturisation, lowering power consumption and reducing the cost of the overall solution are all areas where more innovation is needed. 

Tobii markets a variety of platforms, some of which are USB peripherals operating at less than 10W. They also have systems that sit within a VR headset in a digital signal processor in the Qualcomm system on chips running the headset. One of the innovations of the last two decades has been to reduce the footprint, both from the sensors needed to capture the pictures of the eye and do the painting that you need, and from the chip that performs the higher-level analysis required by the use case. 

Whenever new solutions are found, anyone making eye-tracking platforms must then make those solutions work for a large population and in a variety of environments. “Even if we fix the population problem for a certain type of application, as soon as you miniaturise, you have to get back to that level of population,” says Srivatsa.  

“Ideally, you want to say this works 100% of the time on everybody, which is a goal that is probably impossible to attain. But if you want to be in consumer markets, you have to be in the high 90s in percentage of coverage. This means that every time you miniaturise, you have to work the population problem again to get back to the high 90s. Otherwise, you can’t put a product out.”