Automated analysis of animal behavior through AI

Researchers have developed a new method that uses artificial intelligence to analyze animal behavior. This opens the door to longer-term in-depth studies in the field of behavioral science—while also helping to improve animal welfare. The method is already being tested at Zurich Zoo.

Researchers engaged in animal behavior studies often rely on hours upon hours of video footage which they manually analyze. Usually, this requires researchers to work their way through recordings spanning several weeks or months, laboriously noting down observations on the animals’ behavior. Now researchers at ETH Zurich and University of Zurich have come up with an automated way to analyze these kinds of recordings. The image-analysis algorithm they have developed makes use of computer vision and machine learning. It can distinguish individual animals and identify specific behaviors, such as those that signal curiosity, fear, or harmonious social interactions with other members of their species.

The technology essentially offers scientists a one-click solution for automatically analyzing video footage, however lengthy or detailed the recordings are. Another advantage of the new method is its reproducibility: if different groups of researchers use the same algorithm to analyze their video data, comparing results is easier because everything is based on the same standards. What’s more, the new algorithm is so sensitive that it can even identify subtle behavioral changes that develop very gradually over long periods of time. “Those are the kinds of changes that are often tricky to spot with the human eye,” says Markus Marks, lead author of the research study and a postdoc in the group headed by Professor of Neurotechnology Mehmet Fatih Yanik.

Suitable for all animal species

The researchers trained the machine-learning algorithm with video footage of mice and macaques in captivity. However, they stress that the method can be applied to all animal species. News of their new method has already spread through the scientific community. The ETH researchers have made the algorithm available to other researchers on a public platform, and many of their colleagues around the world are already using it. “Interest has been particularly high among primate researchers, and our technology is already being used by a group that is researching wild chimpanzees in Uganda,” Marks says.

This is probably because the method can also be used to analyze complex social interactions in animal communities, such as identifying which animals groom other members of their group and how often this occurs. “Our method offers some major advantages over previous machine-learning-based behavioral analysis algorithms, especially when it comes to analyzing social behavior in complex settings,” Marks says.

Improving conditions for animals in human care

The new method can also be used to improve animal husbandry, enabling round-the-clock monitoring to automatically single-out abnormal behaviors. By detecting adverse social interactions or the onset of disease early on, keepers can swiftly respond to improve conditions for the animals in their care.

The ETH researchers are also currently collaborating with Zurich Zoo, which wants to further improve its animal husbandry and conduct automated behavioral research. For example, in a recently published study examining patterns of elephant sleep behavior, zoo researchers had to manually annotate nocturnal video recordings. Their hope is that the new method would enable them to automate and upscale such findings in the future.

Finally, the method is used in fundamental research in the fields of biology, neurobiology and medicine. “Our method can recognize even subtle or rare behavioral changes in research animals, such as signs of stress, anxiety or discomfort,” says Yanik. “Therefore, it can not only help to improve the quality of animal studies but also helps to reduce the number of animals and the strain on them.” The ETH Zurich professor is planning to use the method himself as part of his neurobiological research in the field of imitation learning.

The study is published in Nature Machine Intelligence.