Scientists at EPFL have developed MammAlps, a groundbreaking multi-view, multi-modal video dataset capturing the behavior of wild mammals in the Swiss Alps. This innovative resource has the potential to transform wildlife monitoring and conservation efforts.

Have you ever pondered how wild animals behave when they are not being observed? Understanding these behaviors is crucial for safeguarding ecosystems, particularly as climate change and human activities alter natural habitats. However, collecting such information without causing disruptions has always been a challenge.

Traditionally, researchers have relied on direct observation or sensors attached to animals, methods that can be intrusive or have limited scope. While camera traps offer a less invasive option, they produce extensive footage that is difficult to analyze.

AI has the potential to assist in this area, but it requires annotated datasets for learning. Most existing video datasets are either sourced from the internet, lacking the authenticity of real wildlife settings, or are small-scale field recordings with limited detail. Additionally, few datasets include the rich context necessary, such as multiple camera angles and audio, to fully comprehend complex animal behavior.

Introducing MammAlps

To tackle this challenge, EPFL scientists have curated MammAlps, the first richly annotated, multi-view, multimodal wildlife behavior dataset in collaboration with the Swiss National Park. MammAlps is designed to train AI models for species and behavior recognition tasks, ultimately enhancing researchers' understanding of animal behavior and streamlining conservation efforts.

MammAlps was developed by Valentin Gabeff, a PhD student at EPFL, under the guidance of Professors Alexander Mathis and Devis Tuia, along with their respective research teams.

Development of MammAlps

The research team set up nine camera traps that recorded over 43 hours of raw footage across several weeks. They meticulously processed the footage, utilizing AI tools to detect and track individual animals, resulting in 8.5 hours of material showcasing wildlife interactions.

Behaviors were labeled using a hierarchical approach, categorizing each moment at two levels: high-level activities such as foraging or playing, and finer actions like walking, grooming, or sniffing. This structure enables AI models to interpret behaviors more accurately by connecting detailed movements to broader behavioral patterns.

To provide AI models with richer context, the team included audio recordings and "reference scene maps" documenting environmental elements like water sources, bushes, and rocks. This additional data facilitates a better understanding of habitat-specific behaviors. They also cross-referenced weather conditions and individual counts per event to create more comprehensive scene descriptions.

"By integrating other modalities alongside video, we have demonstrated that AI models can more effectively identify animal behavior," explains Alexander Mathis. "This multi-modal approach offers a more holistic view of wildlife behavior."

A New Standard in Wildlife Monitoring

MammAlps sets a new standard in wildlife monitoring, providing a complete sensory snapshot of animal behavior across various angles, sounds, and contexts. It also introduces a benchmark for "long-term event understanding," allowing scientists to study broader ecological scenes over time, rather than isolated behaviors from short clips.

Research is ongoing, with the team currently processing data collected in 2024 and conducting additional fieldwork in 2025. These surveys are crucial for expanding the recordings for rare species like alpine hares and lynx, as well as for developing methods to analyze wildlife behavior temporally across multiple seasons.

Creating more datasets akin to MammAlps could significantly enhance current wildlife monitoring efforts by enabling AI models to identify behaviors of interest from hundreds of hours of video. This would provide wildlife conservationists with timely, actionable insights, making it easier to track the impacts of climate change, human encroachment, or disease outbreaks on wildlife behavior and protect vulnerable species over time.