Learn More

On December 16, 2025, Meta released SAM Audio, claiming it as the first unified multimodal audio segmentation model. The model supports three prompting methods: text prompts (e.g., input 'dog barking'), visual prompts (clicking on a speaking person or sound-emitting object in a video), and industry-first time segment prompts (marking the time segment where the target audio appears). These prompts can be used individually or in any combination, aiming to let users isolate specific sounds from complex audio mixtures in a natural way.

SAM Audio's core architecture employs a generative modeling framework based on a flow-matching diffusion Transformer. It encodes the audio mixture and one or more prompts into a shared representation, then generates the target track and a residual track. Meta also developed a companion data engine combining advanced audio mixing, automated multimodal prompt generation, and pseudo-labeling pipelines to produce realistic training data. The model is trained on real and synthetic mixtures containing speech, music, and general sound events.

PE-AV is the technical engine behind SAM Audio, built on Meta's Perception Encoder, open-sourced in April 2025. PE-AV extracts frame-level video features and aligns them with audio representations to achieve temporal synchronization of audiovisual information. It integrates PyTorchVideo, FAISS, and contrastive learning frameworks, trained on over 100 million videos from open datasets and synthetic caption pipelines. Meta likens PE-AV to the 'ear' and SAM Audio to the 'brain.'

Meta also released SAM Audio-Bench (the first in-the-wild audio segmentation benchmark) and SAM Audio Judge (the first automatic evaluation model). SAM Audio Judge aims to assess segmentation quality in a way close to human perception, compensating for traditional metrics that rely on reference tracks. All models and tools are available for trial on the Segment Anything Playground from the release date.

On March 10, 2026, Meta and the World Resources Institute jointly released Canopy Height Maps v2 (CHMv2), including an open-source model and global maps. The core improvement of CHMv2 is replacing the DINOv2 backbone in v1 with the DINOv3 self-supervised vision model. DINOv3 is pre-trained on the SAT-493M satellite image dataset, learning tree height features from visual cues such as shadows, textures, and crown shapes without extensive manual annotation.

CHMv2's R² jumped from 0.53 in v1 to 0.86, meaning a substantial improvement in the fit between predictions and ground truth. The model's bias on tall trees is significantly reduced, making predictions more reliable. The training dataset was expanded by adding geographically diverse high-quality LiDAR samples; Meta also built an automatic matching tool and developed a specialized loss function for canopy height estimation.

CHMv1 has been deployed in multiple countries: the UK Forest Research agency uses it for national forest monitoring; the European Commission's Joint Research Centre used it for the 2020 Global Forest Cover Map and plans to use CHMv2 for the 3 Billion Tree Initiative (planting at least 3 billion biodiverse trees by 2030); 10 U.S. cities including Atlanta and Baltimore use it in the Cities for Smart Surfaces urban planning tool. The WRI Ross Center will also use these maps in the Cool Cities Lab scenario planning tool.

Meta acknowledges that CHMv2 still faces challenges, including improving predictions in data-sparse regions, handling perspective geometry effects, and expanding temporal coverage to support change detection. All models and maps are open on Google Earth Engine and GitHub.