Microsoft Teams: If your video meeting is laggy, our AI might be able to help | ZDNet

If you’ve ever experienced a Teams video meeting with frozen or laggy video, it’s probably the fault of your patchy broadband, and now Microsoft has detailed how it is using artificial intelligence (AI) to improve calls even in those conditions. 

Microsoft’s latest machine-learning models are being put to the task of distinguishing between situations when end users on Teams are sharing static content like slides, and where video quality could be optimized for constrained bandwidth. 

“To make it easier for the end user, Teams uses machine learning to understand the characteristics of the content the user is sharing to ensure participants experience the highest video quality in constrained bandwidth scenarios,” Microsoft said.

SEE: Even computer experts think ending human oversight of AI is a very bad idea

For static content, it optimizes Teams for readability whereas when motion is detected, it prioritizes smooth playback. This detection happens continuously in the Teams client, which runs on different types of hardware. The client can also allocate bandwidth to different content types.

“To ensure that this experience is great across devices, the classifier is optimized for hardware offloading and is able to run real-time with minimal impact on CPU and GPU usage,” Microsoft states. 

“When users share content with their webcam enabled, Teams balances requirements for video and shared content by adaptively allocating bandwidth based on the needs of each content type. We have recently improved this capability by using machine learning to train models using different network characteristics and source content. This enables Teams to transmit video at higher quality and frame rate while maintaining the screensharing experience, all while utilizing the same bandwidth.”

Microsoft shows two demo split-screen videos comparing the quality of playback where there is no packet loss versus packet loss of 5% and 10%, respectively. The examples show there is barely a detectable difference between no packet loss and some packet loss, which is the goal of its use of pack-loss protection, AI content classification and rate control. 

Other areas Microsoft has applied AI to improve the Teams experience include reducing background noise for those working from home or in noisy places, and work on its new Satin audio codec for Teams.