The takeaway: Microsoft, Sony and other companies have filed several patents in the past year that use generative AI to change content in real time. Patents don’t always turn into products, but the frequency of filings shows a strong interest in real-time AI editing. Microsoft says its latest patent is intended only for broadcast sanitization and translation, but the basic concept raises concerns about accuracy and misinformation.
AI-edited and AI-generated videos have been among the most popular generative AI applications since the technology’s initial explosion in popularity. However, a Microsoft patent application from late last month, recently uncovered by Patentlyze, aims to improve the way AI can handle real-time streaming.
The first claim describes an AI that listens for specific words in a show’s audio track and replaces them with new words based on information from the viewer’s profile, such as age, location and native language. Microsoft claims the AI would remove pauses and filler words to clean up the presentation, but it could also censor profanity and replace words based on internet searches.
While the claim could also support immediate fact-checking, it doesn’t say whether it would tell users what changes were made and why. Additionally, the ability to hide edits by adjusting lip movements in the intervening images suggests that viewers could end up watching AI-edited streams without realizing it. Real-time face swapping using artificial intelligence is already popular with scammers. The ability to change what is said on a livestream could take misinformation and fraud to new levels.
Meanwhile, a separate, independent claim describes the use of the technology for translation. YouTube already supports AI audio translation, but Microsoft’s patent would translate live streams with matching lip sync. Furthermore, contrary to the first claim, the translator retains the original record as the “ground truth.”
These patents share a core idea of transforming the user experience in real time with AI. For example, an ambitious lawsuit filed by Microsoft in July aims to rewrite a game’s code to add content based on player behavior.
Meanwhile, Sony recently filed several patents that would improve gaming performance and reduce input latency by using AI to predict upcoming frames and which key a user will press next. Another would use AI to control timing, difficulty, and other aspects of the game, or even to step in for players when they temporarily leave online sessions.
