Today in Generative Media

Lionsgate & RunwayML; SocialAI: Twitter without humans; YouTube gets genAI videos

News

• Runway Partners with Lionsgate (RunwayML)

  • Lionsgate Inks Deal With AI Firm to Mine Its Massive Film and TV Library (Hollywood Reporter)

  • Lionsgate signs deal to train AI model on its movies and shows (The Verge)

  • Generative AI startup Runway inks deal with a major Hollywood studio (TechCrunch)

• SocialAI: we tried the Twitter clone where no other humans are allowed (The Verge) I Stared Into the AI Void With the SocialAI App (Wired)

• YouTube will use AI to generate ideas, titles, and even full videos (The Verge)

• How we’re increasing transparency for gen AI content with the C2PA (Google Keyword blog)

• A bottle of water per email: the hidden environmental costs of using AI chatbots (Washington Post)

• Google just stepped up its AI fight with Apple (Business Insider)

• AI Adds Human Cries and Creepy Laughs to People’s Songs Without Warning (Vice)

• OpenAI threatening to ban users for asking Strawberry about its reasoning. (Futurism)

Software

• Real Time Face Swap (FaceCam)

• Speech-to-Speech AI on Your Local Computer (X)

• Pixtral 12B - the first-ever multimodal Mistral model. Apache 2.0 (Mistral)

• UltraPixel: Advancing Ultra-High-Resolution Image Synthesis to New Peaks (HuggingFace)

• CogVideoX-5B Huggingface Space🤗 (HuggingFace)

Research

• SplatFields: Neural Gaussian Splats for Sparse 3D and 4D Reconstruction (ECCV 2024, project page)

• 3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion (project page)

• Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think (project page)

• NVLM: Open Frontier-Class Multimodal LLMs (project page)

• Phidias : A Generative Model for Creating 3D Content from Text, Image, and 3D Conditions with Reference-Augmented Diffusion (project page)

• EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer (project page)

• Learning Source Disentanglement in Neural Audio Codec (project page)

• OmniGen: Unified Image Generation (arXiv)

• OSV: One Step is Enough for High-Quality Image to Video Generation (arXiv)

• Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control (arXiv)

Misc

• A guy used Runway video-to-video to turn a clip of his weed stash into a drone flight down the Amazon. (X)

• Let me present to you likely the first AI-to-film transfer in the world. (YouTube)