Today in Generative Media

3-Mile Island for Microsoft AI; Creative process for AI film; UAE invests $1T in US AI

News

• Microsoft deal would reopen Three Mile Island nuclear plant to power AI (Washington Post)

• AI filmmaking is "making the creative process so much smoother," says the director of the new Jordan Rudess music video (Creative Bloq)

• UAE hoping to expand $1 trillion partnership with U.S. through AI, Investment (cnbc.com)

• OpenAI closes in on largest VC round of all time (Axios)

• 'Please halt this activity': Not-so-open OpenAI seems to have gone full mob boss, sending threatening emails to anyone who asks its latest AI models probing questions (PC Gamer)

Software

• Upscale with Universal Upscaler (Leonardo AI)

• Moshi: a speech-text foundation model for real time dialogue (GitHub) fully voice to voice ai running locally on my mac (X)

• This is a custom node for ComfyUI that allows you to use the Luma AI API directly in ComfyUI (GitHub)

• KoolCogVideoX is fine-tuned on CogVideoX specifically for interior design scenarios (HuggingFace)

• Excited to share that next month, Adobe Podcast will be releasing Enhance Speech V2. An all-new AI model that produces natural sounding results, now with the ability to keep music, sound effects, and background noise when needed (X)

Research

• LVCD: Reference-based Lineart Video Colorization with Diffusion Models (SIGGRAPH Asia 2024, project page)

• 3DGS-LM: Faster Gaussian-Splatting Optimization with Levenberg-Marquardt (project page)

• FlexiTex: Enhancing Texture Generation via Visual Guidance (project page)

• CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilities (project page)

• SRIF: Semantic Shape Registration Empowered by Diffusion-based Image Morphing and Flow Estimation (arXiv)

• StoryMaker: Towards Holistic Consistent Characters in Text-to-image Generation (arXiv)

Misc

• Difference between GPT-4o and o1 (X)

• Wow Hailuo handles split screen videos really well (X)

• Kling AI def nailed it with their motion brush implementation (X)

• I think it’s going to be really hard for me to step away from Kling AI text-to-video. The results are just too powerful and addictive! (X)

• Starship Chef (Reddit)