🙌Of course, nothing would have been possible without the lead of Weihan Xu (wx83.github.io) and the amazing collaboration with Paul Liang, Haven Kim, Julian McAuley and Taylor Berg-Kirkpatrick!

📜Paper: arxiv.org/pdf/2410.05586
🎦Demo: wx83.github.io/TeaserGen_Of...

📽And old movies!

🧑‍🏫And TED talks!

🎬We tested our models on documentaries of various topics, including environmental, societal and historical issues.

🛠️We proposed two methods for narration-video matching:
1️⃣a pretraining-based model using pretrained contrastive language-vision models and
2️⃣a deep sequential model that learns the mapping between the narrations and visuals.

🛠️We adopted a two-stage approach for generating a documentary teaser:
1️⃣first, we generate the teaser narration given the transcribed narration of the documentary;
2️⃣then, we select the relevant visual content to accompany the generated narration.

🔥We presented the DocumentaryNet, a collection of 1,200+ high-quality documentaries with their teasers.

🎉Super excited to share that our TeaserGen project led by Weihan Xu (wx83.github.io) has been accepted to #ICLR2025!

🔍We explored a new task of generating teasers for long documentaries.

🤩We presented a new dataset, new models, and new evaluation metrics for teaser generation.