Tooncrafter by Huggingface

ToonCrafter is an AI-powered generative cartoon interpolation tool that uses pre-trained image-to-video diffusion models to seamlessly interpolate between two cartoon images, creating fluid visual narratives. It was developed by researchers at The Chinese University of Hong Kong, City University of Hong Kong, and Tencent AI Lab. ToonCrafter’s AI features include a toon rectification learning strategy to adapt live-action video priors to the cartoon domain, a dual-reference-based 3D decoder to preserve fine details, and a flexible sketch encoder for interactive control over interpolation results. The tool takes in two still images and a text prompt describing the dynamics, generating short video clips (~2 seconds) with 16 frames at a resolution of 512×320. While it offers high-quality cartoon transformations, it may struggle with semantically understanding image contents in certain scenarios. ToonCrafter is available as an open-source model on platforms like Hugging Face, making it accessible for various applications such as cartoon sketch interpolation and reference-based sketch colorization.