There will be model AI video Generator on 14B, 20B, 27B, 32B MoE to active 7B, 8B, 9B? And.... support model AI video Generator type Reasoning/thinking?
Good evening. I saw your model and it looks very interesting, even though it's a small AI model, I think you should try upgrading it from the original 2B to 5B, 7B, 8B, 9B, 10B on MoE. I hope you can outperform the latest closed-code AI models video Generator Wan 2.7, Sora 2, bro.
Thank you! We鈥檙e looking into these directions for Motif Video 2 and will share updates as we progress.
Since Motif is willing to explore ideas for more parameter efficient models, I'd just like to suggest an idea I've had indications would be promising for a while now from hobbyist scale tests. Once the model is stable, you could potentially try removing the text encoder and training direct conditioning embeddings against the frozen DiT (or even initializing them from the text model's encodings, done a word / phrase at a time without padding to build your new tokenizer and embedding layer). I found that could achieve results in older unet models beyond what full finetuning could do, with concept weights able to train in isolated ways without interfering with each other.