# AI Game DevTools (AI-GDT) 🎮

AI-Game

Here we will keep track of the latest AI Game Development Tools, including LLM, Agent, Code, Writer, Image, Texture, Shader, 3D Model, Animation, Video, Audio, Music, Singing Voice and Analytics. 🔥 ## Table of Contents * [Tool (AI LLM)](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#tool) * [Game (Agent)](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#game) * [Code](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#code) * [Writer](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#writer) * [Image](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#image) * [Texture](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#texture) * [Shader](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#shader) * [3D Model](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#model) * [Avatar](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#avatar) * [Animation](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#animation) * [Visual](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#visual) * [Video](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#video) * [Audio](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#audio) * [Music](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#music) * [Singing Voice](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#voice) * [Speech](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#speech) * [Analytics](https://github.com/Yuan-ManX/ai-game-devtools/raw/main/#analytics) ## Project List ### Tool (AI LLM) | Source | Description | Paper | Game Engine | Type | | :------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-----------: | :-----------: | :-------: | | [AgentGPT](https://github.com/reworkd/AgentGPT) | 🤖 Assemble, configure, and deploy autonomous AI Agents in your browser. | | | Tool | | [AICommand](https://github.com/keijiro/AICommand) | ChatGPT integration with Unity Editor. | | Unity | Tool | | [AIOS](https://github.com/agiresearch/AIOS) | LLM Agent Operating System. | | | Tool | | [AI Scientist](https://github.com/SakanaAI/AI-Scientist) | The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. |[arXiv](https://arxiv.org/abs/2408.06292) | | Tool | | [Assistant CLI](https://github.com/diciaup/assistant-cli) | A comfortable CLI tool to use ChatGPT service🔥 | | | Tool | | [Auto-GPT](https://github.com/Significant-Gravitas/Auto-GPT) | An experimental open-source attempt to make GPT-4 fully autonomous. | | | Tool | | [BabyAGI](https://github.com/yoheinakajima/babyagi) | This Python script is an example of an AI-powered task management system. | | | Tool | | [👶🤖🖥️ BabyAGI UI](https://github.com/miurla/babyagi-ui) | BabyAGI UI is designed to make it easier to run and develop with babyagi in a web app, like a ChatGPT. | | | Tool | | [baichuan-7B](https://github.com/baichuan-inc/baichuan-7B) | A large-scale 7B pretraining language model developed by Baichuan. | | | Tool | | [Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B) | A 13B large language model developed by Baichuan Intelligent Technology. | | | Tool | | [Baichuan 2](https://github.com/baichuan-inc/Baichuan2) | A series of large language models developed by Baichuan Intelligent Technology. | | | Tool | | [Bisheng](https://github.com/dataelement/bisheng) | Bisheng is an open LLM devops platform for next generation AI applications. | | | Tool | | [Character-LLM](https://github.com/choosewhatulike/trainable-agents) | A Trainable Agent for Role-Playing. |[arXiv](https://arxiv.org/abs/2310.10158) | | Tool | | [ChatDev](https://github.com/OpenBMB/ChatDev) | Communicative Agents for Software Development. |[arXiv](https://arxiv.org/abs/2307.07924) | | Tool | | [ChatGPT-API-unity](https://github.com/mochi-neko/ChatGPT-API-unity) | Binds ChatGPT chat completion API to pure C# on Unity. | | Unity | Tool | | [ChatGPTForUnity](https://github.com/sunsvip/ChatGPTForUnity) | ChatGPT for unity. | | Unity | Tool | | [ChatRWKV](https://github.com/BlinkDL/ChatRWKV) | ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source. | | | Tool | | [ChatYuan](https://github.com/clue-ai/ChatYuan) | Large Language Model for Dialogue in Chinese and English. | | | Tool | | [Chinese-LLaMA-Alpaca-3](https://github.com/ymcui/Chinese-LLaMA-Alpaca-3) | (Chinese Llama-3 LLMs) developed from Meta Llama 3. | | | Tool | | [Chrome-GPT](https://github.com/richardyc/Chrome-GPT) | An AutoGPT agent that controls Chrome on your desktop. | | | Tool | | [CogVLM](https://www.modelscope.cn/models/ZhipuAI/CogVLM/summary) | CogVLM, a powerful open-source visual language foundation model. |[arXiv](https://arxiv.org/abs/2311.03079) | | Tool | | [CoreNet](https://github.com/apple/corenet) | A library for training deep neural networks. | | | Tool | | [Cosmos](https://github.com/NVIDIA/Cosmos) | Cosmos is a world model development platform that consists of world foundation models, tokenizers and video processing pipeline to accelerate the development of Physical AI at Robotics & AV labs. | | | LLM | | [DBRX](https://github.com/databricks/dbrx) | DBRX is a large language model trained by Databricks. | | | Tool | | [DCLM](https://github.com/mlfoundations/dclm) | DataComp for Language Models. |[arXiv](https://arxiv.org/abs/2406.11794) | | Tool | | [DeepSeek-R1](https://github.com/deepseek-ai/DeepSeek-R1) | DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrated remarkable performance on reasoning. | | | LLM | | [DeepSeek-V3](https://github.com/deepseek-ai/DeepSeek-V3) | DeepSeek-V3 is a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token. |[arXiv](https://arxiv.org/abs/2412.19437) | | LLM | | [DemoGPT](https://github.com/melih-unsal/DemoGPT) | Auto Gen-AI App Generator with the Power of Llama 2 | | | Tool | | [Design2Code](https://github.com/NoviScl/Design2Code) | Automating Front-End Engineering | | | Tool | | [Devika](https://github.com/stitionai/devika) | Devika is an Agentic AI Software Engineer. | | | Tool | | [Devon](https://github.com/entropy-research/Devon) | An open-source pair programmer. | | | Tool | | [Dora](https://www.dora.run/ai) | Generating powerful websites, one prompt at a time. | | | Tool | | [Flowise](https://github.com/FlowiseAI/Flowise) | Drag & drop UI to build your customized LLM flow using LangchainJS. | | | Tool | | [Gemini](https://deepmind.google/technologies/gemini) | Gemini is built from the ground up for multimodality — reasoning seamlessly across text, images, video, audio, and code. | | | Tool | | [Gemma](https://github.com/google/gemma_pytorch) | Gemma is a family of lightweight, state-of-the art open models built from research and technology used to create Google Gemini models. | | | Tool | | [gemma.cpp](https://github.com/google/gemma.cpp) | lightweight, standalone C++ inference engine for Google's Gemma models. | | | Tool | | [GLM-4](https://github.com/THUDM/GLM-4) | GLM-4-9B is the open-source version of the latest generation of pre-trained models in the GLM-4 series launched by Zhipu AI. | | | Tool | | [GPT4All](https://github.com/nomic-ai/gpt4all) | A chatbot trained on a massive collection of clean assistant data including code, stories and dialogue. | | | Tool | | [GPT-4o](https://openai.com/index/hello-gpt-4o/) | GPT-4o (“o” for “omni”) is a step towards much more natural human-computer interaction—it accepts as input any combination of text, audio, image, and video and generates any combination of text, audio, and image outputs. | | | Tool | | [GPTScript](https://github.com/gptscript-ai/gptscript) | Develop LLM Apps in Natural Language. | | | Tool | | [Grok-1](https://x.ai/blog/grok-os) | The weights and architecture of our 314 billion parameter Mixture-of-Experts model, Grok-1. | | | Tool | | [HuggingChat](https://huggingface.co/chat/) | Making the community's best AI chat models available to everyone. | | | Tool | | [Hugging Face API Unity Integration](https://github.com/huggingface/unity-api) | This Unity package provides an easy-to-use integration for the Hugging Face Inference API, allowing developers to access and use Hugging Face AI models within their Unity projects. | | Unity | Tool | | [ImageBind](https://github.com/facebookresearch/ImageBind) | ImageBind One Embedding Space to Bind Them All. |[arXiv](https://arxiv.org/abs/2305.05665) | | Tool | | [Index-1.9B](https://github.com/bilibili/Index-1.9B) | A SOTA lightweight multilingual LLM. | | | Tool | | [InteractML-Unity](https://github.com/Interactml/iml-unity) | InteractML, an Interactive Machine Learning Visual Scripting framework for Unity3D. | | Unity | Tool | | [InteractML-Unreal Engine](https://github.com/Interactml/iml-ue4) | Bringing Machine Learning to Unreal Engine. | | Unreal Engine | Tool | | [InternLM](https://github.com/InternLM/InternLM) | InternLM has open-sourced a 7 billion parameter base model, a chat model tailored for practical scenarios and the training system. |[arXiv](https://arxiv.org/abs/2403.17297) | | Tool | | [InternLM-XComposer](https://github.com/InternLM/InternLM-XComposer) | InternLM-XComposer2 is a groundbreaking vision-language large model (VLLM) excelling in free-form text-image composition and comprehension. |[arXiv](https://arxiv.org/abs/2404.06512) | | Tool | | [Jan](https://github.com/janhq/jan) | Bring AI to your Desktop. | | | Tool | | [Janus](https://github.com/deepseek-ai/Janus) | Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation. |[arXiv](https://arxiv.org/abs/2410.13848) | | LLM | | [Lamini](https://github.com/lamini-ai/lamini) | Lamini allows any engineering team to outperform general purpose LLMs through RLHF and fine- tuning on their own data. | | | Tool | | [LaMini-LM](https://github.com/mbzuai-nlp/LaMini-LM) | LaMini-LM is a collection of small-sized, efficient language models distilled from ChatGPT and trained on a large-scale dataset of 2.58M instructions. | | | Tool | | [LangChain](https://github.com/hwchase17/langchain) | LangChain is a framework for developing applications powered by language models. | | | Tool | | [LangFlow](https://github.com/logspace-ai/langflow) | ⛓️ LangFlow is a UI for LangChain, designed with react-flow to provide an effortless way to experiment and prototype flows. | | | Tool | | [LaVague](https://github.com/lavague-ai/LaVague) | Automate automation with Large Action Model framework. | | | Tool | | [Lemur](https://github.com/OpenLemur/Lemur) | Open Foundation Models for Language Agents. | | | Tool | | [Lepton AI](https://github.com/leptonai/leptonai) | A Pythonic framework to simplify AI service building. | | | Tool | | [Lit-LLaMA](https://github.com/Lightning-AI/lit-llama) | Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. | | | Tool | | [llama2-webui](https://github.com/liltom-eth/llama2-webui) | Run Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). | | | Tool | | [Llama 3](https://github.com/meta-llama/llama3) | The official Meta Llama 3 GitHub site. | | | Tool | | [Llama 3.1](https://github.com/meta-llama/llama-models) | Llama is an accessible, open large language model (LLM) designed for developers, researchers, and businesses to build, experiment, and responsibly scale their generative AI ideas. | | | Tool | | [LLaSM](https://github.com/LinkSoul-AI/LLaSM) | Large Language and Speech Model. | | | Tool | | [LLM Answer Engine](https://github.com/developersdigest/llm-answer-engine) | Build a Perplexity-Inspired Answer Engine Using Next.js, Groq, Mixtral, Langchain, OpenAI, Brave & Serper. | | | Tool | | [llm.c](https://github.com/karpathy/llm.c) | LLM training in simple, raw C/CUDA. | | | Tool | | [LLMUnity](https://github.com/undreamai/LLMUnity) | Create characters in Unity with LLMs! | | Unity | Tool | | [LLocalSearch](https://github.com/nilsherzig/LLocalSearch) | LLocalSearch is a completely locally running search engine using LLM Agents. | | | Tool | | [LogicGamesSolver](https://github.com/fabridigua/LogicGamesSolver) | A Python tool to solve logic games with AI, Deep Learning and Computer Vision. | | | Tool | | [LongWriter](https://github.com/THUDM/LongWriter) | LongWriter: Unleashing 10,000+ Word Generation From Long Context LLMs. |[arXiv](https://arxiv.org/abs/2408.07055) | | Tool | | [Large World Model (LWM)](https://github.com/LargeWorldModel/LWM) | Large World Model (LWM) is a general-purpose large-context multimodal autoregressive model. |[arXiv](https://arxiv.org/abs/2402.08268) | | Tool | | [Lumina-T2X](https://github.com/Alpha-VLLM/Lumina-T2X) | Lumina-T2X is a unified framework for Text to Any Modality Generation. |[arXiv](https://arxiv.org/abs/2405.05945) | | Tool | | [MetaGPT](https://github.com/geekan/MetaGPT) | The Multi-Agent Framework | | | Tool | | [MiniCPM-2B](https://github.com/OpenBMB/MiniCPM) | An end-side LLM outperforms Llama2-13B. | | | Tool | | [MiniGPT-4](https://github.com/Vision-CAIR/MiniGPT-4) | Enhancing Vision-language Understanding with Advanced Large Language Models. |[arXiv](https://arxiv.org/abs/2304.10592) | | Tool | | [MiniGPT-5](https://github.com/eric-ai-lab/MiniGPT-5) | Interleaved Vision-and-Language Generation via Generative Vokens. |[arXiv](https://arxiv.org/abs/2310.02239) | | Tool | | [MiniMax-01](https://github.com/MiniMax-AI/MiniMax-01) | MiniMax-01: Scaling Foundation Models with Lightning Attention. |[arXiv](https://arxiv.org/abs/2501.08313) | | LLM | | [Mixtral 8x7B](https://mistral.ai/news/mixtral-of-experts/) | A high quality Sparse Mixture-of-Experts. |[arXiv](https://arxiv.org/abs/2401.04088) | | Tool | | [Mistral 7B](https://mistral.ai/news/announcing-mistral-7b/) | The best 7B model to date, Apache 2.0. | | | Tool | | [Mistral Large](https://mistral.ai/news/mistral-large/) | Mistral Large is a new cutting-edge text generation model. It reaches top-tier reasoning capabilities. | | | Tool | | [MLC LLM](https://github.com/mlc-ai/mlc-llm) | Enable everyone to develop, optimize and deploy AI models natively on everyone's devices. | | | Tool | | [MobiLlama](https://github.com/mbzuai-oryx/MobiLlama) | Towards Accurate and Lightweight Fully Transparent GPT. |[arXiv](https://arxiv.org/abs/2402.16840) | | Tool | | [MoE-LLaVA](https://github.com/PKU-YuanGroup/MoE-LLaVA) | Mixture of Experts for Large Vision-Language Models. |[arXiv](https://arxiv.org/abs/2401.15947) | | Tool | | [Moshi](https://www.moshi.chat/?queue_id=talktomoshi) | Moshi is an experimental conversational AI. | | | Tool | | [Moshi](https://github.com/kyutai-labs/moshi) | Moshi: a speech-text foundation model for real time dialogue. | | | Tool | | [MOSS](https://github.com/OpenLMLab/MOSS) | An open-source tool-augmented conversational language model from Fudan University. | | | Tool | | [mPLUG-Owl🦉](https://github.com/X-PLUG/mPLUG-Owl) | Modularization Empowers Large Language Models with Multimodality. |[arXiv](https://arxiv.org/abs/2304.14178) | | Tool | | [Nemotron-4](https://arxiv.org/abs/2402.16819) | A 15-billion-parameter large multilingual language model trained on 8 trillion text tokens. |[arXiv](https://arxiv.org/abs/2402.16819) | | Tool | | [NExT-GPT](https://github.com/NExT-GPT/NExT-GPT) | Any-to-Any Multimodal Large Language Model. | | | Tool | | [OLMo](https://github.com/allenai/OLMo) | Open Language Model |[arXiv](https://arxiv.org/abs/2402.00838) | | Tool | | [OmniLMM](https://github.com/OpenBMB/OmniLMM) | Large multi-modal models for strong performance and efficient deployment. | | | Tool | | [OneLLM](https://github.com/csuhan/OneLLM) | One Framework to Align All Modalities with Language. |[arXiv](https://arxiv.org/abs/2312.03700) | | Tool | | [Open-Assistant](https://github.com/LAION-AI/Open-Assistant) | OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so. | | | Tool | | [Open Deep Research](https://github.com/dzhng/deep-research) | An AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, web scraping, and large language models. | | | LLM | | [OpenDevin](https://github.com/OpenDevin/OpenDevin) | An autonomous AI software engineer. | | | Tool | | [Orion-14B](https://github.com/OrionStarAI/Orion) | Orion-14B is a family of models includes a 14B foundation LLM, and a series of models. |[arXiv](https://arxiv.org/abs/2401.12246) | | Tool | | [Panda](https://github.com/dandelionsllm/pandallm) | Overseas Chinese open source large language model, based on Llama-7B, -13B, -33B, -65B for continuous pre-training in the Chinese field. | | | Tool | | [Perplexica](https://github.com/ItzCrazyKns/Perplexica) | An AI-powered search engine. | | | Tool | | [Pi](https://heypi.com/talk) | AI chatbot designed for personal assistance and emotional support. | | | Tool | | [Qwen1.5](https://github.com/QwenLM/Qwen1.5) | Qwen1.5 is the improved version of Qwen. | | | Tool | | [Qwen2](https://github.com/QwenLM/Qwen2) | Qwen2 is the large language model series developed by Qwen team, Alibaba Cloud. | | | LLM | | [Qwen2.5-Coder](https://github.com/QwenLM/Qwen2.5-Coder) | Qwen2.5-Coder is the code version of Qwen2.5, the large language model series developed by Qwen team, Alibaba Cloud. |[arXiv](https://arxiv.org/abs/2409.12186) | | LLM | | [Qwen-7B](https://github.com/QwenLM/Qwen-7B) | The official repo of Qwen-7B (通义千问-7B) chat & pretrained large language model proposed by Alibaba Cloud. | | | LLM | | [RepoAgent](https://github.com/OpenBMB/RepoAgent) | RepoAgent is an Open-Source project driven by Large Language Models(LLMs) that aims to provide an intelligent way to document projects. |[arXiv](https://arxiv.org/abs/2402.16667) | | Tool | | [s1](https://github.com/simplescaling/s1) | s1: Simple test-time scaling. |[arXiv](https://arxiv.org/abs/2501.19393) | | LLM | | [Sanity AI Engine](https://github.com/tosos/SanityEngine) | Sanity AI Engine for the Unity Game Development Tool. | | Unity | Tool | | [SearchGPT](https://github.com/tobiasbueschel/search-gpt) | 🌳 Connecting ChatGPT with the Internet | | | Tool | | [ShareGPT4V](https://sharegpt4v.github.io/) | Improving Large Multi-Modal Models with Better Captions. | | | Tool | | [SkyThought](https://github.com/NovaSky-AI/SkyThought) | Sky-T1: Train your own O1 preview model within $450. | | | LLM | | [Skywork](https://github.com/SkyworkAI/Skywork) | Skywork series models are pre-trained on 3.2TB of high-quality multilingual (mainly Chinese and English) and code data. | | | Tool | | [StableLM](https://github.com/Stability-AI/StableLM) | Stability AI Language Models. |[arXiv](https://arxiv.org/abs/2402.17834) | | Tool | | [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca) | An Instruction-following LLaMA Model. | | | LLM | | [Text generation web UI](https://github.com/oobabooga/text-generation-webui) | A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, OPT, and GALACTICA. | | | Tool | | [TinyChatEngine](https://github.com/mit-han-lab/TinyChatEngine) | On-Device LLM Inference Library. | | | Tool | | [ToolBench](https://github.com/openbmb/toolbench) | An open platform for training, serving, and evaluating large language model for tool learning. | | | Tool | | [Unity ChatGPT](https://github.com/dilmerv/UnityChatGPT) | Unity ChatGPT Experiments. | | Unity | Tool | | [Unity OpenAI-API Integration](https://github.com/himanshuskyrockets/Unity_OpenAI) | Integrate openai GPT-3 language model and ChatGPT API into a Unity project. | | Unity | Tool | | [Unreal Engine 5 Llama LoRA](https://github.com/bublint/ue5-llama-lora) | A proof-of-concept project that showcases the potential for using small, locally trainable LLMs to create next-generation documentation tools. | | Unreal Engine | Tool | | [UnrealGPT](https://github.com/TREE-Ind/UnrealGPT) | A collection of Unreal Engine 5 Editor Utility widgets powered by GPT3/4. | | Unreal Engine | Tool | | [Video-LLaVA](https://github.com/PKU-YuanGroup/Video-LLaVA) | Learning United Visual Representation by Alignment Before Projection. |[arXiv](https://arxiv.org/abs/2311.10122) | | Tool | | [WebGPT](https://github.com/0hq/WebGPT) | Run GPT model on the browser with WebGPU. | | | Tool | | [Web3-GPT](https://github.com/Markeljan/Web3GPT) | Deploy smart contracts with AI | | | Tool | | [WordGPT](https://github.com/filippofinke/WordGPT) | 🤖 Bring the power of ChatGPT to Microsoft Word | | | Tool | | [XAgent](https://github.com/OpenBMB/XAgent) | An Autonomous LLM Agent for Complex Task Solving. | | | Tool | | [Yi](https://github.com/01-ai/Yi) | A series of large language models trained from scratch by developers. | | | Tool | | [01 Project](https://github.com/OpenInterpreter/01) | The open-source language model computer. | | | Tool |

## Game (Agent) | Source | Description | Paper | Game Engine | Type | | :------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-----------: | :-----------: | :-------: | | [AgentBench](https://github.com/thudm/agentbench) | A Comprehensive Benchmark to Evaluate LLMs as Agents. |[arXiv](https://arxiv.org/abs/2308.03688) | | Agent | | [Agent Group Chat](https://github.com/MikeGu721/AgentGroup) | An Interactive Group Chat Simulacra For Better Eliciting Collective Emergent Behavior. |[arXiv](https://arxiv.org/abs/2403.13433) | | Agent | | [Agent K](https://github.com/mikekelly/AgentK) | An autoagentic AGI that is self-evolving and modular. | | | Agent | | [Agent Laboratory](https://github.com/SamuelSchmidgall/AgentLaboratory) | Agent Laboratory: Using LLM Agents as Research Assistants. |[arXiv](https://arxiv.org/abs/2501.04227) | | Agent | | [AgentScope](https://github.com/modelscope/agentscope) | Start building LLM-empowered multi-agent applications in an easier way. |[arXiv](https://arxiv.org/abs/2402.14034) | | Agent | | [AgentSims](https://github.com/py499372727/AgentSims/) | An Open-Source Sandbox for Large Language Model Evaluation. | | | Agent | | [AI Town](https://github.com/a16z-infra/ai-town) | AI Town is a virtual town where AI characters live, chat and socialize. | | | Agent | | [anime.gf](https://github.com/cyanff/anime.gf) | Local & Open Source Alternative to CharacterAI. | | | Game | | [Astrocade](https://www.astrocade.com/) | Create games with AI | | | Game | | [Atomic Agents](https://github.com/KennyVaneetvelde/atomic_agents) | The Atomic Agents framework is designed to be modular, extensible, and easy to use. | | | Agent | | [AutoAgents](https://github.com/Link-AGI/AutoAgents) | A Framework for Automatic Agent Generation. | | | Agent | | [AutoGen](https://github.com/microsoft/autogen) | Enable Next-Gen Large Language Model Applications. |[arXiv](https://arxiv.org/abs/2308.08155) | | Agent | | [behaviac](https://github.com/Tencent/behaviac) | Behaviac is a framework of the game AI development. | | | Framework | | [Biomes](https://github.com/ill-inc/biomes-game) | Biomes is an open source sandbox MMORPG built for the web using web technologies such as Next.js, Typescript, React and WebAssembly. | | | Game | | [Buffer of Thoughts](https://github.com/YangLing0818/buffer-of-thought-llm) | Thought-Augmented Reasoning with Large Language Models. |[arXiv](https://arxiv.org/abs/2406.04271) | | Agent | | [Byzer-Agent](https://github.com/allwefantasy/byzer-agent) | Easy, fast, and distributed agent framework for everyone. | | | Agent | | [Cat Town](https://github.com/ykhli/cat-town) | A C(h)atGPT-powered simulation with cats. | | | Agent | | [Cat Town](https://github.com/ykhli/cat-town) | A C(h)atGPT-powered simulation with cats. | | | Agent | | [CharacterGLM](https://github.com/thu-coai/CharacterGLM-6B) | Customizing Chinese Conversational AI Characters with Large Language Models. |[arXiv](https://arxiv.org/abs/2311.16832) | | Agent | | [ChatDev](https://github.com/OpenBMB/ChatDev) | Communicative Agents for Software Development. |[arXiv](https://arxiv.org/abs/2405.04219) | | Agent | | [CogAgent](https://modelscope.cn/models/ZhipuAI/cogagent-chat/summary) | CogAgent is an open-source visual language model improved based on CogVLM. |[arXiv](https://arxiv.org/abs/2312.08914) | | Agent | | [Cradle](https://github.com/BAAI-Agents/Cradle) | Towards General Computer Control. | | | Agent | | [crewAI](https://github.com/joaomdmoura/crewAI) | Framework for orchestrating role-playing, autonomous AI agents. | | | Agent | | [Dify](https://github.com/langgenius/dify) | Dify is an open-source LLM app building platform. | | | Agent | | [Digital Life Project](https://digital-life-project.com/) | Autonomous 3D Characters with Social Intelligence. |[arXiv](https://arxiv.org/abs/2312.04547) | | Agent | | [everything-ai](https://github.com/AstraBert/everything-ai) | Your fully proficient, AI-powered and local chatbot assistant🤖. | | | Agent | | [fabric](https://github.com/danielmiessler/fabric) | fabric is an open-source framework for augmenting humans using AI. | | | Agent | | [FastGPT](https://github.com/labring/FastGPT) | FastGPT is a knowledge-based platform built on the LLM. | | | Agent | | [fastRAG](https://github.com/IntelLabs/fastRAG) | Efficient Retrieval Augmentation and Generation Framework. | | | Agent | | [GameAISDK](https://github.com/Tencent/GameAISDK) | Image-based game AI automation framework. | | | Framework | | [GameNGen](https://gamengen.github.io/) | Diffusion Models Are Real-Time Game Engines. |[arXiv](https://arxiv.org/abs/2408.14837) | | Game | | [GameGen-O](https://github.com/GameGen-O/GameGen-O) | GameGen-O: Open-world Video Game Generation. | | | Game | | [GenAgent](https://github.com/xxyQwQ/GenAgent) | GenAgent: Build Collaborative AI Systems with Automated Workflow Generation - Case Studies on ComfyUI. |[arXiv](https://arxiv.org/abs/2409.01392) | | Agent | | [Generative Agents](https://github.com/joonspk-research/generative_agents) | Interactive Simulacra of Human Behavior. |[arXiv](https://arxiv.org/abs/2304.03442) | | Agent | | [Genesis](https://github.com/Genesis-Embodied-AI/Genesis) | Genesis: A Generative and Universal Physics Engine for Robotics and Beyond. | | | Game | | [Genie](https://sites.google.com/view/genie-2024/home) | Generative Interactive Environments. | | | Game | | [gigax](https://github.com/GigaxGames/gigax) | Runtime, LLM-powered NPCs. | | | Game | | [HippoRAG](https://github.com/OSU-NLP-Group/HippoRAG) | Neurobiologically Inspired Long-Term Memory for Large Language Models. |[arXiv](https://arxiv.org/abs/2405.14831) | | Agent | | [Interactive LLM Powered NPCs](https://github.com/AkshitIreddy/Interactive-LLM-Powered-NPCs) | Interactive LLM Powered NPCs, is an open-source project that completely transforms your interaction with non-player characters (NPCs) in any game! | | | Game | | [IoA](https://github.com/OpenBMB/IoA) | An open-source framework for collaborative AI agents, enabling diverse, distributed agents to team up and tackle complex tasks through internet-like connectivity. | | | Agent | | [KwaiAgents](https://github.com/KwaiKEG/KwaiAgents) | A generalized information-seeking agent system with Large Language Models (LLMs). |[arXiv](https://arxiv.org/abs/2312.04889) | | Agent | | [LangChain](https://github.com/langchain-ai/langchain) | Get your LLM application from prototype to production. | | | Agent | | [Langflow](https://github.com/logspace-ai/langflow) | Langflow is a UI for LangChain, designed with react-flow to provide an effortless way to experiment and prototype flows. | | | Agent | | [LangGraph Studio](https://github.com/langchain-ai/langgraph-studio) | LangGraph Studio offers a new way to develop LLM applications by providing a specialized agent IDE that enables visualization, interaction, and debugging of complex agentic applications. | | | Agent | | [LARP](https://github.com/MiAO-AI-Lab/LARP) | Language-Agent Role Play for open-world games. |[arXiv](https://arxiv.org/abs/2312.17653) | | Agent | | [LLama Agentic System](https://github.com/meta-llama/llama-agentic-system) | Agentic components of the Llama Stack APIs. | | | Agent | | [LlamaIndex](https://github.com/run-llama/llama_index) | LlamaIndex is a data framework for your LLM application. | | | Agent | | [MindSearch](https://github.com/InternLM/MindSearch) | 🔍 An LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT). | | | Agent | | [Mixture of Agents (MoA)](https://github.com/togethercomputer/MoA) | Mixture-of-Agents Enhances Large Language Model Capabilities. |[arXiv](https://arxiv.org/abs/2406.04692) | | Agent | | [MMRole](https://github.com/YanqiDai/MMRole) | MMRole: A Comprehensive Framework for Developing and Evaluating Multimodal Role-Playing Agents. |[arXiv](https://arxiv.org/abs/2408.04203v1) | | Agent | | [Moonlander.ai](https://www.moonlander.ai/) | Start building 3D games without any coding using generative AI. | | | Framework | | [MuG Diffusion](https://github.com/Keytoyze/Mug-Diffusion) | MuG Diffusion is a charting AI for rhythm games based on Stable Diffusion (one of the most powerful AIGC models) with a large modification to incorporate audio waves. | | | Game | | [Oasis](https://github.com/etched-ai/open-oasis) | Oasis is an interactive world model developed by Decart and Etched. Based on diffusion transformers, Oasis takes in user keyboard input and generates gameplay in an autoregressive manner. | | | Game | | [OmAgent](https://github.com/om-ai-lab/OmAgent) | A multimodal agent framework for solving complex tasks. | | | Agent | | [OpenAgents](https://github.com/xlang-ai/OpenAgents) | An Open Platform for Language Agents in the Wild. | | | Agent | | [Opus](https://opus.ai/) | An AI app that turns text into a video game. | | | Game | | [Pipecat](https://github.com/pipecat-ai/pipecat) | Open Source framework for voice and multimodal conversational AI. | | | Agent | | [Qwen-Agent](https://github.com/QwenLM/Qwen-Agent) | Qwen-Agent is a framework for developing LLM applications based on the instruction following, tool usage, planning, and memory capabilities of Qwen. | | | Agent | | [Ragas](https://github.com/explodinggradients/ragas) | Ragas is a framework that helps you evaluate your Retrieval Augmented Generation (RAG) pipelines. | | | Agent | | [RPBench-Auto](https://github.com/boson-ai/RPBench-Auto) | An automated pipeline for evaluating LLMs for role-playing. | | | Game | | [SIMA](https://deepmind.google/discover/blog/sima-generalist-ai-agent-for-3d-virtual-environments/) | A generalist AI agent for 3D virtual environments. | | | Agent | | [StoryGames.ai](https://storygames.buildbox.com/) | AI for Dreamers Make Games. | | | Game | | [SWE-agent](https://github.com/princeton-nlp/SWE-agent) | Agent Computer Interfaces Enable Software Engineering Language Models. |[arXiv](https://arxiv.org/abs/2405.15793) | | Agent | | [TaskGen](https://github.com/simbianai/taskgen) | A Task-based agentic framework building on StrictJSON outputs by LLM agents. | | | Agent | | [TEN Agent](https://github.com/TEN-framework/TEN-Agent) | TEN Agent is the world’s first real-time multimodal agent integrated with the OpenAI Realtime API, RTC, and features weather checks, web search, vision, and RAG capabilities. | | | Agent | | [Translation Agent](https://github.com/andrewyng/translation-agent) | Agentic translation using reflection workflow. | | | Agent | | [Twitter](https://github.com/wordware-ai/twitter) | Twitter Personality is a web application that analyzes your Twitter handle to create a personalized personality profile using Wordware AI Agent. | | | Agent | | [Unbounded](https://generative-infinite-game.github.io/) | Unbounded: A Generative Infinite Game of Character Life Simulation. |[arXiv](https://arxiv.org/abs/2410.18975) | | Game | | [Video2Game](https://github.com/video2game/video2game) | Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Video. |[arXiv](https://arxiv.org/abs/2404.09833) | | Game | | [V-IRL](https://virl-platform.github.io/) | Grounding Virtual Intelligence in Real Life. |[arXiv](https://arxiv.org/abs/2402.03310) | | Agent | | [WebDesignAgent](https://github.com/DAMO-NLP-SG/WebDesignAgent) | An agent used for webdesign. | | | Agent | | [XAgent](https://github.com/OpenBMB/XAgent) | An Autonomous LLM Agent for Complex Task Solving. | | | Agent |

^ Back to Contents ^

^ Back to Contents ^

## Writer | Source | Description | Paper | Game Engine | Type | | :------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-----------: | :-----------: | :-------: | | [AI-Writer](https://github.com/BlinkDL/AI-Writer) | AI writes novels, generates fantasy and romance web articles, etc. Chinese pre-trained generative model. | | | Writer | | [Notebook.ai](https://github.com/indentlabs/notebook) | Notebook.ai is a set of tools for writers, game designers, and roleplayers to create magnificent universes – and everything within them. | | | Writer | | [Novel](https://github.com/steven-tey/novel) | Notion-style WYSIWYG editor with AI-powered autocompletions. | | | Writer | | [NovelAI](https://novelai.net/) | Driven by AI, painlessly construct unique stories, thrilling tales, seductive romances, or just fool around. | | | Writer |

^ Back to Contents ^

## Image | Source | Description | Paper | Game Engine | Type | | :------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-----------: | :-----------: | :-------: | | [AnyDoor](https://ali-vilab.github.io/AnyDoor-Page/) | Zero-shot Object-level Image Customization. |[arXiv](https://arxiv.org/abs/2307.09481) | | Image | | [AnyText](https://github.com/tyxsspa/AnyText) | Multilingual Visual Text Generation And Editing. |[arXiv](https://arxiv.org/abs/2311.03054) | | Image | | [AutoStudio](https://github.com/donahowe/AutoStudio) | Crafting Consistent Subjects in Multi-turn Interactive Image Generation. |[arXiv](https://arxiv.org/abs/2406.01388) | | Image | | [Blender-ControlNet](https://github.com/coolzilj/Blender-ControlNet) | Using ControlNet right in Blender. | | Blender | Image | | [BriVL](https://github.com/BAAI-WuDao/BriVL) | Bridging Vision and Language Model. |[arXiv](https://arxiv.org/abs/2103.06561) | | Image | | [CatVTON](https://github.com/Zheng-Chong/CatVTON) | CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Models. |[arXiv](https://arxiv.org/abs/2407.15886) | | Image | | [CLIPasso](https://github.com/yael-vinker/CLIPasso) | A method for converting an image of an object to a sketch, allowing for varying levels of abstraction. |[arXiv](https://arxiv.org/abs/2202.05822) | | Image | | [ClipDrop](https://clipdrop.co/) | Create stunning visuals in seconds. | | | Image | | [ComfyUI](https://github.com/comfyanonymous/ComfyUI) | A powerful and modular stable diffusion GUI with a graph/nodes interface. | | | Image | | [ConceptLab](https://github.com/kfirgoldberg/ConceptLab) | Creative Generation using Diffusion Prior Constraints. |[arXiv](https://arxiv.org/abs/2308.02669) | | Image | | [ControlNet](https://github.com/lllyasviel/ControlNet) | ControlNet is a neural network structure to control diffusion models by adding extra conditions. |[arXiv](https://arxiv.org/abs/2302.05543) | | Image | | [CSGO](https://github.com/instantX-research/CSGO) | CSGO: Content-Style Composition in Text-to-Image Generation. |[arXiv](https://arxiv.org/abs/2408.16766) | | Image | | [DALL·E 2](https://openai.com/product/dall-e-2) | DALL·E 2 is an AI system that can create realistic images and art from a description in natural language. | | | Image | | [Dashtoon Studio](https://www.dashtoon.ai/) | Dashtoon Studio is an AI powered comic creation platform. | | | Comic | | [DeepAI](https://deepai.org/) | DeepAI offers a suite of tools that use AI to enhance your creativity. | | | Image | | [DeepFloyd IF](https://github.com/deep-floyd/IF) | IF by DeepFloyd Lab at StabilityAI. | | | Image | | [Depth Anything V2](https://github.com/DepthAnything/Depth-Anything-V2) | Depth Anything V2 |[arXiv](https://arxiv.org/abs/2406.09414) | | Image | | [Depth map library and poser](https://github.com/jexom/sd-webui-depth-lib) | Depth map library for use with the Control Net extension for Automatic1111/stable-diffusion-webui. | | | Image | | [Diffuse to Choose](https://diffuse2choose.github.io/) | Enriching Image Conditioned Inpainting in Latent Diffusion Models for Virtual Try-All. |[arXiv](https://arxiv.org/abs/2401.13795) | | Image | | [Disco Diffusion](https://github.com/alembics/disco-diffusion) | A frankensteinian amalgamation of notebooks, models and techniques for the generation of AI Art and Animations. | | | Image | | [DragGAN](https://github.com/XingangPan/DragGAN) | Interactive Point-based Manipulation on the Generative Image Manifold. |[arXiv](https://arxiv.org/abs/2305.10973) | | Image | | [Draw Things](https://drawthings.ai/) | AI- assisted image generation in Your Pocket. | | | Image | | [DWPose](https://github.com/idea-research/dwpose) | Effective Whole-body Pose Estimation with Two-stages Distillation. |[arXiv](https://arxiv.org/abs/2307.15880) | | Image | | [EasyPhoto](https://github.com/aigc-apps/sd-webui-EasyPhoto) | Your Smart AI Photo Generator. | | | Image | | [Flux](https://github.com/black-forest-labs/flux) | This repo contains minimal inference code to run text-to-image and image-to-image with our Flux latent rectified flow transformers. | | | Image | | [Follow-Your-Click](https://github.com/mayuelala/FollowYourClick) | Open-domain Regional Image Animation via Short Prompts. |[arXiv](https://arxiv.org/abs/2403.08268) | | Image | | [Fooocus](https://github.com/lllyasviel/Fooocus) | Focus on prompting and generating. | | | Image | | [GIFfusion](https://github.com/DN6/giffusion) | Create GIFs and Videos using Stable Diffusion. | | | Image | | [Grounded-Segment-Anything](https://github.com/IDEA-Research/Grounded-Segment-Anything) | Automatically Detect , Segment and Generate Anything with Image, Text, and Audio Inputs. |[arXiv](https://arxiv.org/abs/2401.14159) | | Image | | [HivisionIDPhotos](https://github.com/Zeyi-Lin/HivisionIDPhotos) | HivisionIDPhotos: a lightweight and efficient AI ID photos tools. | | | Image | | [Hua](https://github.com/BlinkDL/Hua) | Hua is an AI image editor with Stable Diffusion (and more). | | | Image | | [Hunyuan-DiT](https://github.com/Tencent/HunyuanDiT) | A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding. |[arXiv](https://arxiv.org/abs/2405.08748) | | Image | | [IC-Light](https://github.com/lllyasviel/IC-Light) | IC-Light is a project to manipulate the illumination of images. | | | Image | | [Ideogram](https://ideogram.ai/login) | Helping people become more creative. | | | Image | | [Imagen](https://imagen.research.google/) | Imagen is an AI system that creates photorealistic images from input text. | | | Image | | [img2img-turbo](https://github.com/GaParmar/img2img-turbo) | One-Step Image-to-Image with SD-Turbo. | | | Image | | [Img2Prompt](https://www.img2prompt.io/) | Get prompts from stable diffusion generated images. | | | Image | | [Infinity](https://github.com/FoundationVision/Infinity) | Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis. |[arXiv](https://arxiv.org/abs/2412.04431) | | Image | | [InstantID](https://github.com/InstantID/InstantID) | Zero-shot Identity-Preserving Generation in Seconds. |[arXiv](https://arxiv.org/abs/2401.07519) | | Image | | [InternLM-XComposer2](https://github.com/InternLM/InternLM-XComposer) | InternLM-XComposer2 is a groundbreaking vision-language large model (VLLM) excelling in free-form text-image composition and comprehension. |[arXiv](https://arxiv.org/abs/2401.16420) | | Image | | [KOALA](https://youngwanlee.github.io/KOALA/) | Self-Attention Matters in Knowledge Distillation of Latent Diffusion Models for Memory-Efficient and Fast Image Synthesis. | | | Image | | [Kolors](https://github.com/Kwai-Kolors/Kolors) | Kolors: Effective Training of Diffusion Model for Photorealistic Text-to-Image Synthesis. | | | Image | | [Komiko](https://komiko.app/) | Komiko is an AI-powered storytelling platform that lets you create original characters, comics, and animations with ease. | | | Comic | | [KREA](https://www.krea.ai/) | Generate images and videos with a delightful AI-powered design tool. | | | Image | | [LaVi-Bridge](https://github.com/ShihaoZhaoZSH/LaVi-Bridge) | Bridging Different Language Models and Generative Vision Models for Text-to-Image Generation. |[arXiv](https://arxiv.org/abs/2403.07860) | | Image | | [LayerDiffusion](https://github.com/layerdiffusion/LayerDiffusion) | Transparent Image Layer Diffusion using Latent Transparency. |[arXiv](https://arxiv.org/abs/2305.18676) | | Image | | [Lexica](https://lexica.art/) | A Stable Diffusion prompts search engine. | | | Image | | [LlamaGen](https://github.com/FoundationVision/LlamaGen) | Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation. |[arXiv](https://arxiv.org/abs/2406.06525) | | Image | | [Lumina-Image 2.0](https://github.com/Alpha-VLLM/Lumina-Image-2.0) | Lumina-Image 2.0 : A Unified and Efficient Image Generative Model. | | | Image | | [Lumina-mGPT](https://github.com/Alpha-VLLM/Lumina-mGPT) | Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining. |[arXiv](https://arxiv.org/abs/2408.02657) | | Image | | [MakeAnything](https://github.com/showlab/MakeAnything) | MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generation. |[arXiv](https://arxiv.org/abs/2502.01572) | | Image | | [MetaShoot](https://metashoot.vinzi.xyz/) | MetaShoot is a digital twin of a photo studio, developed as a plugin for Unreal Engine that gives any creator the ability to produce highly realistic renders in the easiest and quickest way. | | Unreal Engine | Image | | [Midjourney](https://www.midjourney.com/) | Midjourney is an independent research lab exploring new mediums of thought and expanding the imaginative powers of the human species. | | | Image | | [MIGC](https://github.com/limuloo/MIGC) | MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis. |[arXiv](https://arxiv.org/abs/2402.05408) | | Image | | [MimicBrush](https://github.com/ali-vilab/MimicBrush) | Zero-shot Image Editing with Reference Imitation. |[arXiv](https://arxiv.org/abs/2406.07547) | | Image | | [OmniGen](https://github.com/VectorSpaceLab/OmniGen) | OmniGen: Unified Image Generation. |[arXiv](https://arxiv.org/abs/2409.11340) | | Image | | [Omost](https://github.com/lllyasviel/Omost) | Omost is a project to convert LLM's coding capability to image generation (or more accurately, image composing) capability. | | | Image | | [Openpose Editor](https://github.com/fkunn1326/openpose-editor) | Openpose Editor for AUTOMATIC1111's stable-diffusion-webui. | | | Image | | [Outfit Anyone](https://humanaigc.github.io/outfit-anyone/) | Ultra-high quality virtual try-on for Any Clothing and Any Person. | | | Image | | [PaintsUndo](https://github.com/lllyasviel/Paints-UNDO) | PaintsUndo: A Base Model of Drawing Behaviors in Digital Paintings. | | | Image | | [PhotoMaker](https://photo-maker.github.io/) | Customizing Realistic Human Photos via Stacked ID Embedding. |[arXiv](https://arxiv.org/abs/2312.04461) | | Image | | [Photoroom](https://www.photoroom.com/backgrounds) | AI Background Generator. | | | Image | | [Plask](https://plask.ai/) | AI image generation in the cloud. | | | Image | | [Prompt.Art](https://prompt.art/) | The Generators Hub. | | | Image | | [PuLID](https://github.com/ToTheBeginning/PuLID) | Pure and Lightning ID Customization via Contrastive Alignment. |[arXiv](https://arxiv.org/abs/2404.16022) | | Image | | [Rich-Text-to-Image](https://github.com/SongweiGe/rich-text-to-image) | Expressive Text-to-Image Generation with Rich Text. |[arXiv](https://arxiv.org/abs/2304.06720) | | Image | | [RPG-DiffusionMaster](https://github.com/YangLing0818/RPG-DiffusionMaster) | Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs (PRG). | | | Image | | [SEED-Story](https://github.com/TencentARC/SEED-Story) | SEED-Story: Multimodal Long Story Generation with Large Language Model. |[arXiv](https://arxiv.org/abs/2407.08683) | | Image | | [Segment Anything](https://segment-anything.com/) | Segment Anything Model (SAM): a new AI model from Meta AI that can "cut out" any object , in any image , with a single click. |[arXiv](https://arxiv.org/abs/2304.02643) | | Image | | [Segment Anything Model 2 (SAM 2)](https://github.com/facebookresearch/segment-anything-2) | SAM 2: Segment Anything in Images and Videos. |[arXiv](https://arxiv.org/abs/2408.00714) | | Image | | [sd-webui-controlnet](https://github.com/Mikubill/sd-webui-controlnet) | WebUI extension for ControlNet. | | | Image | | [SDXL-Lightning](https://huggingface.co/ByteDance/SDXL-Lightning) | Progressive Adversarial Diffusion Distillation. |[arXiv](https://arxiv.org/abs/2402.13929) | | Image | | [SDXS](https://github.com/IDKiro/sdxs) | Real-Time One-Step Latent Diffusion Models with Image Conditions. | | | Image | | [Stable.art](https://github.com/isekaidev/stable.art) | Photoshop plugin for Stable Diffusion with Automatic1111 as backend (locally or with Google Colab). | | | Image | | [Stable Cascade](https://github.com/Stability-AI/StableCascade) | Stable Cascade consists of three models: Stage A, Stage B and Stage C, representing a cascade for generating images, hence the name "Stable Cascade". | | | Image | | [Stable Diffusion](https://github.com/CompVis/stable-diffusion) | A latent text-to-image diffusion model. | | | Image | | [stable-diffusion.cpp](https://github.com/leejet/stable-diffusion.cpp) | Stable Diffusion in pure C/C++. | | | Image | | [Stable Diffusion web UI](https://github.com/AUTOMATIC1111/stable-diffusion-webui) | A browser interface based on Gradio library for Stable Diffusion. | | | Image | | [Stable Diffusion web UI](https://github.com/Sygil-Dev/sygil-webui) | Web-based UI for Stable Diffusion. | | | Image | | [Stable Diffusion WebUI Chinese](https://github.com/VinsonLaro/stable-diffusion-webui-chinese) | Chinese version of stable-diffusion-webui. | | | Image | | [Stable Diffusion XL](https://clipdrop.co/stable-diffusion) | Generate images from text. |[arXiv](https://arxiv.org/abs/2307.01952) | | Image | | [Stable Diffusion XL Turbo](https://clipdrop.co/stable-diffusion-turbo) | Real-Time Text-to-Image Generation. | | | Image | | [Stable Diffusion 3.5](https://github.com/Stability-AI/sd3.5) | Stable Diffusion 3.5 open release includes multiple model variants, including Stable Diffusion 3.5 Large and Stable Diffusion 3.5 Large Turbo. | | | Image | | [Stable Doodle](https://clipdrop.co/stable-doodle) | Stable Doodle is a sketch-to-image tool that converts a simple drawing into a dynamic image. | | | Image | | [StableStudio](https://github.com/Stability-AI/StableStudio) | StableStudio by Stability AI | | | Image | | [StoryMaker](https://github.com/RedAIGC/StoryMaker) | StoryMaker: Towards Holistic Consistent Characters in Text-to-image Generation. |[arXiv](https://arxiv.org/abs/2409.12576) | | Image | | [StreamDiffusion](https://github.com/cumulo-autumn/StreamDiffusion) | A Pipeline-Level Solution for Real-Time Interactive Generation. | | | Image | | [StyleDrop](https://styledrop.github.io/) | Text-To-Image Generation in Any Style. |[arXiv](https://arxiv.org/abs/2306.00983) | | Image | | [SyncDreamer](https://github.com/liuyuan-pal/SyncDreamer) | Generating Multiview-consistent Images from a Single-view Image. |[arXiv](https://arxiv.org/abs/2309.03453) | | Image | | [UltraEdit](https://github.com/HaozheZhao/UltraEdit) | UltraEdit: Instruction-based Fine-Grained Image Editing at Scale. |[arXiv](https://arxiv.org/abs/2407.05282) | | Image | | [UltraPixel](https://github.com/catcathh/UltraPixel) | UltraPixel: Advancing Ultra-High-Resolution Image Synthesis to New Peaks. |[arXiv](https://arxiv.org/abs/2407.02158) | | Image | | [Unity ML Stable Diffusion](https://github.com/keijiro/UnityMLStableDiffusion) | Core ML Stable Diffusion on Unity. | | Unity | Image | | [Vispunk Visions](https://vispunk.com/image) | Text-to-Image generation platform. | | | Image |

^ Back to Contents ^

## Texture | Source | Description | Paper | Game Engine | Type | | :------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-----------: | :-----------: | :-------: | | [CRM](https://github.com/thu-ml/CRM) | Single Image to 3D Textured Mesh with Convolutional Reconstruction Model. |[arXiv](https://arxiv.org/abs/2403.05034) | | Texture | | [DreamMat](https://github.com/zzzyuqing/DreamMat) | High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models. |[arXiv](https://arxiv.org/abs/2405.17176) | | Texture | | [DreamSpace](https://github.com/ybbbbt/dreamspace) | Dreaming Your Room Space with Text-Driven Panoramic Texture Propagation. | | | Texture | | [Dream Textures](https://github.com/carson-katri/dream-textures) | Stable Diffusion built-in to Blender. Create textures, concept art, background assets, and more with a simple text prompt. | | Blender | Texture | | [InstructHumans](https://github.com/viridityzhu/InstructHumans) | Editing Animated 3D Human Textures with Instructions. |[arXiv](https://arxiv.org/abs/2404.04037) | | Texture | | [InteX](https://github.com/ashawkey/InTeX) | Interactive Text-to-Texture Synthesis via Unified Depth-aware Inpainting. |[arXiv](https://arxiv.org/abs/2403.11878) | | Texture | | [LLaMA-Mesh](https://github.com/nv-tlabs/LLaMA-Mesh) | LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models. |[arXiv](https://arxiv.org/abs/2411.09595) | | Mesh | | [MaterialSeg3D](https://github.com/PROPHETE-pro/MaterialSeg3D_) | MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D Assets. |[arXiv](https://arxiv.org/abs/2404.13923) | | Texture | | [MeshAnything](https://github.com/buaacyw/MeshAnything) | MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D Assets. |[arXiv](https://arxiv.org/abs/2406.10163) | | Mesh | | [Neuralangelo](https://github.com/NVlabs/neuralangelo) | High-Fidelity Neural Surface Reconstruction. |[arXiv](https://arxiv.org/abs/2306.03092) | | Texture | | [Paint-it](https://github.com/postech-ami/paint-it) | Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering. | | | Texture | | [Polycam](https://poly.cam/material-generator) | Create your own 3D textures just by typing. | | | Texture | | [TexFusion](https://research.nvidia.com/labs/toronto-ai/texfusion/) | Synthesizing 3D Textures with Text-Guided Image Diffusion Models. |[arXiv](https://arxiv.org/abs/2310.13772) | | Texture | | [Text2Tex](https://daveredrum.github.io/Text2Tex/) | Text-driven texture Synthesis via Diffusion Models. |[arXiv](https://arxiv.org/abs/2303.11396) | | Texture | | [Texture Lab](https://www.texturelab.xyz/) | AI-generated texures. You can generate your own with a text prompt. | | | Texture | | [With Poly](https://withpoly.com/browse/textures) | Create Textures With Poly. Generate 3D materials with AI in a free online editor, or search our growing community library. | | | Texture | | [X-Mesh](https://github.com/xmu-xiaoma666/X-Mesh) | X-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance. |[arXiv](https://arxiv.org/abs/2303.15764) | | Texture |

## Animation | Source | Description | Paper | Game Engine | Type | | :------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-----------: | :-----------: | :-------: | | [Animate Anyone](https://github.com/HumanAIGC/AnimateAnyone) | Consistent and Controllable Image-to-Video Synthesis for Character Animation. |[arXiv](https://arxiv.org/abs/2311.17117) | | Animation | | [AnimateAnything](https://animationai.github.io/AnimateAnything/) | Fine-Grained Open Domain Image Animation with Motion Guidance. |[arXiv](https://arxiv.org/abs/2311.12886) | | Animation | | [AnimateDiff](https://github.com/guoyww/animatediff/) | Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning. |[arXiv](https://arxiv.org/abs/2307.04725) | | Animation | | [AnimateLCM](https://github.com/G-U-N/AnimateLCM) | Let's Accelerate the Video Generation within 4 Steps! |[arXiv](https://arxiv.org/abs/2402.00769) | | Animation | | [Animate-X](https://github.com/Lucaria-Academy/Animate-X) | Animate-X: Universal Character Image Animation with Enhanced Motion Representation. |[arXiv](https://arxiv.org/abs/2410.10306) | | Animation | | [AnimateZero](https://vvictoryuki.github.io/animatezero.github.io/) | Video Diffusion Models are Zero-Shot Image Animators. |[arXiv](https://arxiv.org/abs/2312.03793) | | Animation | | [AnimationGPT](https://github.com/fyyakaxyy/AnimationGPT) | An AIGC tool for generating game combat motion assets. | | | Animation | | [Deforum](https://deforum.art/) | Deforum leverages Stable Diffusion to generate evolving AI visuals. | | | Animation | | [DrawingSpinUp](https://github.com/LordLiang/DrawingSpinUp) | DrawingSpinUp: 3D Animation from Single Character Drawings. |[arXiv](https://arxiv.org/abs/2409.08615) | | Animation | | [DreaMoving](https://dreamoving.github.io/dreamoving/) | A Human Video Generation Framework based on Diffusion Models. |[arXiv](https://arxiv.org/abs/2312.05107) | | Animation | | [FaceFusion](https://github.com/facefusion/facefusion) | Next generation face swapper and enhancer. | | | Animation | | [FreeInit](https://tianxingwu.github.io/pages/FreeInit/) | Bridging Initialization Gap in Video Diffusion Models. |[arXiv](https://arxiv.org/abs/2312.07537) | | Animation | | [GeneFace](https://github.com/yerfor/GeneFace) | Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis. |[arXiv](https://arxiv.org/abs/2301.13430) | | Animation | | [ID-Animator](https://github.com/ID-Animator/ID-Animator) | Zero-Shot Identity-Preserving Human Video Generation. |[arXiv](https://arxiv.org/abs/2404.15275) | | Animation | | [MagicAnimate](https://showlab.github.io/magicanimate/) | Temporally Consistent Human Image Animation using Diffusion Model. |[arXiv](https://arxiv.org/abs/2311.16498) | | Animation | | [NUWA](https://msra-nuwa.azurewebsites.net/#/) | DragNUWA is an open-domain diffusion-based video generation model takes text, image, and trajectory controls as inputs to achieve controllable video generation. |[arXiv](https://arxiv.org/abs/2308.08089) | | Animation | | [NUWA-Infinity](https://nuwa-infinity.microsoft.com/#/NUWAInfinity) | NUWA-Infinity is a multimodal generative model that is designed to generate high-quality images and videos from given text, image or video input. | | | Animation | | [NUWA-XL](https://msra-nuwa.azurewebsites.net/#/NUWAXL) | A novel Diffusion over Diffusion architecture for eXtremely Long video generation. | | | Animation | | [Omni Animation](https://omnianimation.ai/) | AI Generated High Fidelity Animations. | | | Animation | | [PIA](https://pi-animator.github.io/) | Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models. |[arXiv](https://arxiv.org/abs/2312.13964) | | Animation | | [SadTalker](https://github.com/Winfredy/SadTalker) | Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation. |[arXiv](https://arxiv.org/abs/2211.12194) | | Animation | | [SadTalker-Video-Lip-Sync](https://github.com/Zz-ww/SadTalker-Video-Lip-Sync) | This project is based on SadTalkers Wav2lip for video lip synthesis. | | | Animation | | [Stable Animation](https://stability.ai/news/stable-animation-sdk) | A powerful text-to-animation tool for developers. | | | Animation | | [TaleCrafter](https://github.com/VideoCrafter/TaleCrafter) | An interactive story visualization tool that support multiple characters. |[arXiv](https://arxiv.org/abs/2305.18247) | | Animation | | [ToonCrafter](https://github.com/ToonCrafter/ToonCrafter) | ToonCrafter: Generative Cartoon Interpolation. |[arXiv](https://arxiv.org/abs/2405.17933v1) | | Animation | | [Wav2Lip](https://github.com/Rudrabha/Wav2Lip) | Accurately Lip-syncing Videos In The Wild. |[arXiv](https://arxiv.org/abs/2008.10010) | | Animation | | [Wonder Studio](https://wonderdynamics.com/) | An AI tool that automatically animates, lights and composes CG characters into a live-action scene. | | | Animation |

^ Back to Contents ^

^ Back to Contents ^

## Video | Source | Description | Paper | Game Engine | Type | | :------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-----------: | :-----------: | :-------: | | [360DVD](https://akaneqwq.github.io/360DVD/) | Controllable Panorama Video Generation with 360-Degree Video Diffusion Model. |[arXiv](https://arxiv.org/abs/2401.06578) | | Video | | [Animate-A-Story](https://github.com/VideoCrafter/Animate-A-Story) | Retrieval-Augmented Video Generation for Telling a Story. |[arXiv](https://arxiv.org/abs/2307.06940) | | Video | | [Anything in Any Scene](https://anythinginanyscene.github.io/) | Photorealistic Video Object Insertion. | | | Video | | [ART•V](https://warranweng.github.io/art.v/) | Auto-Regressive Text-to-Video Generation with Diffusion Models. |[arXiv](https://arxiv.org/abs/2311.18834) | | Video | | [Assistive](https://assistive.chat/product/video) | Meet the generative video platform that brings your ideas to life. | | | Video | | [AtomoVideo](https://atomo-video.github.io/) | High Fidelity Image-to-Video Generation. |[arXiv](https://arxiv.org/abs/2403.01800) | | Video | | [BackgroundRemover](https://github.com/nadermx/backgroundremover) | Background Remover lets you Remove Background from images and video using AI with a simple command line interface that is free and open source. | | | Video | | [Boximator](https://boximator.github.io/) | Generating Rich and Controllable Motions for Video Synthesis. |[arXiv](https://arxiv.org/abs/2402.01566) | | Video | | [CoDeF](https://github.com/qiuyu96/codef) | Content Deformation Fields for Temporally Consistent Video Processing. |[arXiv](https://arxiv.org/abs/2308.07926) | | Video | | [CogVideo](https://models.aminer.cn/cogvideo/) | Generate Videos from Text Descriptions. | | | Video | | [CogVideoX](https://github.com/THUDM/CogVideo) | CogVideoX is an open-source version of the video generation model, which is homologous to 清影. | | | Video | | [CogVLM](https://github.com/THUDM/CogVLM) | CogVLM is a powerful open-source visual language model (VLM). | | | Visual | | [CoNR](https://github.com/megvii-research/CoNR) | Genarate vivid dancing videos from hand-drawn anime character sheets(ACS). |[arXiv](https://arxiv.org/abs/2207.05378) | | Video | | [Decohere](https://www.decohere.ai/) | Create what can't be filmed. | | | Video | | [Descript](https://www.descript.com/) | Descript is the simple, powerful , and fun way to edit. | | | Video | | [Diffutoon](https://github.com/modelscope/DiffSynth-Studio) | High-Resolution Editable Toon Shading via Diffusion Models. |[arXiv](https://arxiv.org/abs/2401.16224) | | Video | | [dolphin](https://github.com/kaleido-lab/dolphin) | General video interaction platform based on LLMs. | | | Video | | [DomoAI](https://domoai.app/) | Amplify Your Creativity with DomoAI. | | | Video | | [DreamCinema](https://github.com/chen-wl20/DreamCinema) | DreamCinema: Cinematic Transfer with Free Camera and 3D Character. |[arXiv](https://www.arxiv.org/abs/2408.12601) | | Video | | [DynamiCrafter](https://doubiiu.github.io/projects/DynamiCrafter/) | Animating Open-domain Images with Video Diffusion Priors. |[arXiv](https://arxiv.org/abs/2310.12190) | | Video | | [EDGE](https://github.com/Stanford-TML/EDGE) | We introduce EDGE, a powerful method for editable dance generation that is capable of creating realistic, physically-plausible dances while remaining faithful to arbitrary input music. |[arXiv](https://arxiv.org/abs/2211.10658) | | Video | | [EMO](https://humanaigc.github.io/emote-portrait-alive/) | Emote Portrait Alive - Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions. |[arXiv](https://arxiv.org/abs/2402.17485) | | Video | | [Emu Video](https://emu-video.metademolab.com/) | Factorizing Text-to-Video Generation by Explicit Image Conditioning. | | | Video | | [Etna](https://etna.7volcanoes.com/) | Etna can generate corresponding video content based on short text descriptions. | | | Video | | [Fairy](https://fairy-video2video.github.io/) | Fast Parallelized Instruction-Guided Video-to-Video Synthesis. | | | Video | | [Follow-Your-Canvas](https://github.com/mayuelala/FollowYourCanvas) | Follow-Your-Canvas: Higher-Resolution Video Outpainting with Extensive Content Generation. |[arXiv](https://arxiv.org/abs/2409.01055) | | Video | | [Follow Your Pose](https://follow-your-pose.github.io/) | Pose-Guided Text-to-Video Generation using Pose-Free Videos. |[arXiv](https://arxiv.org/abs/2304.01186) | | Video | | [FullJourney](https://www.fulljourney.ai/) | Your complete suite of AI Creation tools at your fingertips. | | | Video | | [Gen-2](https://research.runwayml.com/gen2) | A multi-modal AI system that can generate novel videos with text, images, or video clips. | | | Video | | [Generative Dynamics](https://generative-dynamics.github.io/) | Generative Image Dynamics. | | | Video | | [Genie](https://sites.google.com/view/genie-2024/home) | Generative Interactive Environments. |[arXiv](https://arxiv.org/abs/2402.15391) | | Video | | [Genmo](https://www.genmo.ai/create/video) | Magically make videos with AI. | | | Video | | [GenTron](https://www.shoufachen.com/gentron_website/) | Diffusion Transformers for Image and Video Generation. | | | Video | | [HiGen](https://higen-t2v.github.io/) | Hierarchical Spatio-temporal Decoupling for Text-to-Video generation. | | | Video | | [Hotshot-XL](https://github.com/hotshotco/Hotshot-XL) | Hotshot-XL is an AI text-to-GIF model trained to work alongside Stable Diffusion XL. | | | Video | | [HunyuanVideo](https://github.com/Tencent/HunyuanVideo) | HunyuanVideo: A Systematic Framework For Large Video Generation Model. |[arXiv](https://arxiv.org/abs/2412.03603) | | Video | | [Imagen Video](https://imagen.research.google/video/) | Given a text prompt, Imagen Video generates high definition videos using a base video generation model and a sequence of interleaved spatial and temporal video super-resolution models. | | | Video | | [InstructVideo](https://instructvideo.github.io/) | Instructing Video Diffusion Models with Human Feedback. |[arXiv](https://arxiv.org/abs/2312.12490) | | Video | | [I2VGen-XL](https://i2vgen-xl.github.io/) | High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models. |[arXiv](https://arxiv.org/abs/2311.04145) | | Video | | [LaVie](https://vchitect.github.io/LaVie-project/) | High-Quality Video Generation with Cascaded Latent Diffusion Models. |[arXiv](https://arxiv.org/abs/2309.15103) | | Video | | [LTX Studio](https://ltx.studio/) | LTX Studio is a holistic, AI-driven filmmaking platform for creators, marketers, filmmakers and studios. | | | Video | | [LTX-Video](https://github.com/Lightricks/LTX-Video) | LTX-Video is the first DiT-based video generation model that can generate high-quality videos in real-time. It can generate 24 FPS videos at 768x512 resolution, faster than it takes to watch them. | | | Video | | [Lumiere](https://lumiere-video.github.io/) | A Space-Time Diffusion Model for Video Generation. |[arXiv](https://arxiv.org/abs/2401.12945) | | Video | | [LVDM](https://yingqinghe.github.io/LVDM/) | Latent Video Diffusion Models for High-Fidelity Long Video Generation. |[arXiv](https://arxiv.org/abs/2211.13221) | | Video | | [MagicVideo](https://magicvideo.github.io/) | Efficient Video Generation With Latent Diffusion Models. |[arXiv](https://arxiv.org/abs/2211.11018) | | Video | | [MagicVideo-V2](https://magicvideov2.github.io/) | Multi-Stage High-Aesthetic Video Generation. |[arXiv](https://arxiv.org/abs/2401.04468) | | Video | | [Magic Hour](https://magichour.ai/) | AI Video for Creators made simple. | | | Video | | [MAGVIT-v2](https://magvit.cs.cmu.edu/v2/) | Tokenizer is key to visual generation. | | | Video | | [MAGVIT](https://magvit.cs.cmu.edu/) | Masked Generative Video Transformer. | | | Video | | [Make-A-Video](https://makeavideo.studio/) | Make-A-Video is a state-of-the-art AI system that generates videos from text. |[arXiv](https://arxiv.org/abs/2209.14792) | | Video | | [Make Pixels Dance](https://makepixelsdance.github.io/) | High-Dynamic Video Generation. |[arXiv](https://arxiv.org/abs/2311.10982) | | Video | | [Make-Your-Video](https://doubiiu.github.io/projects/Make-Your-Video/) | Customized Video Generation Using Textual and Structural Guidance. |[arXiv](https://arxiv.org/abs/2306.00943) | | Video | | [MicroCinema](https://wangyanhui666.github.io/MicroCinema.github.io/) | A Divide-and-Conquer Approach for Text-to-Video Generation. |[arXiv](https://arxiv.org/abs/2311.18829) | | Video | | [MIMO](https://github.com/menyifang/MIMO) | MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modeling. |[arXiv](https://arxiv.org/abs/2409.16160) | | Video | | [Mini-Gemini](https://github.com/dvlab-research/MiniGemini) | Mining the Potential of Multi-modality Vision Language Models. | | | Vision | | [MobileVidFactory](https://arxiv.org/abs/2307.16371) | Automatic Diffusion-Based Social Media Video Generation for Mobile Devices from Text. | | | Video | | [Mochi 1](https://github.com/genmoai/models) | Mochi 1 is an open state-of-the-art video generation model with high-fidelity motion and strong prompt adherence in preliminary evaluation. | | | Video | | [MOFA-Video](https://github.com/MyNiuuu/MOFA-Video) | Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model. |[arXiv](https://arxiv.org/abs/2405.20222) | | Video | | [MoneyPrinterTurbo](https://github.com/harry0703/MoneyPrinterTurbo) | Use large models to generate short videos with one click. | | | Video | | [Moonvalley](https://moonvalley.ai/) | Moonvalley is a groundbreaking new text-to-video generative AI model. | | | Video | | [Mora](https://github.com/lichao-sun/Mora) | More like Sora for Generalist Video Generation. |[arXiv](https://arxiv.org/abs/2403.13248) | | Video | | [Morph Studio](https://www.morphstudio.com/) | With our Text-to-Video AI Magic, manifest your creativity through your prompt. | | | Video | | [MotionClone](https://github.com/Bujiazi/MotionClone) | MotionClone: Training-Free Motion Cloning for Controllable Video Generation. |[arXiv](https://arxiv.org/abs/2406.05338) | | Video | | [MotionCtrl](https://wzhouxiff.github.io/projects/MotionCtrl/) | A Unified and Flexible Motion Controller for Video Generation. |[arXiv](https://arxiv.org/abs/2312.03641) | | Video | | [MotionDirector](https://github.com/showlab/MotionDirector) | Motion Customization of Text-to-Video Diffusion Models. |[arXiv](https://arxiv.org/abs/2310.08465) | | Video | | [Motionshop](https://aigc3d.github.io/motionshop/) | An application of replacing the characters in video with 3D avatars. | | | Video | | [Mov2mov](https://github.com/Scholar01/sd-webui-mov2mov) | Mov2mov plugin for Automatic1111/stable-diffusion-webui. | | | Video | | [MovieFactory](https://arxiv.org/abs/2306.07257) | Automatic Movie Creation from Text using Large Generative Models for Language and Images. |[arXiv](https://arxiv.org/abs/2306.07257) | | Video | | [Neural Frames](https://www.neuralframes.com/) | Discover the synthesizer for the visual world. | | | Video | | [NeverEnds](https://neverends.life/) | Create your world. | | | Video | | [Open-Sora](https://github.com/hpcaitech/Open-Sora) | Democratizing Efficient Video Production for All. | | | Video | | [Open-Sora](https://github.com/PKU-YuanGroup/Open-Sora-Plan) | Open-Sora Plan. | | | Video | | [Phenaki](https://phenaki.video/) | A model for generating videos from text, with prompts that can change over time, and videos that can be as long as multiple minutes. |[arXiv](https://arxiv.org/abs/2210.02399) | | Video | | [Pika Labs](https://www.pika.art/) | Pika Labs is revolutionizing video-making experience with AI. | | | Video | | [Pixeling](https://hidream.ai/#/Pixeling) | Pixeling empowers our customers to create highly precise, ultra-realistic, and extremely controllable visual content including images, videos and 3D models. | | | Video | | [PixVerse](https://app.pixverse.ai) | Create breath-taking videos with AI. | | | Video | | [Pollinations](https://pollinations.ai/c/Video) | Creating gets easy, fast, and fun. | | | Video | | [Reuse and Diffuse](https://anonymous0x233.github.io/ReuseAndDiffuse/) | Iterative Denoising for Text-to-Video Generation. |[arXiv](https://arxiv.org/abs/2309.03549) | | Video | | [Ruyi](https://github.com/IamCreateAI/Ruyi-Models) | Ruyi is an image-to-video model capable of generating cinematic-quality videos at a resolution of 768, with a frame rate of 24 frames per second, totaling 5 seconds and 120 frames. | | | Video | | [ShortGPT](https://github.com/RayVentura/ShortGPT) | An experimental AI framework for automated short/video content creation. | | | Video | | [Show-1](https://showlab.github.io/Show-1/) | Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation. |[arXiv](https://arxiv.org/abs/2309.15818) | | Video | | [Step-Video-T2V](https://github.com/stepfun-ai/Step-Video-T2V) | Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model. |[arXiv](https://arxiv.org/abs/2502.10248) | | Video | | [SkyReels-A1](https://github.com/SkyworkAI/SkyReels-A1) | SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformers. |[arXiv](https://arxiv.org/abs/2502.10841) | | Video | | [SkyReels-V1](https://github.com/SkyworkAI/SkyReels-V1) | SkyReels V1: Human-Centric Video Foundation Model. | | | Video | | [Snap Video](https://snap-research.github.io/snapvideo/) | Scaled Spatiotemporal Transformers for Text-to-Video Synthesis. |[arXiv](https://arxiv.org/abs/2402.14797) | | Video | | [Sora](https://openai.com/sora) | Creating video from text. | | | Video | | [SoraWebui](https://github.com/SoraWebui/SoraWebui) | SoraWebui is an open-source Sora web client, enabling users to easily create videos from text with OpenAI's Sora model. | | | Video | | [StableVideo](https://github.com/rese1f/stablevideo) | Text-driven Consistency-aware Diffusion Video Editing. | | | Video | | [Stable Video Diffusion](https://github.com/Stability-AI/generative-models) | Stable Video Diffusion (SVD) Image-to-Video. | | | Video | | [StoryDiffusion](https://github.com/HVision-NKU/StoryDiffusion) | Consistent Self-Attention for Long-Range Image and Video Generation. |[arXiv](https://arxiv.org/abs/2405.01434) | | Video | | [StreamingT2V](https://github.com/Picsart-AI-Research/StreamingT2V) | Consistent, Dynamic, and Extendable Long Video Generation from Text. |[arXiv](https://arxiv.org/abs/2403.14773) | | Video | | [StyleCrafter](https://gongyeliu.github.io/StyleCrafter.github.io/) | nhancing Stylized Text-to-Video Generation with Style Adapter. |[arXiv](https://arxiv.org/abs/2312.00330) | | Video | | [TATS](https://songweige.github.io/projects/tats/index.html) | Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer. | | | Video | | [Text2Video-Zero](https://github.com/Picsart-AI-Research/Text2Video-Zero) | Text-to-Image Diffusion Models are Zero-Shot Video Generators. |[arXiv](https://arxiv.org/abs/2303.13439) | | Video | | [TF-T2V](https://tf-t2v.github.io/) | A Recipe for Scaling up Text-to-Video Generation with Text-free Videos. |[arXiv](https://arxiv.org/abs/2312.15770) | | Video | | [Tora](https://github.com/ali-videoai/Tora) | Tora: Trajectory-oriented Diffusion Transformer for Video Generation. |[arXiv](https://arxiv.org/abs/2407.21705) | | Video | | [Track-Anything](https://github.com/gaomingqi/Track-Anything) | Track-Anything is a flexible and interactive tool for video object tracking and segmentation, based on Segment Anything and XMem. |[arXiv](https://arxiv.org/abs/2304.11968) | | Video | | [Tune-A-Video](https://github.com/showlab/Tune-A-Video) | One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation. |[arXiv](https://arxiv.org/abs/2212.11565) | | Video | | [TwelveLabs](https://www.twelvelabs.io/) | Multimodal AI that understands videos like humans. | | | Video | | [UniVG](https://univg-baidu.github.io/) | Towards UNIfied-modal Video Generation. | | | Video | | [Vchitect-2.0](https://github.com/Vchitect/Vchitect-2.0) | Vchitect-2.0: Parallel Transformer for Scaling Up Video Diffusion Models. | | | Video | | [VGen](https://github.com/ali-vilab/i2vgen-xl) | A holistic video generation ecosystem for video generation building on diffusion models. |[arXiv](https://arxiv.org/abs/2311.04145) | | Video | | [ViewCrafter](https://github.com/Drexubery/ViewCrafter) | ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis. |[arXiv](https://arxiv.org/abs/2409.02048) | | Video | | [Video-ChatGPT](https://github.com/mbzuai-oryx/Video-ChatGPT) | Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. |[arXiv](https://arxiv.org/abs/2306.05424) | | Video | | [VideoComposer](https://videocomposer.github.io/) | Compositional Video Synthesis with Motion Controllability. |[arXiv](https://arxiv.org/abs/2306.02018) | | Video | | [VideoCrafter1](https://arxiv.org/abs/2310.19512) | Open Diffusion Models for High-Quality Video Generation. |[arXiv](https://arxiv.org/abs/2310.19512) | | Video | | [VideoCrafter2](https://ailab-cvc.github.io/videocrafter2/) | Overcoming Data Limitations for High-Quality Video Diffusion Models. |[arXiv](https://arxiv.org/abs/2401.09047) | | Video | | [VideoDrafter](https://videodrafter.github.io/) | Content-Consistent Multi-Scene Video Generation with LLM. |[arXiv](https://arxiv.org/abs/2401.01256) | | Video | | [VideoElevator](https://github.com/YBYBZhang/VideoElevator) | Elevating Video Generation Quality with Versatile Text-to-Image Diffusion Models. |[arXiv](https://arxiv.org/abs/2403.05438) | | Video | | [VideoFactory](https://arxiv.org/abs/2305.10874) | Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation. | | | Video | | [VideoGen](https://videogen.github.io/VideoGen/) | A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation. |[arXiv](https://arxiv.org/abs/2309.00398) | | Video | | [VideoLCM](https://arxiv.org/abs/2312.09109) | Video Latent Consistency Model. |[arXiv](https://arxiv.org/abs/2312.09109) | | Video | | [Video LDMs](https://research.nvidia.com/labs/toronto-ai/VideoLDM/) | Align your Latents: High- resolution Video Synthesis with Latent Diffusion Models. |[arXiv](https://arxiv.org/abs/2304.08818) | | Video | | [Video-LLaVA](https://github.com/PKU-YuanGroup/Video-LLaVA) | Learning United Visual Representation by Alignment Before Projection. |[arXiv](https://arxiv.org/abs/2311.10122) | | Video | | [VideoMamba](https://github.com/OpenGVLab/VideoMamba) | State Space Model for Efficient Video Understanding. |[arXiv](https://arxiv.org/abs/2403.06977) | | Video | | [Video-of-Thought](https://github.com/scofield7419/Video-of-Thought) | Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition. | | | Video | | [VideoPoet](https://sites.research.google/videopoet/) | A large language model for zero-shot video generation. |[arXiv](https://arxiv.org/abs/2312.14125) | | Video | | [Vispunk Motion](https://vispunk.com/video) | Create realistic videos using just text. | | | Video | | [VisualRWKV](https://github.com/howard-hou/VisualRWKV) | VisualRWKV is the visual-enhanced version of the RWKV language model, enabling RWKV to handle various visual tasks. | | | Visual | | [V-JEPA](https://github.com/facebookresearch/jepa) | Video Joint Embedding Predictive Architecture. |[arXiv](https://arxiv.org/abs/2404.08471) | | Video | | [W.A.L.T](https://walt-video-diffusion.github.io/) | Photorealistic Video Generation with Diffusion Models. |[arXiv](https://arxiv.org/abs/2312.06662) | | Video | | [Wan2.1](https://github.com/Wan-Video/Wan2.1) | Wan: Open and Advanced Large-Scale Video Generative Models. | | | Video | | [Zeroscope](https://huggingface.co/spaces/fffiloni/zeroscope) | Zeroscope Text-to-Video. | | | Video |

## Speech | Source | Description | Paper | Game Engine | Type | | :------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-----------: | :-----------: | :-------: | | [Applio](https://github.com/IAHispano/Applio) | Ultimate voice cloning tool, meticulously optimized for unrivaled power, modularity, and user-friendly experience. | | | Speech | | [Audyo](https://www.audyo.ai/) | Text in. Audio out. | | | Speech | | [Bark](https://github.com/suno-ai/bark) | Text-Prompted Generative Audio Model. | | | Speech | | [Bert-VITS2](https://github.com/fishaudio/Bert-VITS2) | VITS2 Backbone with multilingual bert. | | | Speech | | [ChatTTS](https://github.com/2noise/ChatTTS) | ChatTTS is a generative speech model for daily dialogue. | | | Speech | | [CLAPSpeech](https://clapspeech.github.io/) | Learning Prosody from Text Context with Contrastive Language-Audio Pre-Training. | [arXiv](https://arxiv.org/abs/2305.10763) | | Speech | | [CosyVoice](https://github.com/FunAudioLLM/CosyVoice) | Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability. | | | Speech | | [DEX-TTS](https://github.com/winddori2002/DEX-TTS) | Diffusion-based EXpressive Text-to-Speech with Style Modeling on Time Variability. | [arXiv](https://arxiv.org/abs/2406.19135) | | Speech | | [EmotiVoice](https://github.com/netease-youdao/EmotiVoice) | A Multi-Voice and Prompt-Controlled TTS Engine. | | | Speech | | [Fliki](https://fliki.ai/) | Turn text into videos with AI voices. | | | Speech | | [GLM-4-Voice](https://github.com/THUDM/GLM-4-Voice) | GLM-4-Voice is an end-to-end voice model launched by Zhipu AI. GLM-4-Voice can directly understand and generate Chinese and English speech, engage in real-time voice conversations, and change attributes such as emotion, intonation, speech rate, and dialect based on user instructions. | | | Speech | | [Glow-TTS](https://github.com/jaywalnut310/glow-tts) | A Generative Flow for Text-to-Speech via Monotonic Alignment Search. | [arXiv](https://arxiv.org/abs/2005.11129) | | Speech | | [GPT-SoVITS](https://github.com/RVC-Boss/GPT-SoVITS) | A Powerful Few-shot Voice Conversion and Text-to-Speech WebUI. | | | Speech | | [LOVO](https://lovo.ai/) | LOVO is the go-to AI Voice Generator & Text to Speech platform for thousands of creators. | | | Speech | | [MahaTTS](https://github.com/dubverse-ai/MahaTTS) | An Open-Source Large Speech Generation Model. | | | Speech | | [Matcha-TTS](https://github.com/shivammehta25/Matcha-TTS) | A fast TTS architecture with conditional flow matching. | [arXiv](https://arxiv.org/abs/2309.03199) | | Speech | | [MeloTTS](https://github.com/myshell-ai/MeloTTS) | High-quality multi-lingual text-to-speech library by MyShell.ai. Support English, Spanish, French, Chinese, Japanese and Korean. | | | Speech | | [MetaVoice-1B](https://github.com/metavoiceio/metavoice-src) | AI for human-level speech intelligence. | | | Speech | | [Narakeet](https://www.narakeet.com/) | Easily Create Voiceovers Using Realistic Text to Speech. | | | Speech | | [Mini-Omni](https://github.com/gpt-omni/mini-omni) | Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming. Mini-Omni is an open-source multimodel large language model that can hear, talk while thinking. Featuring real-time end-to-end speech input and streaming audio output conversational capabilities. | [arXiv](https://arxiv.org/abs/2408.16725) | | Speech | | [One-Shot-Voice-Cloning](https://github.com/CMsmartvoice/One-Shot-Voice-Cloning) | One Shot Voice Cloning base on Unet-TTS. | | | Speech | | [OpenVoice](https://github.com/myshell-ai/OpenVoice) | Instant voice cloning by MyShell. | | | Speech | | [OverFlow](https://github.com/shivammehta25/OverFlow) | Putting flows on top of neural transducers for better TTS. | | | Speech | | [RealtimeTTS](https://github.com/KoljaB/RealtimeTTS) | RealtimeTTS is a state-of-the-art text-to-speech (TTS) library designed for real-time applications. | | | Speech | | [SenseVoice](https://github.com/FunAudioLLM/SenseVoice) | SenseVoice is a speech foundation model with multiple speech understanding capabilities, including automatic speech recognition (ASR), spoken language identification (LID), speech emotion recognition (SER), and audio event detection (AED). | | | Speech | | [SpeechGPT](https://github.com/0nutation/SpeechGPT) | Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities. | [arXiv](https://arxiv.org/abs/2305.11000) | | Speech | | [speech-to-text-gpt3-unity](https://github.com/dr-iskandar/speech-to-text-gpt3-unity) | This is the repo I use Whisper and ChatGPT API from OpenAI in Unity. | | Unity | Speech | | [Stable Speech](https://github.com/sanchit-gandhi/stable-speech) | Stability AI's Text-to-Speech model. | | | Speech | | [StableTTS](https://github.com/KdaiP/StableTTS) | Next-generation TTS model using flow-matching and DiT, inspired by Stable Diffusion 3. | | | Speech | | [Step-Audio](https://github.com/stepfun-ai/Step-Audio) | Step-Audio: Unified Understanding and Generation in Intelligent Speech Interaction. | [arXiv](https://arxiv.org/abs/2502.11946) | | Speech | | [StyleTTS 2](https://github.com/yl4579/StyleTTS2) | Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models. | [arXiv](https://arxiv.org/abs/2306.07691) | | Speech | | [tortoise.cpp](https://github.com/balisujohn/tortoise.cpp) | tortoise.cpp: GGML implementation of tortoise-tts. | | | Speech | | [TorToiSe-TTS](https://github.com/neonbjb/tortoise-tts) | A multi-voice TTS system trained with an emphasis on quality. | | | Speech | | [TTS Generation WebUI](https://github.com/rsxdalv/tts-generation-webui) | TTS Generation WebUI (Bark, MusicGen, Tortoise, RVC, Vocos, Demucs). | | | Speech | | [VALL-E](https://valle-demo.github.io/) | Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers. | [arXiv](https://arxiv.org/abs/2301.02111) | | Speech | | [VALL-E X](https://vallex-demo.github.io/) | Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling | [arXiv](https://arxiv.org/abs/2303.03926) | | Speech | | [Vocode](https://docs.vocode.dev/) | Vocode is an open-source library for building voice-based LLM applications. | | | Speech | | [Voicebox](https://github.com/SpeechifyInc/Meta-voicebox) | Text-Guided Multilingual Universal Speech Generation at Scale. | [arXiv](https://arxiv.org/abs/2306.15687) | | Speech | | [VoiceCraft](https://github.com/jasonppy/VoiceCraft) | Zero-Shot Speech Editing and Text-to-Speech in the Wild. | | | Speech | | [Whisper](https://github.com/openai/whisper) | Whisper is a general-purpose speech recognition model. | | | Speech | | [WhisperSpeech](https://github.com/collabora/WhisperSpeech) | An Open Source text-to-speech system built by inverting Whisper. | | | Speech | | [X-E-Speech](https://github.com/X-E-Speech/X-E-Speech-code) | Joint Training Framework of Non-Autoregressive Cross-lingual Emotional Text-to-Speech and Voice Conversion. | | | Speech | | [XTTS](https://github.com/coqui-ai/TTS) | XTTS is a library for advanced Text-to-Speech generation. | | | Speech | | [YourTTS](https://github.com/Edresson/YourTTS) | Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone. | [arXiv](https://arxiv.org/abs/2112.02418) | | Speech | | [ZMM-TTS](https://github.com/nii-yamagishilab/ZMM-TTS) | Zero-shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-supervised Discrete Speech Representations. | [arXiv](https://arxiv.org/abs/2312.14398) | | Speech |

^ Back to Contents ^

^ Back to Contents ^