DRL-Renegades-Game-Bots

![Breakout](https://camo.githubusercontent.com/a5883c01515fc3da797f5d25cd2ff0e06668b381cd4d8a5acffecfab3ffea8a4/68747470733a2f2f7777772e67796d6c6962726172792e6465762f5f696d616765732f627265616b6f75742e676966) ![MsPacman](https://camo.githubusercontent.com/375a74c3bee22d5e2bf20e01ba3c2dd82747b8d3c6f6b4caee2f2e1d295d85e9/68747470733a2f2f7777772e67796d6c6962726172792e6465762f5f696d616765732f6d735f7061636d616e2e676966) ![Pong](https://camo.githubusercontent.com/df6b83797a4d475d3cb569f870a0529d8ea8698d2d8e94e10a43ce4531d02e97/68747470733a2f2f7777772e67796d6c6962726172792e6465762f5f696d616765732f706f6e672e676966) ![Space-Invaders](https://camo.githubusercontent.com/75749b3bed713f73a5e2efc8abf4678212719b85cfced6280470db06199fba66/68747470733a2f2f7777772e67796d6c6962726172792e6465762f5f696d616765732f73706163655f696e7661646572732e676966)

![Snowball-Target](https://camo.githubusercontent.com/744ba99a1caf76122ec8649743204c58c4cdebcaaf52a14dd99bcc67cdcab01d/68747470733a2f2f656e637279707465642d74626e302e677374617469632e636f6d2f696d616765733f713d74626e3a414e643947635242585777412d734d67625751394f72765046637a424556437941783950686554446b475a64462d4a786a70775a493531305157485343463464554f384b5745474c5a475126757371703d434155)

A collection of my implemented **simple & intermediate RL agents** for games like Pacman, Lunarlander, Pong, SpaceInvaders, Frozenlake, Taxi, Pixelcopter, Pyramids and a lot more by implementing various DRL algorithms using gym, unity-ml, pygame, sb3, rl-zoo, pandagym and sample factory libraries. A lot of agents implemented as part of this free tutorial series on DRL at HuggingFace: [Link](https://huggingface.co/learn/deep-rl-course/unit0/introduction) by [Thomas Simonini](https://x.com/ThomasSimonini). The rests were done based on various relevant resources and my ideas. I have added some links in **Acknowledgement** section below. ## Advanced-DRL-Renegades-Game-Bots To see my advanced & complex agents for complex games like soccer, rubiks-cube, mortal kombat, street fighter, vizdoom, montezuma, kungfu-master and more; visit this repository: https://github.com/hishamcse/Advanced-DRL-Renegades-Game-Bots ## Covered DRL Topics * Q-learning + DQN * Bellman Equation + Monte Carlo * Hyperparameter Tuning using Optuna * Policy Gradient * DDPG & DDPGfD * Unity Ml Agents * Advantage Actor Critic(A2C) for Robotics * Proximal Policy Optimization(PPO) with All Variants * Clipped Surrogate Objective Function * Sample Factory ## Table of Implemented Agents | **Environments** | **Libraries Used(includes HF)** | **Algos** | **Kaggle Notebooks** | |----------------------------------------|-----------------------------------------------------------------------|------------------------------|----------------------| | LunarLander-v2 | gym, stable-baselines3 | PPO | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-1-lunarlander) | | LunarLander-v2 | gym | DQN(Scratch) | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-1-lunarlander) | | Huggy | unity-mlagents | PPO | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-1-bonus-huggydog) | | FrozenLake-v1 (all variants) | gym | Q-Learn (Scratch) | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-2-frozenlake-v1-taxi-v3) | | Taxi-v3 | gym | Q-Learn (Scratch) | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-2-frozenlake-v1-taxi-v3) | | SpacesInvadersNoFrameskip-v4 | RL-Baselines3-Zoo, gym, atari | DQN(CNNPolicy) | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-3-space-invaders) | | CartPole-v1 | stable-baselines3, sb3-contrib, optuna, gym, atari, RL-Baselines3-Zoo | A2C(MlpPolicy) | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-3-optuna-cartpole-pong-br-out) | | PongNoFrameskip-v4 | stable-baselines3, sb3-contrib, optuna, gym, atari, RL-Baselines3-Zoo | PPO(CNNPolicy) | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-3-optuna-cartpole-pong-br-out) | | BreakoutNoFrameskip-v4 | stable-baselines3, sb3-contrib, optuna, gym, atari, RL-Baselines3-Zoo | PPO(CNNPolicy) | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-3-optuna-cartpole-pong-br-out) | | MsPacman-v5 | gym, atari | DQN(CNNPolicy) - Scratch | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-extra-unit-3-mspacmandqn-scratch) | | CartPole-v1 | pytorch, gym | Policy Gradient-scratch | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-4-pg-cartpole-pixelcopter) | | Pixelcopter-PLE-v0 | pytorch, gym, pygame | Policy Gradient-scratch | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-4-pg-cartpole-pixelcopter) | | Pendulum-v1 | pytorch, gym | DDPG-Scratch | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-extra-personal-unit-4-ddpg-ddpgfd-pendulum-v1) | | Pendulum-v1 | pytorch, gym | DDPGfD - Scratch | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-extra-personal-unit-4-ddpg-ddpgfd-pendulum-v1) | | Snowball-Target | unity-mlagents | PPO | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-5-unity-ml-snowball-pyramid) | | Pyramids | unity-mlagents | PPO + RND | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-5-unity-ml-snowball-pyramid) | | PandaReachDense-v3 | gym, panda-gym, stable-baselines3 | A2C(MultiInputPolicy) | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-6-pandagym-reachdns-pickplace) | | PandaPickAndPlace-v3 | gym, panda-gym, stable-baselines3 | A2C(MultiInputPolicy) | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-6-pandagym-reachdns-pickplace) | | LunarLander-v2 | pytorch, gym | PPO (All Variations Scratch) | [Link](https://www.kaggle.com/code/syedjarullahhisham/drl-huggingface-unit-8-i-ppo-scratch-lunarlander) | ## HuggingFace Models Find all my traned agents at [hishamcse agents](https://huggingface.co/hishamcse) ## Acknowledgements & Resources * [Deep RL Course](https://huggingface.co/learn/deep-rl-course/unit0/introduction) * [Thomas Simonini](https://x.com/ThomasSimonini) * [Deep RL Course Leaderboard - HuggingFace](https://huggingface.co/spaces/huggingface-projects/Deep-Reinforcement-Learning-Leaderboard) * [Open RL Leaderboard - HuggingFace](https://huggingface.co/spaces/open-rl-leaderboard/leaderboard) * [Stable-Baseline3 Agents - HuggingFace](https://huggingface.co/sb3) * [Stable-Baseline3](https://stable-baselines3.readthedocs.io/en/master/) * [All PPO Implementation Details](https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/) * [CleanRL Single File Implementations](https://docs.cleanrl.dev/) * [Unity-mlagents](https://github.com/Unity-Technologies/ml-agents) * [Coursera Machine Learning Specialization Course 3 by Andrew Ng](https://www.coursera.org/learn/unsupervised-learning-recommenders-reinforcement-learning?specialization=machine-learning-introduction) * [gym](https://www.gymlibrary.dev/index.html) * [Sample_Factory](https://www.samplefactory.dev/) * [RL-Zoo3](https://stable-baselines3.readthedocs.io/en/master/guide/rl_zoo.html) * [Panda-gym](https://panda-gym.readthedocs.io/en/latest/) * [Tools for Robotic RL](https://github.com/araffin/tools-for-robotic-rl-icra2022) * [Optuna](https://github.com/optuna/optuna) * [QRDQN - Quantile Regression DQN](https://advancedoracademy.medium.com/quantile-regression-dqn-pushing-the-boundaries-of-value-distribution-approximation-in-620af75ec5f3) * [Advanced DQN Pacman](https://github.com/jakegrigsby/AdvancedPacmanDQNs), [DQN Pacman](https://github.com/StarVeteran/Ms-Pacman-DQN) * [All Policy Gradient Algorithms Implementations](https://github.com/MrSyee/pg-is-all-you-need) * [DDPGfD - DDPG from Demonstrations](https://wikidocs.net/204469) * [Deep RL Paradise](https://github.com/alirezakazemipour/DeepRL-Paradise) * [OpenAI Spinning Up](https://spinningup.openai.com/en/latest/) * [Arthur Juliani Simple Reinforcement Learning in Tensorflow Series](https://awjuliani.medium.com/simple-reinforcement-learning-with-tensorflow-part-0-q-learning-with-tables-and-neural-networks-d195264329d0) * [Awesome Deep RL](https://github.com/kengz/awesome-deep-rl) * [Dennybritz RL](https://github.com/dennybritz/reinforcement-learning) * [HuggingFace Hub Python Library](https://huggingface.co/docs/huggingface_hub/index) ## My DRL Projects You can view my **Deep Reinforcement Learning** projects - [DRL-Renegades-Game-Bots](https://github.com/hishamcse/DRL-Renegades-Game-Bots) - [Advanced-DRL-Renegades-Game-Bots](https://github.com/hishamcse/Advanced-DRL-Renegades-Game-Bots) - [Robo-Chess](https://github.com/hishamcse/Robo-Chess) - [Kaggle Notebooks on DRL](https://www.kaggle.com/syedjarullahhisham/code?tags=13313)