Blog
Reinforced Machine AI Game Mechanics
Reinforced Machine is a twin-stick shooter action game where you are equipped with an AI Smart Weapon trained with Deep Reinforcement Learning. Each weapon has its own AI that modifies the weapon to adjust to the current situation.
Players can activate weapon AI mode or “Reinforced Mode” as called in the game, for a limited time. When in “Reinforced Mode”, the AI continuously makes weapon modifications while scanning for observable components in the environment; enemy position and volume, enemy behaviour, velocity, walls, and others.
Each weapon is different and assigned with its own AI Agent having its own choices of weapon modifications, or what we call Actions in Reinforcement Learning. The Weapon AI agent is also different where it uses the most appropriate sensor for collecting environment observations based on the weapon type.
Grid Sensor for Scanning Enemies and Environment
In the prototype gameplay video, the Energy Gun AI is using Grid Scanner to scan the environment. The AI can modify the weapon by changing its weapon mode:
- Rapid Fire – normal weapon mode
- Double Shot – fire straight double projectiles
- Spread Shot – spread small projectiles, low damage
- Sine Wave – hard to hit enemies but high damage
The AI adapts to the current situation and changes the weapon mode as it scans and detects enemies and the environment.
Players have a way to personalize their own Weapon AI to adapt its behavior based on their play style. For example, a player who prefer to charge towards the enemies should use a modified weapon efficient in close combat range. While a player who prefers to keep distance from the enemies would need a weapon with high accuracy.
AI training is purely OPTIONAL and players of ALL levels can play the game without touching any machine learning libraries. We will be providing pre-trained weapon AI models with different behaviors or use a community-trained AI model. Training is done through a Gym environment, enabling players to choose their preferred Machine Learning algorithm and set up how their AI will learn.
What’s exciting about integrating Reinforcement Learning in games is the AI unorthodox behaviour. We can only suggest a behaviour through training but the AI Agent can make its own strategy on what it deems the best action at the current situation. A great example is DeepMind’s AlphaGo [1] beating a Go champion. AlphaGo made an unconventional move, surprising everyone, which later turned out to give it an advantage. Another one is DeepMind’s AlphaStar [2] playing StarCraft 2, beating high level players with its own unexpected strategy [3]. We expect players to train their own weapon AI having its unique behaviour, personalizing their own game experience.
In upcoming posts, we’ll discuss how the training environment is created and how we expanded the grid level technique used in Assault Android Cactus [4]. This grid level creation enables us to create hand-crafted training environment in less than 10 minutes.
We are aiming for an Early Access launch this year. Follow us in our Twitter Page to keep updated. Steam page coming soon.
References
[1] “DeepMind AlphaGo” url: https://www.deepmind.com/research/highlighted-research/alphago
[2] “DeepMind AlphaStar: Mastering the real-time strategy game StarCraft II” url: https://www.deepmind.com/blog/alphastar-mastering-the-real-time-strategy-game-starcraft-ii
[3] “DeepMind StarCraft II Demonstration” url: https://www.youtube.com/watch?v=cUTMhmVh1qs
[4] “Assault Android Cactus – Transforming Floors and Dressing Levels” url: https://witchbeam.com.au/2013/05/14/transforming-floors-and-dressing-levels/
