What it takes to Apply AI to Pokémon Sword/Shield
Pushing Ai forward by applying scraping techniques to modern games.
Intro:
I am The First Son of the Hurricane and the Grand Master of the Storm Order.
This article desires to remind people the importance of scraping and collecting data the hard way. If you can manage to pull off such feats, you can perform wonders that rival OpenAi and Deepmind.
Video demonstration is here and at the bottom of the article.
Hopefully after watching the video and reading the article, you understand why I am the Emperor of Pokémon Ai.
Background:
Ai bots for games are held back due to the difficulty of collecting the necessary data to give a neural network. Games are not designed for this in mind. Fortunately, there are ways around this limitation. Either the data is given to you, or it is taken.
Game data exists in the Ether. It exists, and nothing prevents you from taking it aside from your skill and will.
We’re all familiar with Alpha Go tackling Go and OepnAi Five tackling Dota 2. Those feats are both nice, but they have limits.
In the game of Go, the environment was create by Deepmind and thus all information of interest is readily available for them to train on.
In the event of OpenAi Five, they are provided with tools that are not available to the public. So that is data being handed to them from the creators.
For this Article, Pokémon Sword and Shield will be used, but you can easily replace with other games.
Observation and Action Space:
To make Ai bots for Pokémon of the simplest form, you might need to enter all the information for your team and who you are fighting.
When playing the game, all this information is readily available to you, even during a match. You just have to collect the data. This data can represent your observation space.
Your attacks/actions will represent your action space.
When it comes to generating the observation space from your team information, you could write down by hand all this information. After some frustration you might come to your senses and realize it is better to have software collect this information programmatically.
Extraction Notes:
Game data usually appears in fixed positions on the screen. You can use various tools to find the bounding box positions of all the information you are interested. Then sending those regions to the OCR tool you can construct the team information.
On the screen below, you can see some stats information of a Pokémon. The stats for each Pokémon appear in the same location on the screen, so you can use these patterns to your advantage.
On the following two screens, you can information related to your opponent. You can use this to deduce who you are fighting and add this to your observation space. You can either use a combination of the Pokémon’s name and elements to determine who you are fighting, or an object detector on the character’s mini icon. The object detector on the icon is more reliable than the name since people can change the names, however it is a lot more work to get all 800+ Pokemon icons for training. Some battles are not worth fighting.
On this screen you can look at your available/legal attacks. These can represent the different actions you want to take: action1, action2, action3, action4 respectively.
By scanning all these screens you can construct an observation space that contains your Team’s stats and attacks, their status (Poisoned, Fainted, Frozen, etc), and your current opponent. Your actions will be the available attacks.
From there you can use any behavior cloning techniques you know to make Ai bots that can play similar to a human.
It is important to note that in Pokémon, not all actions are legal at a given time. One example would be if a Pokémon is taunted, a good portion of attacks are not legal. How to handle that logic is beyond the scope of this article.
I must warn you, this style of programming is no walk in the park. If you dare to try something like this, I STRONGLY recommend games simpler than Pokémon Battles. Or if you insist on Pokémon, then either automate everything or automate nothing. You have no idea the horrors that await you.
Video Demonstration:
In the video at the bottom, you can see all of this automated. But starting out, you do not need to automate. You could manually navigate to the screens of interest, and run your script to collect the data.
Korean Voice Over English Closed Captions.
English Voice Over Version
About Me:
I am a Digidestined who is a Pokémon Ai specialist. I have been studying Pokémon Ai in secret for a long time. That is why I can do these things.
I have many tools related to Pokémon Ai. This is just one of them.
I am also trying out Media Coverage as a way to get my articles to a larger audience.
Ivermectin:
https://www.youtube.com/watch?v=ec6XslfVSDE
https://www.youtube.com/watch?v=Tq8SXOBy-4w
For journalists:
https://www.foxnews.com/us/us-russian-soldiers-train-together-in-colorado
