Examining Pokemon AI as Inspiration for Scoring Systems
Examining Pokemon AI as Inspiration for Scoring Systems
November 2025
With LLMs all the rage, there is a desire for it to make more informed decisions. Perhaps there are multiple (good) choices one can make – how would we induce an LLM to select a ‘good’ choice?
One way to do this is using LLM as a judge. This boils down to a score card style system. The loop looks like this:
- Come up with your list of factors.
- Rate them 1-5.
- Add them all up and return the score.
How does the Pokemon AI extend this?
It focuses on 3 levels of score cards (this is an extremely stripped down explanation):
- Basic: discourages using moves without effect: stat-boosting moves when already at +6, healing moves when already at full HP, Fling when not holding any item, etc
- Strong: encourages doing the most damage possible and going for a KO if one is available.
- Expert: encourages using smarter strategies on a per-move basis
The great thing about this approach is the scalability, it is used in these games as a way to scale difficulty, though this approach also demonstrates how one can impart or build AI systems that start from a simplistic manner, and towards more advance mechanisms that impart domain knowledge.
To extend this to a more general use case:
- Basic: only consider candidate (allowable) actions with simple heuristics to determine whether or not they are effective or not
- Strong: encourage greedy actions that are known to be optimal/“win” the particular scenario
- Expert: encourage strategies (policies) that optimise or consider multi-turn, longer term planning
In the case for Pokemon, the final ’ensemble’ of these systems naively averages the scores across all levels to determine the ’next best move’. Just like in the machine learning scenario this generally is sufficient/good enough to inject “smart” actions in a system.
This kind of setup is definitely one I’ll be exploring and thinking about more in my next personal projects, and its ideas definitely can be used for creating video game AIs with scalable difficulties.