Building a dictionary-natural Wordle Solver quick
The Wordle game has attracted a lot of interest not soon after its release. The challenge of guessing a five-letter hidden word is incredibly captivating. The ingeniously simple rules of the game fascinate human players as well as computer/maths people who want to find the best solution. The video made by maths educator 3Blue1Brown about the use of Information Theory in solving Wordle is amongst the most well-known solutions available on the web. The popularity of Wordle also led to the emergence of unofficial implementations. Some of these unofficial versions support different word lengths and word lists of multiple languages.
Personally, I struggle with vocabulary. I have not been able to solve a Wordle challenge within six tries without assistance. So, I imagined a World solver that would not only help others who also suffer from vocabulary problems to enjoy the game but also provide an answer to the hype of Wordle.
A Wordle solver is a tool that receives a player’s input of Wordle feedback and then outputs candidate words to help the player guess the hidden word within six attempts or as soon as possible.
Unfortunately, I did not have the luxury of time to invest lots of effort in building a brilliant solver. Therefore, any approach that required a high number of simulations or training a model would not be a viable path for me to build a solver.
Most approaches to solving Wordle published online require:
a.) Access to the word list of a specific Wordle implementation; and
b.) Simulations that iterate over all, or a randomly sampled subset of, possible pairs of a hidden word and a guessed word, with worst-case complexity of O(n²)
to build.
Furthermore, I also wished that my solver could be “future proof” and “low maintenance” in a way that it could adapt to unofficial Wordle versions with minimal or even no modification.
I strategically posit that my Wordle solver should:
● Be future-proof & low maintenance;
● Take a short time to build; and
● Be cheap to deploy.
Therefore, practically, I should not:
● Fixate on the word list of a specific Wordle version and five-letter words; and
● Experiment with approaches that require supervised learning and a high number of simulations.
1. Word lists
First, I needed a list of English words. The search was pretty straightforward. I found two lists that I could use: MIT’s 10k commonly used English words and dwyl’s 466k English words.
It is possible to extract two word lists from JavaScript code of the official version of Wordle. One is “allowed guesses” and another one is “answers”. However, these two lists were not very helpful in my case as I did not target a specific implementation of Wordle.
2. The plural issue
The use of a comprehensive/generic word list presents a “plural issue”. Generic word lists usually contain both the singular and plural forms of a word. For example, one may find both “pale” and “pales” on a list.
The inclusion of plurals would enlarge the search space dramatically for both humans and computers. For example, in a challenge of guessing a hidden word of five letters, the inclusion of plurals means that the challenge becomes guessing five-letter singular words plus four-letter singular words.
Therefore, my solver removes plurals by default but provides an option to keep the plurals. In case a particular Wordle version is known for using plurals, the players have the option to include plurals in the search space.
3. Sorting candidate words
Suggesting good words to players to end the game sooner is the key functionality of a Wordle solver. The best-performing approaches require running high numbers of word-guessing simulations to find the best words that makes Wordle give the most informative feedback. As I mentioned earlier, I deliberately avoided simulations and supervised learning, except for evaluation.
I started with the intuitive and naïve approach of using conditional probability. Each time the solver takes in feedback from Wordle, the solver transforms the feedback into search criteria. Words that do not match the search criteria are filtered out. Then, the solver ranks the remaining words by the conditional probabilities of characters in each position and suggests the top few ones to the players.
Unfortunately, the naïve approach alone did not work very well because the solver failed to pinpoint the hidden words within six tries in most cases.
As an example, for the word “alone”, its score would be:
score = P(‘a’ in position 1) * P(‘l’ in position 2) * P(‘o’ in position 3) * P(‘n’ in position 4) * P(‘e’ in position 5)
4. Dual-list
I downloaded two word lists for my experiment: MIT’s 10k commonly used English words and dwyl’s comprehensive English words. I refer to them as “short list” and “long list”, respectively. I wondered which list would work better. I found that the short list was “too short”. Some possible hidden words in the official Wordle could not be found in the short list. On the other hand, the long list was “too long”. The search space was too large, and it took more than six attempts to guess the hidden word correctly very often.
After some experiments, I found that it would be possible to find the hidden word within six tries if the choice of the list is alternated midway. It worked when the solver used the short list on the first two attempts and then switched to the long list from the third attempt on.
When I combined the dual-list approach with sorting by conditional probabilities of characters, the solver was able to solve both official and unofficial versions of Wordle within six tries in most cases.
5. An opener list
The next thing is to find a way to select words to form my own short list for the purpose of solving Wordle. I would like to call this new short list the “opener list”. From the theory, we know that the words on the short list, which is based on MIT’s 10k list, provide more information to shrink the search space. Presumably, the words on the 10k list were selected on the basis of their occurrence in daily use.
3Blue1Brown’s video told us how to use information theory to find the most informative words. However, his approach requires running a high number of simulations to generate data to calculate the entropies of words. This approach was impractical for me as I did not have the luxury of time.
So, I experimented with unsupervised learning. I tried to find a sub of words on the long list which is representative of the whole set in terms of spelling. To be precise, I used k-means clustering to find clusters of words based on their spelling similarity.
I converted the words into vectors by assigning each character a number. I plugged the vectors into the k-means clustering method of Scikit-learn. In a matter of seconds, I got clusters of words. There are a number of words in each cluster. In each cluster, I ran simulations (within the cluster) to see which word would yield more combinations of feedback. The best word from each cluster is selected to form the new “opener list”.
Clustering requires that the vectors are in the same shape. So, only words of the same length may be processed at a time. The words on the full list are segmented by their word length before vectorisation and clustering. Representative words from clusters of different word lengths are calculated separately and then merged into a single opener list.
6. Performance
Although performance is not a top priority of my solver, it is still meaningful to compare the performance of my approach with those of others. Botfights.ai is a website dedicated to bots solving Wordle. The website opened tournaments of different word lists. I did not have time to take part in every tournament.
For the tournament of 2,000 words on the hidden word list of the official Wordle, my solver ranked 4th/5th (3.6 guesses on average; 0.55 more guesses than the champion). For the tournament of variable word length challenge, my solver ranked 7th (3.5 guesses on average; 0.45 more guesses than the champion). (Note: The lower the number of guesses, the better the performance.)
To me, the performance is acceptable.
7. Making it publicly available
A nice UI is another key component of my solver. The Python-CLI version is not user-friendly enough for most people. I spent some time building a single-page web app using vue.js.
I exposed the solver as a WSGI function and packaged it as a cloud function.
The static UI is hosted on Google Cloud bucket and the solver function is hosted on Google Function. Google charges just USD $0.02 every month for hosting the static vue.js frontend and the function.
My Wordle Solver (https://solvewordle.games/) was not hugely popular, but I got users almost every hour over the last few months. Perhaps more needs to be done to promote my solver.
Discussion
The Wordle Solver that I built is not top-notch but is reasonably usable. I managed to strike a balance between building time and performance.
The intuitive and naïve approach of using the conditional probability of characters’ positions could not work alone but turned out to work well in conjunction with an opener list of representative words generated using clustering.
I hope that this word list-natural Wordle Solver is able to cope with more variants of Wordle before any change is needed.
