I’m training an AI to learn my weird D&D character names

Kylie Szymesko
Apr 24, 2018 · 6 min read

Most people’s first impressions of Dungeons and Dragons is a bunch of people sitting around a table, rolling dice, and acting like adventurers. While this is accurate, D&D players know there’s a lot more to it than just that. Building dynamic characters, remembering abilities and making sure you aren’t rolling cursed dice are just some of the things we always keep in mind. And along with all that, taking notes.

From inventories to puzzles to clues about large overarching plots, D&D players take a lot of notes.

As a player in three campaigns and the Dungeon Master of another, I take an excessive amount of notes. Having recently started to fill my 3rd notebook in just a little over a year.

A small portion of my D&D related Google Docs

My usual process is to take as detailed notes as I can during a session, be it about items or plot or even just anything we do as players. A few days later I type these notes up in a Google Doc to share them with the rest of the party. For inventory management, I have a few tables as well as multiple spreadsheets for both tracking items as well as critical NPCs (non-player characters) who we might be at odds with.

However, I have recently started to get some freedom from my excessive note taking by using my own company’s business tool during sessions.

While my gang of online players already recorded full videos of our sessions (which is great for double checking possibly mixed memories), I was still tempted to start capturing all of our sessions in Luffa to see how well it could transcribe our adventures and because having searchable audio would be a huge time saver.

Organized my Luffa Spaces around each campaign I play in or run

Luffa is a capturing app that makes it possible to search through everything said during a session. Meaning, at the end of a three-hour session, I can search for the word “gold” and find every instance of the word “gold” said out loud. Letting me tally up our treasure total after the game instead of worrying about missing anything during gameplay. The platform was primarily designed to capture in-person meetings or work sessions, but ever since I first heard about it, my thought was “Okay but Dungeons and Dragons sessions.”

Now searching for things like “gold” and “treasure” is nice, but as someone who loves to track finer character and world details, Luffa can’t really help because it struggles with the fantasy jargon. At least it does at the start.

Roll20 on one screen and Luffa running on the other

Luffa is actively doing speech-to-text transcriptions in real time during a session, letting you reveal pieces of the transcript whenever you flag a Moment. In these pieces of transcripts attached to Moments, users have the opportunity to edit the transcript manually. Editing transcripts is an essential feature for companies with unique lingo, with the added unanticipated benefit of helping people who play D&D, a game filled with a bunch of weird fantasy world words.

As an example of weird words I’d like to search for, here are my current character names across four games:

Naivara
Skril’x (Yes, this is a Skrillex joke)
Quee
Cozamalotl

These aren’t names that are going to be easily translated by a speech-to-text system set to US English (or any of the other language options at that). So if I wanted to do a search to jump to parts of the session when things were happening involving my character, it would be near impossible. It’s not just my own character names that are difficult, my DM has spent hours coming up with unique names for enough places to fill a whole fictional world which are also extremely difficult for an AI to understand or transcribe.

One more layer on top of all that is dealing with drow language habits. One of our campaigns takes place in the Underdark (a vast underground world of tunnels and caverns that’s deep beneath the surface of the world). One of the principal inhabitants of the Underdark are the dark elves, drow, who really enjoy putting apostrophes in their names. See: Drizzt Do’Urden, Xen’drik, Bregan D’aerthe (not to mention the ridiculous words they don’t put apostrophes in).

Put all those things together and D&D is one of the worst use cases for a young speech-to-text platform.

But thankfully, Luffa isn’t a static platform.

With the ability to edit transcripts while they generate I’ve been going through selected portions of the transcript and correcting misspellings of proper nouns that are easy to spot. The system learns over time and after correcting some of my characters names around 4–8 times the platform is already starting to recognize their names on its own.

Every little green icon here is an instance of the name ‘Skril’, I only manually edited about 4 of them before it picked up the rest.

After putting in about 30 minutes of transcription editing across different campaigns and captured sessions, Luffa now recognizes about 80% of my unique character names. Making it super easy to jump to relevant parts of the session and find the exact information I want.

While Luffa is making it easier for me to double check game events that get muddled in my mind after playing a game for 3–4 straight hours, it hasn’t completely replaced the excessive notes I take. But it has undoubtedly led me to write down way less and stay focused and engaged with the story for a greater portion of the game. It’s also just been really fun to see the ridiculous misspellings that the system spits on when trying to transcribe cities names like Erelhei-Cinlu or Port Nyanzaru.

Editing a part of the transcript

With a brand new feature recently introduced to Luffa, I’ve now started to use the platform for more than just captured and searchable audio sessions. With the introduction of a Zapier connector, I’m able to take D&D info management to another level. By connecting Luffa to Google Sheets I’m able to populate my item sheet by flagging specific Moment types in Luffa during the game. Told about a glowing sword I just picked up? Flag it, have that moment added to my Google Sheet with a link back to that exact moment.

Party inventory of magical items stored in a Google Sheet now being auto populated

While the link is only able to populate a few of my columns when flagged, the URL takes me to the exact moment in the session so I can listen to it later to grab the details I need. Meaning I don’t need to pause the game to ask for repeated information. Instead, I can flag a Moment that links to this sheet to return to it later.

While I currently only have a Zapier connector in place to populate items spreadsheets I’m hoping to set up another one for tracking NPCs and key locations. While this is excessive information for a typical player, it’s a dream set-up for a Dungeon Master who is often having to make up information on the fly.

Given a few more captured sessions and some more time editing-in place names for newly introduced locations or characters, Luffa is quickly becoming the exact tool I want for managing my in-game knowledge.


Disclaimer: I work for Luffa but have been wanting to use the platform for D&D before I joined the team. People were getting curious as to how I use the software in my regular life so I thought I’d write up my most unique use case thinking that other people may find it useful as well!

Luffa

Luffa is the capturing app for teams that makes meetings more useful, productive and smart.

Kylie Szymesko

Written by

Zenyatta for startups and D&D enthusiast

Luffa

Luffa

Luffa is the capturing app for teams that makes meetings more useful, productive and smart.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade