Best of Both Worlds: Flexible Project Files in Character Engine

Aaron A. Reed
May 24, 2019 · 5 min read

Anyone designing an authoring tool for a dynamic system faces a particular dilemma: IDE or DSL? That is, should you make an integrated development environment that provides an abstracted authoring interface into the underlying data, with high-level features and visualizations tailored to working with that particular data, or design a domain-specific language, a specification and compiler for authoring content in your system purely as lines of text?

Each approach offers advantages and drawbacks. A DSL can be used in the writer’s environment of choice, including powerful third-party text editors that may bring decades of design iteration and developer expertise to bear. On the other hand, a DSL can feel much more like programming, perhaps locking out non-engineers from content creation (whether through legitimate skill barriers or mere intimidation). IDEs can be much more user-friendly, but also risk growing overly complex as more and more features are added, and may be more difficult to keep up to date with a changing system. DSLs can offer more power, but IDEs generally make that power easier to deploy.

Previous interactive narrative systems have used a variety of approaches. Inkle first offered the more IDE-like Inklewriter, but later discontinued it to focus on the easier-to-maintain (and more powerful) DSL, Ink. Twine’s graphical editor for creating nodes, or the in-house Telltale Tool used to create that studio’s narrative games, are each more IDE-like. Inform 7 is a DSL coupled with a powerful IDE with features for automated testing, release packaging, and diagnostics.

When we started designing the authoring tool for Spirit AI’s Character Engine several years ago, we had a number of discussions about the best approach to take. We settled on an IDE in part because one of our key missions is opening up the creation of complex interactive characters to writers more broadly, not just writers with coding experience. The tool has incorporated a few DSL-like conventions from the beginning, though: most notably, a “screenplay” view where authors can sketch in the bulk of their content by typing conventional-looking dialogue, and wait until they’re ready to step out of a creative flow to go back and modify these nodes to make them more dynamic and procedural.

Image for post
Image for post
Character Engine Authoring tool Plot View lets you create lines of dialogue just by typing, then go back later to add conditions, effects, procedural text tags, and other details.

A limitation of some IDEs is that their native file formats are not (or not easily) human editable. Authors who do have some programming expertise might find themselves wanting to create custom automation shortcuts, generate batches of similar content procedurally, or create special high-level commands that can be mechanically translated into the system’s native format — things that are all easier to do with a DSL than an IDE. This kind of automation has not previously been possible with Character Engine, but we’re happy to announce that in our upcoming Release 11 we’re switching to a new project format that will make it more possible to achieve these kinds of advanced workflows.

Specifically, our project format is changing from the single-file compressed .aiproj format to a folder with plain text files called a .sheaf. Written in the human-readable TOML format, the .sheaf format provides a number of advantages. It makes it easier for multiple authors to collaborate on the same project via version control, since each authoring change can easily be seen in a diff. It’s now possible to make small edits directly within the .sheaf, outside the context of the authoring tool (although you’ll still want to use the tool for day-to-day editing). But we’re especially excited about the possibility this opens up to integrate writing for Character Engine into more complex authoring workflows.

Image for post
Image for post
Example of an authoring update to a .sheaf project in a version control tracker, showing a change where a line had its text edited and a tag removed.

For instance, say you wanted to author a large number of lines with a complex set of preconditions and state-change effects. There were a few ways to achieve this previously — you could put all of these tags in a reusable Fragment used by each line, for instance — but sometimes there was no alternative but to duplicate large numbers of effects and conditions by hand. With the .sheaf format, you could create your own custom marker (say, a comment like #SpecialLine) and use a find/replace tool across the sheaf files to replace this with the full set of tags and conditions you wanted to appear. Other possible uses: automatically generating or importing large numbers of topics or knowledge model entries, project-specific validation (if, say, your scenario requires that each line spoken by Simon must transition to a line spoken by Jerry), or even just running your project through a language- or project-specific spell checker before committing an update.

We’re also working on a standardized Python library for manipulating .sheaf data — although this will not appear in Release 11, since we want to thoroughly exercise it internally before making it public. But we’re hoping when this comes out, it will make it even simpler to procedurally modify .sheafs to your heart’s content.

Sometimes the best solution to the IDE or DSL question is to offer the flexibility to choose whichever approach is best for your current needs. We’re hoping the new .sheaf format is another step towards continuing to make Character Engine useful in a wider and wider variety of scenarios and workflows.

The new format is just one of several new features in Release 11, which will launch in the next few weeks. Look for the official release announcement soon!

Spirit AI

AI for humans

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store