Ulysses for Scientific Writing

1 Preamble

In this article, I present a workflow for scientific writing using Ulysses. It is not a tutorial to get you started using this brilliant writing tool. It is intended to complement an introductory tutorial to expand your skills and allow you to write scientific content.

The motivation for this article is to share the learnings I gathered over the last few weeks as I increased my productivity with Ulysses writing long machine learning content. I migrated to Ulysses from Scrivener because the latter lacked support for writing and managing equations on macOS. As I explain in this short document, Ulysses has great extensibility and can be exploited to write scientific and technical content. In addition, although it has far fewer features than Scrivener, it meets my needs with simplicity so that I focus on writing.

2 Ulysses

2.1 Introducing Ulysses

Ulysses1 is a writing application that helps writers focus on writing, use flexible structures, experiment, and leave the formatting to the end of the project. Ulysses is a lighter-weight and simpler alternative to Scrivener2.

What I like about Ulysses is that it does not fix a structure to a document. For instance, document writing software like Word and Pages adopt a linear structure. To use them effectively, I must have the document's structure fixed before writing. It is easy to experiment with different flows in a small document. But when working on a book or a long essay, it is difficult to have a visual summary of all chapters and sub-chapters, concepts and characters, and the general flow of the content. Ulysses’ simple user interface and flexible structure allow writers to focus on writing (not formatting) and readjust the writing flow as ideas form and the project progresses.

2.2 Ulysses Structure

Ulysses organises content hierarchically. The atomic level is the writing card, called Sheet in the Ulysses lingo, where you write and edit text, add resources etc. A writing card belongs to one group. A group belongs to another group unless it is at the top of the hierarchical structure of the project. You can nest as many cards and/or groups as you need under a group.

The number of cards and their content lengths are under your control. You may choose to have a few cards with a lot of content each, many cards with short text, or anything in between. In any case, you can visually see the structure of your project and drag and drop groups and cards to dynamically change the flow.

Figure 1 below shows a screenshot of the Ulysses project I am using to write this guide. The project name is in the top left corner of the image. The groups are just below the project name in the left pane, and the writing cards are to the right. Finally, the content of the selected card is shown in the main editor pane in the centre of the screen.

Figure 1: Ulysses Structure

The group called Tools for Scientific Writing contains four subgroups (Images, Equations etc.), while the group called Ulysses only contains writing cards. Ulysses has two writing cards Ulysses and Ulysses under the hood. The card Ulysses is selected, and its content is shown in the main editor pane in the centre. I will add a new writing card if I need another sub-section in this group. If I change my mind and decide that Ulysses under the hood should be in the Tools for Scientific Writing section, I drag it and drop it there. You can also move entire groups around to readjust your document structure and improve your flow.

2.3 Ulysses under the hood

Under the hood, Ulysses uses markup to format text and prepare it for export. Markup languages create formatted text in a plain-text editor. They allow you to do everything you can in Word or Pages related to formatting text, managing objects such as images and equations, managing document structure etc., without needing to point and click on various icons in the user interface. Instead, markup languages use special characters and syntax to indicate formatting options.

For instance, the title of a document is indicated by a leading %. Thus, the markup syntax of the title of this document is: %Ulysses for Scientific Writing.

Ulysses uses Markdown, one of the lightweight markup languages. Ulysses’ philosophy is that writers should be focused on writing, not formatting. Formatting happens once the writing is finished and at the export stage. In the writing process, writers focus on generating text. Being able to format while writing in the same text editor ensures that writers do not interrupt their creative flow. Or so goes the theory. In practice, this is true once the writer has mastered the basics of Markdown. To help the learning process, Ulysses has a Markup menu (screenshot 2 below) so writers quickly insert Markdown elements from the menu.

Figure 2: Ulysses Markup Menu

Out of the box, Ulysses offers several formatting templates that convert your writing into Markdown, pdf, HTML, ePub etc. In the export process, Ulysses attempts to render markups into formatted text according to the template’s rules. Thus, the basic Ulysses template is: write → format with markup → export.

The rest of the document presents a workflow that uses Markdown to add features essential to scientific writing without leaving Ulysses and another piece of technology called pandoc to produce the final formatted document.

3 Tools for Scientific Writing

Most scientific and technical content contains figures, tables, and equations. Writers often need to reference them, as well as sections and chapters. Ulysses does not offer native support for these features3, so I introduce the markup tricks that easily extend it to enrich your writing.

These tricks involve injecting raw markdown syntax within your Ulysses text content. Ulysses recognises raw syntax when it is encapsulated between two ~. Thus, to set the title of this document, the first card contains the following syntax ~%Ulysses for Scientific Writing~. Figure 3 below shows how this raw Markdown line looks in the text editor. Ulysses recognises that it is not plain text and displays it in green font on a black background (inspired by old-fashioned computer screens).

Figure 3: Setting document title with Markdown

We can now deal with specific items with this simple trick in the bag.

3.1 Images

An image can be inserted in a markdown text using the syntax:

![Caption](path_to_image.ext)

where Caption is the text describing the images, path_to_image.ext is the location of the image on your computer or the URL to it on the web. Note that ext is the extension of the image file, such as jpeg, png, or svg. To ensure that Ulysses recognises the Markdown syntax, you need to encapsulate it between two ~.

The raw source Markdown below inserts fig. 1 in section 2.2. The caption Ulysses Structure appears between the square brackets after the exclamation mark, and the path to the image is enclosed between the two parentheses immediately after the caption block.

~![Ulysses Structure](/Users/.../
        My Writing/Ulysses/
            Ulysses Structure.jpeg)~

• Note: to avoid formatting issues, inserting blank lines before and after the raw source text is advised.

3.2 Equations

Scientific writing often involves equations. Ulysses does not offer a native equation editor. Instead, it allows inserting raw source code for LaTeX equations4 within Markdown documents.

An inline equation or scientific notation, such as “Cᵢ”, is encapsulated between two single dollar signs $. Thus, the example above appears in the raw source block as ~$C_i$~, with C_i being the LaTeX code.

If you would like your equation to appear on a separate line, as eq. 1 below, it needs to be written between two double dollar signs $$. Equation 1 below appears as ~ $$C_i=\underbrace{\frac{x}{b}}_{\ddot{a}}$$~ in the raw Markdown block in Ulysses.

eq. 1

3.2.1 Rendering Equations in HTML

To properly render equations in HTML, you need to explicitly point to the MathJax resource online that renders the LaTeX raw source. The HTML code below does the job.

<script type="text/x-mathjax-config">
    MathJax.Hub.Config(
        {
            tex2jax: {
                inlineMath:[['$','$']]
                     }
        }
                    );
</script>
<script src='https://cdnjs.cloudflare.com
            /ajax/libs/mathjax/
            2.7.5/latest.js?
            config=default' async>
</script>

I usually insert the above code in a writing card at the start of the project so that it applies to the HTML file that contains all content. This works if I export the entire project into one Markdown file and create one HTML from it. If I need one HTML file per chapter, I insert the above code snippet at the beginning of every chapter.

• Note 1: you must still embed the HTML code snippet above between two ~ symbols.
• Note 2: when converting the project Markdown to pdf any HTML code is ignored. This is not necessarily the case with ePub.

3.3 Tables

3.3.1 Native Ulysses tables

Ulysses offers a native table management feature. It is flexible as it allows to add and remove rows and columns. However, further editing, such as changing border colours, is impossible.

The listing below shows the markup for a sample table built using Ulysses’ native markup functionality.

| On functions |
| — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — |
| A function is a mathematical expression describing a relationship between input and output sets. For instance, the square of whole numbers is a relationship between the set of integers 0, 1, 2, 3, 4… and the set 0, 1, 4, 9, 16… Its mathematical expression |

: Table 1 Sample Native Table

3.4 HTML Tables

The workflow described in pandoc does not pass raw HTML to LaTeX and vice versa. This limits the editing options for tables. It is best to use the native Ulysses option, which gets translated to markdown syntax that pandoc translates. There are convoluted ways to go around this limitation, but it is probably best to focus on writing than messing around with pandoc.

4 End-to-end workflow for scientific writing

4.1 Basic workflow

You can publish your writing now that you have written your content and added images, equations, and tables. In this section, I describe the basic workflow that moves your content from Ulysses to a file format of your choice.

The first step is to export your content in Markdown format. To do so, you either click the three dots inside a circle at the top of the navigation pane to export the entire project or right-click on a group or a writing card to export a section of your project. Figure 4 below shows screenshots of the two options.

Figure 4: Basic Ulysses Workflow for Scientific Writing

Next, you need to run a pandoc command to convert the Markdown file to the format of your choice, be it pdf, HTML, DOCX, LaTeX or other. Before showing you how to do so, I will first briefly introduce pandoc.

4.2 pandoc

pandoc is a free software that converts between numerous markup and word processing formats, and a command line tool that uses the software. pandoc does not come with a graphical user interface; you need to learn how to run commands in Terminal5 to use it. Under the hood, pandoc takes the input markup, converts it to a native format (called abstract syntax tree or AST) with a reader and then converts the AST representation of the input to the desired output using a writer. This way, pandoc runs fast and avoids creating converters between every combination of input and output formats.

The basic usage of pandoc is as follows:

pandoc input.txt -o output.html.

The command line instruction above tells pandoc to convert the plain text file input.txt to an HTML file called output.html.

Alternatively, we can instruct pandoc to convert a file from one format to another as follows:

pandoc --from markdown --to html input.md.

• Note: pandoc’s final output depends on the reader-writer pairs. If you convert Markdown to pdf it will look different to the output of the conversion from Markdown to html.

4.2.1 Installing pandoc and other dependencies

On Mac, installing pandoc is very easy; all you need to do is run the following homebrew command in the terminal:

brew install pandoc.

It is possible to install pandoc from an executable, however, I prefer homebrew because it makes it easy to upgrade to newer versions.

You also need to install MacTeX which is a LaTeX distribution for macOS. It is essential for rendering equations. You can download and install it from its official website6.

4.3 Referencing

The final tip we will cover is referencing your objects from within the text and publishing lists of items (e.g. list of figures) and a table of content.

4.3.1 pandoc-crossref

pandoc-crossref is a pandoc filter for numbering figures, equations, and tables and cross-referencing them. A pandoc filter is a programme that modifies the abstract syntax tree (AST) (sec. 4.2) between the reader and the writer. By doing so, a filter can add extra features unavailable in the base pandoc programme.

To run a filter, we need to have its executable (available in the Github repository of the filter) and add the option --filter to the pandoc command line, as the example below shows:

pandoc --filter /path_to_executable/pandoc-crossref.h --from markdown --to epub input.md.

For more details about pandoc-crossref, consult its excellent documentation.

4.3.2 Referencing with pandoc-crossref

pandoc-crossref offers a specific syntax to add references to objects and retrieve them within the content. The basic syntax is: {#object:Label} where object is fig for figures and images, tbl for tables, sec for sections and chapters, and lst for code listings, and Label is a unique reference to the object. Of course, such a Markdown code is added within a raw source block in Ulysses.

To retrieve a reference within your text, you use the following structure: ~@item:Label~. To insert more than one reference, you list them between square brackets ~[@item1:Label1; @item2:Label2]~. item1 and item2 need not be the same type; you can reference a list that includes equations, figures, tables etc.

Please note that the items in the list are separated by ; not by ,.

pandoc-crossref does the numbering, incrementing etc., on your behalf. You only need to give a unique text reference to your object. pandoc-crossref documentation7 explains how this works and gives details on various options, such as resetting numbers at which header depth, whether to include the chapter number in the reference or to use section headers instead of numbers. Using text labels to reference objects and setting the numbering scheme in the publishing phase gives you flexibility over the structure of your content without worrying about numbering.

Imagine that you are manually numbering chapters 1 to 10. If you inspect your flow and realise that chapters 6 and 8 are better before chapter 4, you have to manually edit chapter numbers (and any section below them and references). This is not what Ulysses wants you to do. It wants you to focus on writing. Technology does the rest.

Although the syntax is the same for all types of objects, where you insert differs. In the following few sections, I will provide the basic syntax. I also give in section sec. 4.4.3.2 a few details on how to customised pandoc-crossref~ by providing metadata.

4.3.3 Referencing images

Once you insert an image with the markdown syntax explained in sec. 3.1, append {#fig:Label} to the raw source block, where Label is a unique reference for the image.

Remember to leave a blank line before and after the image, otherwise the {} won’t be recognised.

To reference fig. 1 in section 2.2, I used the following code:

~![Ulysses Markup Menu]
    (/Users/../Ulysses Markup Menu.jpeg){#fig:ulysses_markup_menu}~

And to retrieve it within my text, I use ~@fig:ulysses_markup_menu~ to get the result: fig. 2.

I can also reference a list of figures (such as figs. 2, 4) by inserting the following Markdown:

~[@fig:ulysses_markup_menu;@fig:basic_workflow]~.

4.3.4 Referencing equations

We can also reference equations as in eq. 1 with {#eq:Label} where Label is a unique identifier. To retrieve a reference to an equation, we add @eq:Label at the end of the Markdown block, including the LaTeX of your equation. The full code block I used for equation 1 is:

~$$C_i=\underbrace{\frac{x}{b}}_{\ddot{a}}$$ {#eq:frac_eq}~.

4.3.5 Referencing tables

Tables are referenced by appending {#tbl:Label} at the end of the table caption block where Label is a unique reference. For instance, in this document, we have a table in sec. 3.3, it is referenced as tbl. 1. This reference is inserted below the table I created using Ulysses’ native table functionality. I also added a caption for this table using : Caption syntax. Thus the full raw source code for referencing tables is:

~: Caption {#tbl:Label}~.

As an example, the full syntax for table 1 is:

~: Sample Native Table {#tbl:sample_table}~

Please note that the line above should be preceded and followed by at least one blank like.

4.3.6 Referencing sections

The pandoc-crossref syntax for referencing sections is {#sec:Label}, where Label is a unique reference. A section is a heading at any depth.

A reference to section 4.3.2 is created using ~{#sec:referencing.pandoc}~ and retrieved using @sec:referencing.pandoc.

4.3.7 Referencing another sheet

Ulysses has native referencing between sheets (writing cards). To create one, right-click the sheet you would like to reference while holding option. In the ensuing dropdown menu, click Copy URL Callback. The URL is now in your clipboard, and you can paste it into another sheet. Select the text you would like to use as hypertext, press command+k, and paste the copied reference in the URL field in the pop-up window that appears.

Here, I pasted the reference to the Equations sheet:

ulysses://x-callback-url/open?id=qLbJVjCa7ODHasJzPwh2gA.

4.3.8 Managing prefixes

You may have noticed that a simple reference recall with ~@item:Label~ returns an abbreviation of the object type before its reference number, for instance, “fig. 2.1”. You have full control over this behaviour that I detail below.

4.3.8.1 Prefix suppression

Prepending - before the @ and encapsulating the reference between square brackets8 will suppress the prefix of the reference. Thus, @fig:ulysses_structure gives the full reference with the prefix “fig” (fig. 1), and [-@fig:ulysses_structure] will just give the figure number (“1”).

4.3.8.2 Custom prefix

One can add a custom prefix manually to a reference using this syntax: [Prefix @reference].

For instance, if we want @tbl:sample_table to be referenced as note we write [note @tbl:sample_table] to get “note 1”.

4.4 Publishing

4.4.1 Basic publishing

Section 4.1 introduced us to the basic workflow of exporting our projects from Ulysses as Markdown9, and then used pandoc to publish them in different formats. This is necessary if we would like to render Markdown and LaTeX raw source code10.

The basic commands to convert your Markdown are:

pandoc file.md --output file.pdf
pandoc file.md --output file.docx
pandoc file.md --output file.html
pandoc file.md --output file.epub --webtex --metadata
        title="Test equations with Pandoc"

Note that ePub requires minimum metadata (a title), and the option --webtex to render equations.

4.4.2 Publishing with pandoc-crossref

To get the benefits of flexible referencing, we use the same process as above, except that we now need to add the option --filter pandoc-crossref to include referencing throughout our document.

The simplest command is:

pandoc --filter pandoc-crossref file.md --output file.pdf

Note that if you have equations and would like to publish in ePub format, you still need to add the pandoc option --webtex to the command.

4.4.3 Advanced options

4.4.3.1 Pandoc options

--number-sections automatically adds section numbers to get a reference to a section such as sec. 4.3.3.

--top-level-division=chapter inserts a page break between chapters in the output and may be needed to include the chapter number in the reference.

--toc tells pandoc to insert a table of content inferred from our document headings.

4.4.3.2 crossref options

A YAML can be created to hold settings for crossref. The listing below shows an example. numberSections: 1 tells crossref to start chapter numbering from 1 (default is 0).

numberSections: 1
chaptersDepth: -1
figPrefix:
  - "fig."
  - "figs."
tblPrefix:
  - "tbl."
  - "tbls."
eqPrefix:
  - "eq."
  - "eqs."
chapters: true

chapterDepth: -1 tells crossref to number sections at all levels.

chapters: true tells crossref to include the chapter number in references to figures, equations, tables etc.

figPrefix, eqPrefix and tblPrefix set the prefixes to be included with references in singular and plural forms.

To take account of these options, pandoc command becomes:

pandoc --filter pandoc-crossref 
-M pandoc-crossref-metadata.yml 
--number-sections 
--top-level-division=chapter 
file.md --output file.pdf --toc

4.4.3.3 epub options

pandoc documentation11 explains how to add metadata for ePub format. This format requires extra metadata about your work, such as a title, author(s), editor(s), and copyright. These rich options can be defined in a text file of the YAML format. For instance, I created a YAML file called epub-metadata.yaml that holds the following metadata for ePub export:

title:
- type: main
  text: Ulysses for Scientific Writing
- type: subtitle
  text: Write fluently and easily while managing
        tables, figures, equations and references
creator:
- role: author
  text: Dr Ahmed Khamassi
- role: editor
  text: Rebecca Miriam Khamassi
identifier:
- scheme: DOI
  text: doi:10.234234.234/33
publisher:  Glisco Press
rights: © 2023 Ahmed Khamassi, KK CA-BB
ibooks:
  version: 1.3.4

Without a YAML metadata file, the command line requires --matadata title="My Book" to provide the minimal metadata the ePub expects.

The full pandoc command to publish our work in ePub format becomes:

pandoc --filter pandoc-crossref 
-M pandoc-crossref-metadata.yml
--number-sections  
file.md 
--output file.epub 
epub-metadata.yaml 
--webtex --toc

4.4.4 Lists output

Lists of figures, equations tables etc., can be provided with LaTeX native \listoffigures, \listoftables.... In the following few sections, you will see the lists of figures and tables in this document.

5 List of Figures

1. Ulysses Structure
2. Ulysses Markup Menu
3. Setting document title with Markdown
4. Basic Ulysses Workflow for Scientific Writing

6 List of Tables

1. Sample Native Table

1. https://ulysses.app/↩︎
2. https://www.literatureandlatte.com/↩︎
3. Strictly speaking, Ulysses supports native insertion of media and tables. However, we need to go a little further into the Markdown world to be able to reference and list them.↩︎
4. For more details on how to write LaTeX equations, consult this cheat sheet https://www.reed.edu/academic_support/pdfs/qskills/latexcheatsheet.pdf↩︎
5. For more details, see: https://support.apple.com/en-gb/guide/terminal/trmlddf3251b/2.13/mac/13.0↩︎
6. https://www.tug.org/mactex/↩︎
7. https://lierdakil.github.io/pandoc-crossref/↩︎
8. Even when there is only one element to reference.↩︎
9. command+6 is a short cut to exporting in Ulysses.↩︎
10. If you do not have any of these objects, then you can use Ulysses’ native exporting options.↩︎
11. https://pandoc.org/chunkedhtml-demo/11.1-epub-metadata.html↩︎