Codux (by Wix) Case study: How we built The Elements Tree Panel: Architecture, technologies, and implementation.
Integrated Development Environment
The term “Integrated Development Environment” was coined in the early 1970s to describe a single program that combines authoring, modifying, compiling, deploying, and debugging software.
Integrating these capabilities into a single program allowed for many innovations down the road, such as Syntax Highlighting, Code Completion, Type Checking; but above all: it made software flexible because it closed a feedback loop: developers could type in a command and see the result right away.
Codux
Codux is a visual IDE for React made by Wix. It displays your components on the stage, in the middle of the screen, so you can what you’re working on while coding. Put another way: Codux closes the feedback loop for React developers by integrating rendering into the IDE.
Integrating rendering with the rest of the traditional IDE features has enabled a bunch of innovations and productivity improvements. I’m very proud to have been the lead developer on one of these innovations — The Elements Tree, and I want to share how we built it: its architecture, the technologies used, and some of the insights behind it.
The Elements Tree
The elements Tree visualizes the structure of your app, helping devs with several things:
- It makes it easier to understand the structure of your app, kinda like syntax highlighting on steroids.
- Every row in the elements tree represents a sub-structure in your code; clicking it will allow you to edit it in other panels. For example, you could select a
<span>to change itsfont-size.
- It allows you to make visual changes, for example: you can drag a
<br/>into the<Layout>components’contentproperty, drag it into anmap(x => y)expression, or delete it altogether.
Architecture
The Elements Tree is a good case study to explain how exactly this integration works:
Codux reads files from your filesystem and parses your code into a Typescript AST data structure. Next, Codux runs a few transformers that add useful metadata to the runtime code. Finally, Codux renders* your code and displays it on the “Stage” in the middle of the screen.
*Obviously, rendering requires compiling, building, and bundling — very cool stuff, but outside the scope of this article.
The Elements Tree has three main inputs, represented by the blue boxes in the diagram.
- TypeChecker provides types for arguments, component properties, variables, consts, function return values, and everything else that can be typed.
- Typescript AST — is a tree data structure representing your code. AST stands for Abstract Syntax Tree.
- React devtools provides a FiberNode structure, representing what has been rendered on the Stage.
The Elements Tree has one main output: The ASTModification API, I’d say it is the key for visually editing code. It is responsible for applying various modifications to the AST, such as: adding a component, removing an expression, or passing a value into a property. Once a modification is applied, Codux generates new code to reflect the change on your filesystem.
Let’s dive into each of these in more detail:
Technologies and Implementation
Abstract Syntax Tree (AST) in Typescript
An Abstract Syntax Tree (AST) is a data structure representing a program’s code. ASTs are essential for working with code. More info in the Wikipedia article here. It allows Codux to do several things:
Code Analysis
Typescript provides great tools to create syntax trees directly from code:
sourceFile is a parsed AST; it is of type ts.SourceFile, extending ts.Node, which represents a node in the syntax tree. To traverse the tree, we can use node.forEechChild. For example:
This is a recursive structure, and you can call childNode.forEachChild.
Every node has a .kind property, which lets you identify it with a corresponding type guard. For example, once we call ts.isJsxElement(node), typescript identifies that node’s type is ts.JsxElement allows you to use properties unique to this type, such as .openingElement and .tagName. For example:
findRootJsx will traverse the nodes until it finds a ts.JsxElement and return its tagName, and position. node.forEachChild function will return whatever its’ callback will return. This code logs { tagName: ‘h1’, from: 127, to: 160 }.
Modification
Modifying your code is relatively straightforward. You can parse the code into a ts.sourceFile, then every ts.node inside has a node.getStart() and node.getEnd() methods that can help us find the exact position in the sourceText string. We can replace in these positions with anything else. However, there is a better way.
It would be simpler and less prone to errors to modify the syntax tree and let Typescript handle the changes to the actual code. Transformers are the tool for that.
Transformers
A Transformer is a function that transforms a given ts.Node, changing it in some way. For example: let’s say we want a transformer to replace all <h1>s with <b>s to make titles bold.
will become ➡️
to do that, we first need a ts.TransformerFactory object:
Next, to find the ones we want to transform we need to iterate all the nodes in sourceFile, we can use ts.visitEachChild:
Now that we’ve found the jsxElement, we want to check that it’s an <h1>:
Finally, we want to replace the <h1> tag with found with a <b> tag:
That’s it! To see the result, we apply the transformer and print the result:
The log would display:
These are very powerful tools, but so far, we haven’t seen anything new — traditional IDEs can analyze code and transform it; in fact, linters have taken advantage of these and similar tools since the 70’s when scientists were writing C code in Bell Labs. It is time to see how Codux integrates these tools with rendering.
React devtools
To understand what was rendered and visualize the structure of your app the Elements Tree employs React devtools. For example, check out the ColorfulList component below and its’ Elements Tree:
The {items.map(…)}function is represented by the green lane (with the square on top) inside the div.root.
The recurring {item.emoji}and {item.title}expressions are represented by grey lanes. Because Codux renders your code, the Elements Tree can display all of the items, with their respective .emoji and .title values.
React’s devtools are open source and can be found here and here; the inlined version requires simple initialization, this has to happen before importing react
after calling initialize(window), react-devtools provides a very useful hook to listen to rendering events:
To see what was rendered, react also provides a Fiber structure:
This is very cool stuff: the Fiber structure contains all sorts of goodies: keys, elementType (React component or an element (<div>, <img />, etc.), props, and just about everything you might want to know about the rendered result. You can, and should definitely, check all that out in the Fiber definition here.
Every Fiber represents a react component, fragment, HTML element, or a text. Fibers are a singly linked list tree structure, meaning that every Fiber in the tree has a link to its’ closest sibling and its’ child. This structure is more space-efficient in memory, but it has implications for traversal. You can read more about it in the Wikipedia article here.
Finally, to understand where exactly in the code each Fiber originates, Codux transpiles the code with transformers, adding information that helps figure out which exact file/component/expression/loop/condition/variable/const/string created the Fiber, or in other words: Codux modifies the code, adding hints that later map each Fiber to a node in the code syntax tree.
How it all comes together for the user
In the video below, you can see the ColorfulList component from before, notice how each of the rendered items appears on the Elements Tree. In addition, all the items originate from a singleitems.map(...) function, so the Elements Tree displays them inside a green lane representing the map function (we call it a “repeater”).
In other words: to visualize the component, the Elements Tree integrates the code, and the rendered result. Also, If you watch the video to the end, you can see that Codux “undetstands” each item, and can edit the specific classNames applied to it, I’ll leave this to another Article.
