Patch cord design: how to give your GUI an analog look
Imagine a GUI widget that leverages the full power of control flow without the user ever having to write a line of code... “Patch cords” are a powerful and under-appreciated strategy for bringing interaction to your web app.
In the browser, you’re probably most familiar with them from flowchart apps like Draw.io and “mind mappers” like Coggle.it and MindNode. But they have potential in game design, learning tools, and basically any use case in which the user should have scripting-language-like control over an app via symbolic graphic elements.
A patch cord is a visual linkage between a frame or panel and one or more additional visual elements. The concept comes from analog audio routing used in early telephone switchboards, which were based on even earlier telegraphy switchboards used in the early nineteenth century.
But the direct antecedent to many digital patch-cord-stye interfaces is the analog modular audio synthesizer. Modular synthesizers move voltages through a series of stages or modules like plumbing in a building.
A signal is usually output by an oscillator and then passed through a series of transformations like the ball tumbling down through a Rube Goldberg machine. As it passes through different modules, the voltage can be split off into multiple separate streams and can be used not only as an audio signal but also as control voltages or logic conditions for the modules themselves.
The “pipes” that make up this “plumbing” between modules are called “patch cables” or (or “patch cords”), and in an analog synthesizer they are physical wires arbitrarily plugged into input and output jacks on the modules at the discretion of the performer, yielding nearly infinitely customizable circuits.
The first digital audio software didn’t have a GUI, of course. (People have been making sounds with computers since well before the first Fortran compiler was released in 1957.) But with the addition of a GUI, many software synthesizer emulators carried over the idea of patching into the digital domain, preserving “data flow” programming in the era of personal computing.
Two programming environments widely used for audio and video synthesis today — Pure Data (Pd) and Max/MSP — are built around the idea of patching together visual elements representing functions and control flow.
Another example is Scratch, MIT’s visual programming language aimed primarily at teaching programming concepts to children, which arguably also relies on a patching modality, although the block-based GUI does not actually use patch cords.
Build a Flowchart App with Interact.js in Under 30 Minutes
Over the course of this tutorial, I’ll show you how to build this cool little flow-chart application in under 30 minutes!
To do this we’ll be using a lightweight JavaScript library called Interact.js to simplify drag-and-drop behavior. (If we were building in React we could also use React DnD, but I’ll save that for a future tutorial.)
I’ve setup the CodePen with the Interact.js CDN files as well as randomColor which I’ll be using to prettify the app a bit.
We’ll start by just using the methods from Interact’s quick-start example almost verbatim. However, I’ve set the inertia method to false, and I’ve also removed the boundary box condition for simplicity.
interact('.draggable')
.draggable({
inertia: false,
autoScroll: true,
onmove: dragMoveListener
});function dragMoveListener (event) {
var target = event.target;
x = (parseFloat(target.getAttribute('data-x')) || 0) + event.dx,
y = (parseFloat(target.getAttribute('data-y')) || 0) + event.dy;
target.style.webkitTransform =
target.style.transform =
'translate(' + x + 'px, ' + y + 'px)';
target.setAttribute('data-x', x);
target.setAttribute('data-y', y);
}
Now if we create a node on the DOM and give it the class draggable we’ll be able to move that <div> around the screen with the curser!
Since any flowchart will require more than a single state, we need a way for the user to add child nodes to the DOM. We’ll make a button which calls a createDiv() function.
<div class= "holder">
<div id="button">
<button class="button" onClick="createDiv()">new state</button>
</div>
<div id="main">
<div class="draggable">
<input type="text" id="text" value="text">
</div>
</div>
</div>The createDiv() function creates a new <div> as a child of the main node and gives it the draggable class name. We can also set the background color of the element to a random color here:
function createDiv() {
let div = document.createElement('div');
document.getElementById('main').appendChild(div);
div.className = 'draggable';
div.style.backgroundColor = randomColor({luminosity: 'light'});
}There are a couple of other things we might want to do in the createDiv() function.
Each <div> should have a unique ID so we can refer to it later when we patch between these <div> elements, and we’ll add those IDs as key in an object that we’ll use later:
// Set a unique ID for each div
div.id = "div" + divCounter;
divs[div.id] = [];
divCounter += 1;We also don’t want the new <div> to appear at the origin. That would cover up the button. Also, we need this.focus(), which allows an inner div to be edited so we can add text. Very important for any flowchart!
// Move the new div down 70px from the origin
div.style.transform = "translate(0px, 70px)";
div.setAttribute('data-y', 70);
div.setAttribute('onclick', 'this.focus()');And finally, we can set the size of the child <div> with a CSS variable, meaning that we can avoid using “magic numbers” in our JavaScript to refer to different edges of the child <div> when it comes time to connecting our patch cords.
// Set the height and width of the div with CSS variables
root.style.setProperty('--height', height + "px");
root.style.setProperty('--width', width + "px");I’ve added a little CSS to make everything look a bit prettier. Here are the results. So good so far!
Building the Patch Cords
To render the patch cords we’ll be using SVG lines. To do this we’ll first add an empty SVG within the “main” <div>. In the CSS we need to set pointer-events to none. This allows us to click “through” the invisible SVG so we can still interact with a <div> “beneath” it. (The SVG’s z-index is also set to an arbitrary high number so that the patch cords will always appear on top of other elements.)
#patchCords {
position:absolute;
width: 100%;
height: 100%;
pointer-events: none;
stroke-linecap: round;
z-index: 100;
}Accounting for Interaction
Like any complex behavior, we can break patching down into a list of possible states. Then we can build listeners and conditionals to respond to each state. Let’s think about what states are involved in interacting with patch cords.
We want to be able to click on an element and have the patch cord be “attached” to the cursor until and unless we drop the cord onto another element. To do that we’ll need a boolean to represent whether a patch cord is active on the cursor or not. Let’s call that variable patchCordActive.
let patchCordActive = false;patchCordActive starts as false because no patch cord is “attached” to the mouse until the user clicks on a <div>.
We can break down the patching behavior into the following states:
patchCordActiveisfalseand user moves one of the child<div>elements.patchCordActiveisfalseand user clicks outside of any<div>.patchCordActiveisfalseand user clicks inside of a child<div>. (Result: creates new patch cord)patchCordActiveistrueand the user clicks inside of the same<div>twice. (drops patch cord)patchCordActiveistrueand the user clicks outside of any<div>. (Result: drops patch cord.)patchCordActiveistrueand the user clicks inside of a different<div>. (Result: connects patch cord)patchCordActiveistrueand the user moves their curser anywhere within the browser window.patchCordActiveisfalse, but<div>elements are connected with patch cords and the user moves one of the connected<div>elements.
Adding Event Listeners
Looking at the list above, it’s clear that we will be dealing with two kinds of user interactions: curser moves and curser clicks. To account for this we’ll need at least two types of event listeners:
window.addEventListener('click', function (event) {
...
});window.addEventListener('mousemove', function (event) {
...
});
Now based on our list of possible patching states we can build conditionals that account for each of the possible behaviors.
window.addEventListener('click', function (event) {// ====== CLICK INSIDE DIV (PATCH CORD INACTIVE) ======
if (document.getElementById("main").contains(event.target)
&& !(patchCordActive)) {
}// ====== CLICK AGAIN INSIDE THE SAME DIV (PATCH CORD ACTIVE) ======
else if (document.getElementById("main").contains(event.target)
&& (patchCordActive)) {
}// ====== CLICK INSIDE A DIF DIV (PATCH CORD ACTIVE) ======
else if (document.getElementById("main").contains(event.target)
&& (patchCordActive)
&& (event.target.id != selectedDiv)) {
}// ====== CLICK OUTSIDE A DIV (PATCH CORD ACTIVE) ======
else if (patchCordActive) {
}// ====== CLICK OUTSIDE A DIV (PATCH CORD INACTIVE) ======
else if (!patchCordActive) {
// pass
}// ====== FALLBACK ======
else {
throw "Patch-cord logic fell through to the error condition!";
}
});window.addEventListener('mousemove', function (event) {
if (patchCordActive) {
}
});
Conditions 1–7 are satisfied here, but we don’t yet have a way to deal with number 8, because we haven’t yet set up a way to keep track of connected patch cords and their vertices. We’ll do that now.
Structuring Patching Data
The patch cords are just SVG paths. That means every time we drag-and-drop one of the child <div> elements, all cords “attached” to that div need to move along with it. We’ll start by creating a structure to manage this connection between nodes on the DOM.
SVGs need two vertices to draw a line, and a patch cord will need to be updated anytime anything happens that affects it. Patch cords are conceptually basically attributes of the <div> elements they connect, so it makes sense to use the <div> IDs as keys to each line.
We have a problem though. What we want is an object that would look something like the following. Both <div> elements serve equally as a key to the vertices of a line.
patchCords = {[div1, div2]: [[x1, y1], [x2, y2]],
...
...
}However, that won’t work. JavaScript objects only allow one key to be used for each value in an object. What we need is something like MySQL’s foreign key which allows different tables to be cross-referenced.
There’s a JavaScript pattern that we can use to do something similar though:
const divs = {
"div0" : ["line0", "line1"],
"div1" : ["line0"],
"div2" : [line1],
"div3" : [],
etc.
}const cords = {
"line0" : [ { "div0" : [x1, y1] }, "->", { "div1" : [x2, y2] } ],
"line1" : [ { "div0" : [x1, y1] }, "->", { "div2" : [x2, y2] } ], etc.
}
Here the divs object uses the IDs of each <div> as a key. The value associated with the key is itself a key to a line in the “cords” object. In the example, div0 is patched to both div1 and div2, while div3 is not currently patched to anything.
The cords object preserves each vertex as an array of JavaScript objects keyed to the line’s parent <div>. This allows the two tables to be cross referenced in both directions. Given only a key to a line, we can get back to the parent <div> elements.
To increase the clarity of the cords object, and to preserve clear directionality, I’m also using a made-up syntax that separates starting and ending line indices with the -> symbol. The ordered array already preserves this information, but the arrow greatly increases readability by clearly showing the directionality of the patch cord, which could be crucial later on.
Building the Conditional Logic
Let’s review how the event-listener logic works. At the highest level we have two event listeners and an Interact.js function which gets called every time a child node is moved.
window
├── click event listener
│ ├── click inside div (patch cord inactive)
│ ├── click again within the same div (patch cord active)
│ ├── click inside a different div (patch cord active)
│ ├── click outside any div (patch cord active)
│ ├── click outside any div (patch cord inactive
│ └── fallback (throw error)
├── mousemove event listener
│ └── mousemove (patch cord active)
├── function dragMoveListener
│ └── drag child div with attached patch cordNow we can simply write the logic for each state.
Click Inside <div> (patch cord inactive)
We want to detect a click inside each child <div> but not inside of the text-editable area of the <div>. Outside of any event listener we can write an arrow function that returns a boolean:
let notTextDiv = (divId) => {return true ? (divId != "inner") && (divId != "flex") : false};Now we setup the conditional to detect the location of the click only when no current patch cord is actively “attached” to the curser.
if (document.getElementById("main").contains(event.target)
&& !(patchCordActive)
&& notTextDiv(event.target.id)
)
{}
Next we get the location of the target <div> and the location of the mouse within the window, and we connect them with a line.
selectedDiv = event.target.id;
currentSelectionX = event.target.getBoundingClientRect().x;
currentSelectionY = event.target.getBoundingClientRect().y;
let mouse_x = event.clientX; // Get the div coordinates
let mouse_y = event.clientY;drawLine(currentSelectionX + (width/2), currentSelectionY + height, mouse_x, mouse_y);
Finally we update our divs object and our cords object. We add the ID of the selected <div> as an index of the new patch cord:
divs[selectedDiv].push(`line${patchCordCounter}`);And then we add one vertex of the patch cord to the cords object using ES6’s Computed Property Names initializer syntax to cleanly set the key of the nested object:
cords[`line${patchCordCounter}`] =
[{[selectedDiv]: [currentSelectionX, currentSelectionY]}, "->"];Finally, we set the patchCordActive boolean:
patchCordActive = true;Click Again Inside the Same <div> (patch cord active)
Clicking a second time inside the same <div> should drop (delete) the newly-created patch cord and set the “patchCordActive” boolean back to false.
else if (document.getElementById("main").contains(event.target)
&& (patchCordActive)
&& notTextDiv(event.target.id)
&& (event.target.id == selectedDiv)
)
{
deleteCord();
divs[selectedDiv].pop();
delete cords[`line${patchCordCounter}`]
patchCordActive = false;
}We detect a second click within the same <div> by comparing the current event target to the selectedDiv variable that we set when a <div> is first clicked.
The deleteCord() function works by simply removing the last (most recently created) child of the patchCords node.
function deleteCord() {
const select = document.getElementById('patchCords');
select.removeChild(select.lastChild);
}Click Inside a Different <div> (patch cord active)
This is at the heart of patching, the ability to drop a patch cord onto a second <div> and connect two <div> elements together! We start by removing the patch cord from the mouse. Then we update the cords object to contain the new starting and ending vertices of the line:
deleteCord();
divs[event.target.id].push(`line${patchCordCounter}`);
cords[`line${patchCordCounter}`].push({[event.target.id]: [event.target.getBoundingClientRect().x,
event.target.getBoundingClientRect().y]});Recall that the syntax of the vertices in the cords object was an object nested within an array:
const cords = {
"line0" : [ { "div0" : [x1, y1] }, "->", { "div1" : [x2, y2] } ]
}So to actually draw the new line we need to drill down into the object to get the first and only value within each nested object.
let first = v => v[Object.keys(v)[0]];let x1 = first(cords[`line${patchCordCounter}`][0]);
let y1 = first(cords[`line${patchCordCounter}`][0]);
let x2 = first(cords[`line${patchCordCounter}`][2]);
let y2 = first(cords[`line${patchCordCounter}`][2]);drawLine(x1[0] + (width/2), y1[1] + height, x2[0] + (width/2), y2[1]);
Finally we reset the value of patchCordActive and increment patchCordCounter, which is used to assign unique IDs to each cord.
patchCordActive = false;
patchCordCounter += 1;Click Outside of Any <div>
Clicking outside of a <div> should drop (delete) the patch cord if patchCordActive is set to true.
Clicking outside of any <div> with patchCordActive set to false has no effect but could play a part is future features. The “else” condition is set as a fallback condition to throw an error.
// ====== CLICK OUTSIDE A DIV (PATCH CORD ACTIVE) ======
else if (patchCordActive)
{
deleteCord();
divs[selectedDiv].pop();
delete cords[`line${patchCordCounter}`]
patchCordActive = false;
}// ====== CLICK OUTSIDE A DIV (PATCH CORD INACTIVE) ======
else if (!patchCordActive) {
// pass
}// ====== FALLBACK ======
else {
throw "Patch-cord logic fell through to the error condition!";
}
The MouseMove Event Listener
The only state which is specific to curser movement (as apposed to a clicks) is triggered by patchCordActive being set to true. When a patch cord is “active” we want one end of that cord to follow the curser until it’s “dropped” onto a <div>.
This is relatively straightforward. Inside a conditional, we set the x2/y2 attribute of the last child of the patchCords node to the current location of the curser.
window.addEventListener('mousemove', function (event) { // If there's an active patch cord, center one end on the curser if (patchCordActive) { let mouse_x = event.clientX;
let mouse_y = event.clientY; document.getElementById("patchCords")
.lastElementChild.setAttribute('x2', mouse_x); document.getElementById("patchCords")
.lastElementChild.setAttribute('y2', mouse_y); }
});
The DragMoveListener
As things stand, we can can drag and drop patch cords onto <div> elements. But if we move those <div> elements with our Interact.js code, the patch cords will just stay in their last location!
That’s because we haven’t yet explicitly built the connection between patch cords and the <div> elements they connect. To do that, we need the “dragMoveListener” function to check the “divs” object (which in tern indexes all connected lines) anytime a <div> is moved.
First we use the ID of the target <div> to lookup an array of all patch cords attached to that <div>.
let movingDiv = target.id;
let movingCords = divs[movingDiv];Then we need to iterate over all attached patch cords and update their starting or ending coordinates to those of the <div> as it moves.
for (const pc of movingCords) {
// Just get one of the cords attached to a div
let oneCord = document.getElementById(pc);if (Object.keys(cords[pc][0])[0] == movingDiv) {
oneCord.setAttribute('x1', x + (width/2));
oneCord.setAttribute('y1', y + height);
}
else if (Object.keys(cords[pc][2])[0] == movingDiv) {
oneCord.setAttribute('x2', x + (width/2));
oneCord.setAttribute('y2', y);
}
else {
throw "oneCord.setAttribute fell through to the
error condition";
}
}
Notice that the conditional is being used to make sure we update the position of the correct end of the patch cord! We need to know which end of the cord is the “from” end and which is the “to” end. The cords object encodes that information as the key to each coordinate pair.
Final Result
Here’s the final result:
Next Steps
Here we have, at best, an MVP of a flowchart app. But I hope this tutorial has shown that even a relatively complex patching behavior can be simple to build in a short amount of time if we break the problem down into smaller steps. To make the flowchart app more fully featured, we would want to build at least some of the following features:
- Add a method to delete child
<div>elements and patch cords. - Allow child
<div>elements to be resized and provide a way to change their shape. - Build functionality to allow patching from specific edges of
<div>elements, not just from the top and bottom. - Implement multi-segmented patch cords.
- Allow patch cords to be styled to show directionality (for example with arrows).
The code for this tutorial can be found on CodePen.io and downloaded from my GitHub. Please comment — or Tweet at me @PleathrStarfish — if you have a suggestion, observation, or a cool fork of my code!
