Devlog: dependency graph as architecture
Last updated
Time for devlog #4! Last week's was here.
The visual, live development environment I've been working on could not have been built without Gooey, my frontend web framework.
Let's get into how it works.
Gooey: a dependency graph engine
Though it calls itself a "focused and flexible frontend web framework," Gooey is really a dependency graph engine.
In this dependency graph, nodes are either pieces of data or calculations, and the edges represent the flow of this data.
To demonstrate, here's a little interactive tool to visualize the angle of a line segment:
The full code can be found here.
import Gooey, { calc, field, mount, ref } from '@srhazi/gooey';
import type { Component, Field } from '@srhazi/gooey';
const NumericInput: Component<{
value: Field<number>;
id?: string;
type: string;
min: string;
max: string;
}> = ({ value, ...rest }) => (
<input
{...rest}
value={value}
on:input={(e, el) => value.set(el.valueAsNumber)}
/>
);
const Visualization: Component = (props, { onMount }) => {
const canvasRef = ref<HTMLCanvasElement>();
let ctx: CanvasRenderingContext2D | null = null;
// Input data, just X & Y coordinates
const x = field(40);
const y = field(30);
// Derived data, calculations that show angle & degrees
const angle = calc(() => {
const baseAngle = Math.atan(y.get() / x.get());
if (x.get() < 0) {
return baseAngle + Math.PI;
}
if (y.get() < 0) {
return baseAngle + 2 * Math.PI;
}
return baseAngle;
});
const degrees = calc(() => (angle.get() * 360) / (2 * Math.PI));
// A calculation which is re-evaluated when dependencies change, but doesn't return anything.
// This can be used to subscribe to multiple things at once.
const render = calc(() => {
update({
x: x.get(),
y: y.get(),
angle: angle.get(),
});
});
onMount(() => {
if (canvasRef.current) {
ctx = canvasRef.current.getContext('2d');
}
// By subscribing to our render calculation, it becomes "alive" and is recalculated when the x/y/angle changes
return render.subscribe(() => {});
});
const update = ({
x,
y,
angle,
}: {
x: number;
y: number;
angle: number;
}) => {
if (!ctx) {
return;
}
ctx.fillStyle = '#F4F2F4';
ctx.rect(0, 0, ctx.canvas.width, ctx.canvas.height);
ctx.fill();
ctx.save();
ctx.scale(ctx.canvas.width / 200, ctx.canvas.height / 200);
ctx.translate(100, 100);
// Draw outline for angle
ctx.beginPath();
ctx.save();
ctx.strokeStyle = '#C0C000';
ctx.setLineDash([2.5, 2.5]);
ctx.arc(0, 0, 20, 0, 2 * Math.PI, angle > 0);
ctx.stroke();
ctx.restore();
// Draw orange arc for angle
ctx.beginPath();
ctx.strokeStyle = '#C0C000';
ctx.fillStyle = '#808000';
ctx.arc(0, 0, 20, 0, -angle, angle > 0);
ctx.stroke();
ctx.lineTo(0, 0);
ctx.fill();
// Draw red line for X axis
ctx.beginPath();
ctx.strokeStyle = '#800000';
ctx.fillStyle = '#C00000';
ctx.moveTo(0, 0);
ctx.lineTo(0 + x, 0);
ctx.stroke();
// Draw green line for Y axis
ctx.beginPath();
ctx.strokeStyle = '#008000';
ctx.fillStyle = '#00C000';
ctx.moveTo(0 + x, 0);
ctx.lineTo(0 + x, 0 - y);
ctx.stroke();
// Draw blue line for hypotenuse
ctx.beginPath();
ctx.strokeStyle = '#0000C0';
ctx.fillStyle = '#000080';
ctx.moveTo(0, 0);
ctx.lineTo(0 + x, 0 - y);
ctx.stroke();
// Red X point
ctx.beginPath();
ctx.strokeStyle = '#800000';
ctx.fillStyle = '#C00000';
ctx.arc(0 + x, 0, 4, 0, 2 * Math.PI);
ctx.stroke();
ctx.fill();
// Green Y point
ctx.beginPath();
ctx.strokeStyle = '#008000';
ctx.fillStyle = '#00C000';
ctx.arc(0 + x, 0 - y, 4, 0, 2 * Math.PI);
ctx.stroke();
ctx.fill();
// Blue center point
ctx.beginPath();
ctx.strokeStyle = '#0000C0';
ctx.fillStyle = '#000080';
ctx.arc(0, 0, 4, 0, 2 * Math.PI);
ctx.stroke();
ctx.fill();
ctx.restore();
};
return (
<div>
<style>
{`
.triangle-canvas {
position: relative;
cursor: crosshair;
width: 400px;
height: auto;
}
.triangle-canvas-controls {
display: grid;
padding: 16px;
gap: 16px;
grid-template-columns: auto 1fr;
}
.triangle-angle-input {
text-align: right;
}
`}
</style>
<canvas
class="triangle-canvas"
ref={canvasRef}
width="800"
height="800"
on:mousemove={(e, el) => {
const box = el.getBoundingClientRect();
const mouseX =
2 * ((e.clientX - box.left) / box.width - 0.5);
const mouseY =
2 * ((e.clientY - box.top) / box.height - 0.5);
x.set(Math.round(mouseX * 1000) / 10);
y.set(Math.round(-mouseY * 1000) / 10);
}}
on:touchmove={(e: TouchEvent, el) => {
if (e.targetTouches[0]) {
e.preventDefault();
const box = el.getBoundingClientRect();
const touchX =
2 *
((e.targetTouches[0].clientX - box.left) /
box.width -
0.5);
const touchY =
2 *
((e.targetTouches[0].clientY - box.top) /
box.height -
0.5);
x.set(Math.round(touchX * 10000) / 10);
y.set(Math.round(-touchY * 1000) / 10);
}
}}
/>
<fieldset class="triangle-canvas-controls">
<legend>Controls</legend>
<label for="x-value">X Value</label>
<NumericInput
id="x-value"
type="number"
value={x}
min="-100"
max="100"
/>
<span />
<NumericInput type="range" value={x} min="-100" max="100" />
<label for="y-value">Y Value</label>
<NumericInput
id="y-value"
type="number"
value={y}
min="-100"
max="100"
/>
<span />
<NumericInput type="range" value={y} min="-100" max="100" />
<label for="angle">Angle</label>
<input
class="triangle-angle-input"
id="angle"
type="text"
value={calc(() => `${angle.get().toFixed(3)} radians`)}
readonly
/>
<label for="degrees">Degrees</label>
<input
class="triangle-angle-input"
id="degrees"
type="text"
value={calc(() => `${degrees.get().toFixed(3)} degrees`)}
readonly
/>
</fieldset>
</div>
);
};
const angleViz = document.getElementById('angleViz');
if (angleViz) {
mount(angleViz, <Visualization />);
}
The most important part of that code is its data model.
It is built from Fields and Calculations, which form a dependency graph.
const Visualization: Component = (props, { onMount }) => {
// ...canvas code omitted...
// Input data, just X & Y coordinates
const x = field(40);
const y = field(30);
Here we have a Visualization component—a render function that creates 2 fields: x and y.
Fields are like containers, they hold a piece of data that be swapped out over time.
// Derived data, calculations that show angle & degrees
const angle = calc(() => {
const baseAngle = Math.atan(y.get() / x.get());
if (x.get() < 0) {
return baseAngle + Math.PI;
}
if (y.get() < 0) {
return baseAngle + 2 * Math.PI;
}
return baseAngle;
});
const degrees = calc(() => (angle.get() * 360) / (2 * Math.PI));
And here we have two calculations defined: angle and degrees.
Calculations are like derived fields, they produce a value that depends the data read.
Both fields and calculations live on the dependency graph as nodes.
When a calculation's function reads a piece of data, an edge is added to the dependency graph going from the node read to the node doing the reading.
- When
anglecallsy.get()andx.get(), two edges are added:y -> angle&x -> angle - When
degreescallsangle.get(), one edge is added:angle -> degrees
Calculations don't always need to return a value:
// A calculation which is re-evaluated when dependencies change, but doesn't return anything.
// This can be used to subscribe to multiple things at once.
const render = calc(() => {
update({
x: x.get(),
y: y.get(),
angle: angle.get(),
});
});
Here, a render calculation reads some data from the fields and calculations it depends on (x, y, and angle), and
passes their values to an update() function which does the work of drawing to the canvas.
This may feel a bit like an useEffect in React, except you don't need to list out all the dependencies.
If we draw this dependency graph out as a diagram, we get this visual:
Processing the dependency graph
When Gooey sees that an authoritative piece of data has changed (i.e. x.set(10)), the corresponding node is marked as
"dirty" and an asynchronous task is scheduled to process the graph (via
queueMicrotask).
Dirty nodes are processed in topological order and may mark their downstream neighbors as "dirty".
When a calculation is processed after being marked as dirty, its function is called again. If its return value differs, it marks its downstream neighbors as dirty.
Always in order
Gooey always processes the graph in topological order—even if nodes and edges in the graph change as nodes are processed!
This is tricky, and may cause a node to be processed multiple times. For example, a calculation's function may be called multiple times if it gains a dependency by reading a new node which happens to depend on a dirty value.
To manage this in-order processing, Gooey maintains a linear ordering of all nodes in its dependency graph. Think of this ordering as an array of nodes, where all of the edges go left-to-right.
If an edge is added which goes in the opposite direction, the ordering is now wrong. To correct this, the out-of-order slice of that array is reordered to maintain the left-to-right invariant.
Always memoized
Because the dependency graph is processed following the flow of data, calculations can be stateful: they hold onto their function's return values until marked as dirty.
This means that any calc() encountered in Gooey is automatically memoized based on the data it has read. It's a bit
like a useMemo in React, except you also don't need to list out all the dependencies.
Not just single-value fields
Collections in Gooey are like arrays of Fields: each item in the collection lives in the dependency graph as a node.
Unlike fields, collections may be subscribed to in a more array-oriented way. Collections have a subscribe method for observing changes.
Similar to how calculations are the derived counterpoint to authoritative fields, views are the derived counterpoint to authoritative collections.
- Want a derived collection where each item is transformed? Use .mapView()
- Want a derived collection where each item is conditionally present? Use .filterView()
- Want both? Use .flatMapView()
Binding nodes to UI
Gooey also allows you to bind nodes in the dependency graph directly to the DOM via JSX.
In our visualization, we can see this in how the input text boxes have their value property bound directly to the
result of a calculation:
<input
class="triangle-angle-input"
id="angle"
type="text"
value={calc(() => `${angle.get().toFixed(3)} radians`)}
readonly
/>
And if JSX is returned by a calculation, it will be rendered to the DOM in place and replaced by its contents when the calculation is processed.
A dependency graph as an architecture
If all of your application's data is built from these fields and calculations (and collections, etc...), then your application has a clear defined separation between authoritative data and derived data.
A funny thing happens when you start structuring your application this way.
Since this state can live anywhere in your application, you stop needing rules around the structure and subscriptions of your "data store".
You can model your application's state naturally, using field(), calc(), and collection(). And regardless of where
that state is placed or how it's accessed, it will be automatically updated and memoized efficiently.
For example, the development environment I'm building roughly models a JavaScript module.
There's a ModuleModel class which holds its name bindings in a collection of ModuleNameBinding instances. These name
bindings have fields representing the name (so names can change without impacting the value), and value at that binding.
class ModuleNameBinding {
name: Field<string>;
value: CellModel | FunctionModel | ...;
// ... implementation code ...
}
class ModuleModel {
bindings: Collection<ModuleNameBinding>;
// ... implementation code ...
}
This allows name bindings to be dynamically added/removed from the module, and bindings to be renamed / their value changed.
Similarly, "Cells" in the application are instances of the CellModel class. They hold their dynamic properties in a
collection of property bindings which hold AtomModel class instances, which are "dynamic property objects" which
contain source code that is evaluated to produce their result.
class AtomModel {
sourceCode: Field<string>;
value: Calculation<any>; // sourceCode evaluation result
// ... implementation code ...
}
class CellModelProperty {
name: Field<string>;
value: AtomModel;
// ... implementation code ...
}
class CellModel {
properties: Collection<AtomModel>
// ... implementation code ...
}
Since calculations automatically add dependencies when they're read, there's no need to worry about adding subscriptions to this data. If the underlying data changes, the Calculation will be re-evaluated.
This is even true for AtomModel which has user-provided source code. If the evaluation of that source code happens to
read any data in the dependency graph, it will be re-evaluated if (and only if) the dependencies change.
This gives a ton of flexibility to the data model over time. Since calculations don't need to worry about subscribing to changes, fields can change freely over time and calculations don't need to change at all!
It's a lot more forgiving than other state management solutions, which can be brittle and hard to change over time.