Introducing Gooey

For the past year or so, I've been working on a new, experimental JavaScript web UI framework. It's called Gooey.

Here's why it exists and what makes it interesting. Grab a cup of tea or beverage of choice and get comfy, there's a lot of info here.

After working with React professionally for years, I've grown to both love it and wince at it. There are many critiques already written about React as a tool, but without an alternative, these critiques are just talk. So my main objective in building a new framework was to push the boundaries to see what alternative idioms are possible.

That being said, to state this as plainly as possible: React is a perfectly fine framework to use and I recommend new applications consider using it. Gooey is not meant to replace React or any other framework, but rather contribute to the set of ideas and idioms that form the frontend landscape.

Distilled to its essence, a UI framework ought to streamline the complexity of building applications, leading to simpler, more flexible, and more robust applications. This complexity is traditionally broken down into three responsibilities when viewed through the Model-View-Controller architectural pattern:

• The View is what you see. Its responsibility is to ensure the DOM reflects the application state defined in the Model.
• The Controller is what you interact with. Its responsibility is to observe user input and respond by altering the Model.
• The Model is the underlying state machine you are presenting. Its responsibility is to notify the View of changes. Its complexity is endless and depends on the application you are building.

In web applications, there's a fourth hidden actor here, the Browser, which is ultimately what users interact with. The Controller interprets user actions through how they're exposed within the Browser's DOM API:

Many of the existing web frameworks focus on some subset of these categories, and many encourage sharing of the responsibilities of the View and Controller.

Gooey is focused on both the Model and the View.

Its Model interface allows authors to define authoritative mutable state and derived read-only state that automatically and efficiently recalculates itself when needed. Its View interface encourages View-Controller abstractions, which may use DOM events / APIs directly and may shape the overall tree structure of the UI. Both the Model and View use the same mechanisms to bind state to the DOM and to create derived read-only state.

Gooey is like a spreadsheet engine, where cells of data/UI automatically get updated when the cells' dependencies are changed. In fact, it's so much like a spreadsheet engine that it takes surprisingly little code to build a spreadsheet with it!

Build systems and web UI frameworks have a lot in common. Both involve figuring out when a noun (build target vs the DOM) should be updated (rebuilt vs DOM mutated) when its dependencies (files vs application state) change.

About a year ago, after spending lots of time in React optimizing code with useMemo and fighting with useEffect, I couldn't shake the feeling: this is all so repetitive. I was spending time fighting against a problem that should be able to be avoided entirely: unnecessary re-rendering and its side effects.

This struggle reminded me of both make and tup, the build systems. They're both build tools that rebuilds files when the files' dependencies (source files or other build targets) change.

The way tup figures out what exactly a file depends on is both very clever and very flexible: it uses a fuse filesystem to observe all the files accessed while running the command(s) that produce the build target. These accessed files are assumed to be dependencies of the build target. After a single initial build, tup holds a complete dependency graph for all files that contribute to the build. Given a file change, it knows exactly what commands to run and in which order to run them. And since it's observing all files accessed while building, things normally forgotten (like the version of the compiler you used / the globally installed library / the shell script's global profile files / etc...) all are part of the dependency graph, so your project will rebuild if anything that could possibly contribute to the build has changed (you update your compiler, operating system, a shared library, etc...).

Contrast this with make, which requires you to explicitly list out the dependencies of each build target. This is a frustrating and tedious experience. You can get incorrect builds when a dependency is missed or cause performance problems when an unused dependency is not removed. This can be alleviated by "generating" your Makefiles with additional tooling, like automake, which can output a list of all the dependencies your build target needs.

Sound familiar? Let's substitute a few words:

Contrast this with React hooks, which requires you to explicitly list out the dependencies of each hook callback function. This is a frustrating and tedious experience. You can get incorrect behavior when a dependency is missed or performance problems when an unused dependency is not removed. This can be alleviated by "generating" your hook dependencies with additional tooling, like eslint-plugin-react-hooks, which can output a list of all the dependencies your hook callback functions needs.

React is very much tied to make's view of the world: it forces you to list all your dependencies. And it's is going even further in the direction of automake: it aims to "solve memoization for good with a compiler".

Personally, I'd prefer to build and work with a system like tup (learn things by observing what they do) rather than a system like automake (infer things by enforcing code structure and building several layers of linters/compilers), and I suspect you would too.

In addition to toying with the idea of building a build system, but for UI, there were a few ideas I wanted to play around with: Lists, Teleportation, and "Observers".

Lists are everywhere: they're the building block of UI. Pretty much everything we build is a collection of lists with various filters / sorting mechanisms applied. There should be few if any barriers for application authors to create a list of UI elements from a list of data. When the data has items added/removed or gets resorted, the list of UI elements should be updated efficiently and with as little intervention as possible. In particular, I wanted to see if this was possible without needing the special key prop equivalent in React.

Moving a DOM element from one location to another (Teleportation) is one of those things that was easy without React, and worryingly hard with it. It is technically possible within React, but involves deep magic. Why not make it possible to detach DOM element from a location and reattach the same DOM element to a different location? And if that's possible, could it be possible for a temporarily detached/hidden DOM subtree to continue to update in the background, so that when it's reattached, the entire subtree does not need to be recreated from scratch at the time of reattaching?

Sometimes you want a component abstraction that adds behavior to children without them knowing about it. Take for instance a tooltip component: it'd be great to be able to write <Tooltip content="Hello!"><MyComponent /></Tooltip> in order to have a Hello! tooltip revealed once anything rendered by MyComponent is hovered over—regardless of how MyComponent is implemented. "Observers" are not the right word for this (please let me know if you think of a better name), but this is a common pattern I've seen in various applications that's quite hard to accomplish well.

In Gooey, these are solved via myCollection.mapView(item => renderJSX()), myJsx.retain() & myJsx.release(), and IntrinsicObserver.

Aside from a JSX transformation, Gooey does not and will never depend on a compiler (or compiler "plugin" or recommend using a linter) in order for it to do its job or you to do your job with it. The cost of a compiler is deceptively high in the long term and should be avoided at all costs if possible.

If an API has a strict contract that requires you use a linter in order to not unintentionally break by "calling it in a loop" or "calling it outside of a render function", that's a sign that your API is poorly designed (The Rules of Hooks, I'm looking at you).

Don't get me wrong, compilers and linters are fantastic tools. I just think that requiring a custom compiler or linter to use your "library" is deceptive. If your library needs static tooling to work right, it's not a library anymore, it's an ecosystem, which comes with lots of hidden complexity and weight.

Under the hood, there are some parts of Gooey that make me very excited. These lie within two areas: the data layer and the view layer.

Gooey's data model is the same as the core of a build system or spreadsheet: it builds and maintains a directed graph holding all of the data dependencies. When a piece of data changes, the change can propagate through the graph by processing (recalculating) each vertex.

Gooey takes a different approach to Model data: while most frameworks encourage/enforce immutable data (and fall apart poorly when mutable data leaks in), Gooey wholeheartedly embraces mutability.

Application state in Gooey is flexible, but universal: there is no distinction between state that lives outside the confines of a UI component and state that is local to a UI component. Authors can create mutable (authoritative) state and read-only (derived) state. Gooey ensures that this derived state is recalculated in the correct order. Dependencies that form cycles are automatically detected and can be handled gracefully by each piece of derived state.

Mutable (authoritative) state:

• A field is a variable that can hold a single value.
• A model is a dictionary object that holds any number of members. It acts just like a plain old JavaScript object.
• A collection is a list that holds any number of items. It acts just like a plain old JavaScript array.

Read-only (derived) state:

• A view is a derived list of items that are transformed or filtered items within a source collection.
• A calculation is a function that takes no parameters and returns a single value. While a calculation is executing, its dependencies are automatically tracked. Its resulting calculation is automatically memoized.

This derived state is automatically recalculated once any of its dependencies are changed.

In code, here's what this all looks like:

import { model, collection, field, calc } from '@srhazi/gooey';

// Models act like plain old JavaScript objects
const state = model({
  name: 'Gooey',
  description: 'an experimental UI framework',
});

// Collections act like plain old JavaScript arrays
const numbers = collection([2, 3, 5, 7, 11]);

// Calculations are recalculated when the derived data changes
// Here, when the numbers collection changes, the sum will be recalculated
const sum = calc(() => numbers.reduce((acc, num) => acc + num, 0));

// Views can be created from collections via a transformation function
const squared = numbers.mapView((num) => num * num);

// Views act just like read-only collections
// Here, when the squared collection changes, the sum will be recalculated
const squaredSum = calc(() => squared.reduce((acc, num) => acc + num, 0));

// Here's a calculation that summarizes all of the above in a message
const summary = calc(() => `
${state.name}: ${state.description}
${'='.repeat(state.name.length + 2 + state.description.length)}

Numbers: ${numbers.join(', ')}
Numbers sum: ${sum()}
Squared: ${squared.join(', ')}
Squared sum: ${squaredSum()}
`);

// Calculations are called like functions
console.log(summary());

// Every time this summary changes, we'll be notified of it:
summary.subscribe((err, value) => {
  if (!err) console.log(value);
  else console.error('Error', err);
});

// Everything is exported to window! Open up devtools and run:
// - demo.numbers.push(13)
// - demo.numbers[0] = 100;
// - demo.state.description = 'a new thing';
(window as any).demo = { state, numbers, sum, squared, squaredSum, summary };

This may sound a bit familiar if you've used SolidJS, S.js, Preact Signals or various other libraries that tend to call themselves "Reactive" or have "Automatic dependency tracking" qualities. Gooey is similar, but takes this a step further in how these are batched, processed, and cached. More on this later.

The processing model of Gooey involves keeping a global dependency graph and traversing it in topological order, while maintaining the topological order as the graph changes over time.

This is best demonstrated visually with an example, here's a very rudimentary two-key system, where two checkboxes need to be checked prior to revealing & activating a button:

import Gooey, { calc, model, mount } from '@srhazi/gooey';

const App = () => {
  const state = model({ left: false, right: false });
  const isLocked = calc(() => !state.left || !state.right);

  return (
    <fieldset>
      <legend>Two Key Lock</legend>
      <p>
        <label>
          Left Key:{' '}
          <input
            type="checkbox"
            checked={calc(() => state.left)}
            on:input={(e, el) => {
              state.left = el.checked;
            }}
          />
        </label>
      </p>
      <p>
        <label>
          Right Key:{' '}
          <input
            type="checkbox"
            checked={calc(() => state.right)}
            on:input={(e, el) => {
              state.right = el.checked;
            }}
          />
        </label>
      </p>
      <p>
        {calc(() =>
          isLocked() ? (
            'System locked'
          ) : (
            <button on:click={() => alert('Activated!')}>
              Activate
            </button>
          )
        )}
      </p>
    </fieldset>
  );
};

mount(document.getElementById('lock-example')!, <App />);

The application itself has a few pieces of state and a few calculations:

• Two booleans representing the state of each checkbox (state.left state.right)
• A calculation to determine if the two key system is locked (either the left key or right key are not checked) (isLocked)
• Two calculations bound to the "checked" prop of each checkbox input (left:checked, right:checked)
• One calculation to render either a "System locked" message or a button to activate the system (launch:button)

This forms a dependency graph, which can be visualized as a directed graph where vertices represent model fields & calculations and edges pointing in the direction of the flow of data.

Here's a visualization of the dependency graph, showing how it is processed in response to the state.left checkbox becoming checked:

The flow of updated data flows in the direction of the arrows, so changes propagate from state.left to left:checked & isLocked and then recursively propagate in a similar fashion.

Since this is a dynamically determined set of dependencies, it represents the dependencies collected at runtime: Initially no arrow between right:checked and isLocked because left:checked is false and the implementation short circuits before reading right:checked. Once isLocked is recalculated, it gains a dependency on the right:checked boolean.

The ordering of propagating and recalculating changes is the topological ordering of the graph. In order for Gooey to efficiently process the graph, so that it visits vertices in the correct order, it must maintain a topological ordering of the dependency graph while edges and vertices are actively being added and removed. This is tricky!

Gooey uses a possibly novel combination of the Pearce-Kelly Dynamic Topological Sort Algorithm for Directed Acyclic Graphs and Tarjan's Strongly Connected Components Algorithm to maintain a topological sorting and strongly connected component detection in a directed graph that is fully dynamic (edges added and removed) and may contain cycles.

The comment preamble in the code is a good overview of how this all works.

This is a bit of a bold claim, so we'll see how it pans out. But all calculations are automatically (and perfectly) memoized. Multiple calls to obtain the value from a calculation will return the same value if the underlying data has not changed. This is possible because Gooey keeps track of the entire dependency graph, and as such knows when to evict the cached value and recalculate the result.

Gooey's view layer binds the result of calculations and collections to the DOM. Authors mark all of the areas where the UI is dynamic, and Gooey will recalculate those dynamic areas when their dependencies change. It allows for UI to be rendered and updated while detached from the DOM, and allows for portions of UI to move locations within the DOM without having their underlying DOM nodes / contained state recreated.

Calculations are like cells in a spreadsheet, which contain formulas (functions) that reference other calculations or pieces of state. Calculations may be bound to intrinsic element attributes or properties (like a div's class attribute or an input's checked property), so that their DOM attributes reflect the produced values over time. Calculations can also be bound as a JSX child node, which means it is bound to the DOM and updated automatically when its referencing data changes. So calculations are just like spreadsheet cells, but for UI! But more powerful, as calculations that produce JSX can also define inner calculations, which themselves will get bound to the DOM & updated appropriately.

This means that when writing UI, you explicitly mark all of the places where your UI is dynamic. Everything else is static and unchanging.

import Gooey, { Component, calc, model, mount } from '@srhazi/gooey';

const App: Component = () => {
  const state = model({ clicks: 0 });

  return (
    <>
      <p>
        Click count: <b>{calc(() => state.clicks)}</b>
      </p>
      <button on:click={() => state.clicks++}>Click me</button>
    </>
  );
};

mount(document.getElementById('button-clicker')!, <App />);

One pleasant benefit of this explicit marking of dynamic areas in your UI is that it decouples functions that determine the tree structure of your DOM from functions that produce the values that get bound to the DOM. In other words, render functions get called exactly once, and the dynamically bound calculations get called whenever they need to be updated.

This means there's no need to keep references across re-renders, because there aren't re-renders. This is starkly in contrast with React/Redux where performance regressions can easily be caused by unintentionally creating a new object reference on each re-render, which can unintentionally trigger a cascade of needless re-renders.

Collections of JSX are also renderable as JSX nodes. A collection of <li> JSX elements could be placed as a child of a <ol> element and you'll get an ordered list that reflects the order of items in the collection. As items are added to the collection, removed from the collection, or even if the items are moved/reordered, the underlying DOM nodes will not be recreated; they'll just be placed in the correct location based on the position in the collection itself.

Consider the following dice rolling application. A list of items is projected onto rows in the middle of a list.

import Gooey, { model, collection, calc, mount } from '@srhazi/gooey';

const DICE_SIDES = [2, 4, 6, 8, 10, 12, 20];
const scores = collection<{ sides: number; roll: number }>([]);
const state = model({ total: 0 });

const onRoll = (sides: number) => {
    const roll = 1 + Math.floor(Math.random() * sides);

    state.total += roll;
    scores.unshift({ sides, roll });
    if (scores.length > 10) {
        scores.splice(10, scores.length - 10);
    }
};

mount(
    document.getElementById('dice-roller')!,
    <>
        <p>Dice Roller</p>
        {DICE_SIDES.map((sides) => (
            <button on:click={() => onRoll(sides)}>d{sides}</button>
        ))}
        <ul>
            <li>Roll results:</li>
            {scores.mapView(({ sides, roll }) => (
                <li>
                    d{sides} rolled <strong>{roll}</strong> at {new Date().toLocaleTimeString()}
                </li>
            ))}
            <li>Total: {calc(() => state.total)}</li>
        </ul>
    </>
);

Observe how the timestamp of the items rendered in the list are unchanged, even as items are added above/below them. This is because these portions of the DOM are not marked as dynamic (not wrapped in a calc call), so they will never be re-rendered.

The "behind the scenes" data structures used when rendering JSX are called RenderNodes. These represent the underlying tree structure that is used by Gooey to render UI to the DOM. There are many different types of RenderNodes, but they roughly correspond to JSX nodes: TextRenderNode to render plain text, IntrinsicRenderNode to render native DOM elements, FunctionComponentRenderNode to render components, CalculationRenderNode to render calculations, etc...

Let's step through the typical lifecycle of an intrinsic element render node that gets mounted, like the <h1> JSX node rendered here:

import Gooey, { mount } from '@srhazi/gooey';

const unmount = mount(document.body, <h1>Hello, world!</h1>);

setTimeout(unmount, 1000);

When the <h1>...</h1> expression is evaluated, a RenderNode is created in the "Dead" state. It only really has information about what it will become (the h1 tag name, an empty set of props, and an array of child RenderNodes, also in the "Dead" state).

Once passed to the mount function, this RenderNode is internally .retain()ed. This is because mount "activates" the RenderNode to allow it to start rendering. It remains retained until the unmount function (returned from mount) is called. At this time, the h1 RenderNode needs to render itself, so it allocates an HTMLHeadingElement, binds props, retains its children (rendering them), and asks its children to .attach() themselves to their parent.

When the h1 RenderNode asks its children to .attach() themselves to their parent, the h1 RenderNode begins accepting a stream of DOM Operation Events that it will process and handle. These events are things like: "add Element X at index N", "remove the node at index N", "move the node at index X to index Y". This allows the h1 RenderNode to do some arithmetic accounting and keep track of the position of all of its children's emitted DOM nodes.

At this time, the h1 RenderNode has been retained by mount (in the "Rendered" state), the h1's children have all been retained and attached to the h1 RenderNode (in the "Attached" state). But the h1 RenderNode has not yet been attached to anything. The mount function then asks the h1 RenderNode to attach itself, opening up the same stream of DOM Operation Events. This stream of events allows mount to attach the emitted/removed/moved DOM nodes to document.body. The h1 RenderNode is now in the "Attached" state.

After mount has attached the h1 RenderNode to the DOM (at document.body), it then tells the h1 RenderNode that it has been mounted by calling .onMount(). It then calls its children's .onMount() method, and so on and so forth until the signal has propagated through the DOM. Nothing happens at this time in this example, but component lifecycle methods, ref callbacks, and other things happen during this phase. Now the h1 RenderNode and all its children are in the "Mounted" state.

After 1000 milliseconds have passed, unmount() is called, which essentially performs the above sequence in reverse: .onUnmount() is called propagating through the RenderNode tree; .detach() is called, which tells the h1 to immediately emit DOM Operation Events to remove any remaining DOM nodes and stop emitting further events; and .release() is called, which causes the h1's refcount to go to 0, so it can release its children and clean up any remaining state.

One thing to underline: a node delegates the act of placing its DOM nodes in the correct location to its parent. This is how Gooey avoids needing a key prop. Since each node knows how many children it has and how many DOM nodes they've emitted, each node can "shift" the index of the DOM Operation Events emitted by a child by the number of DOM nodes that have been emitted by earlier children. Additionally, RenderNodes don't have to emit any DOM Operations themselves: they can also act as "filters"/"transformers" of DOM Operation Events.

This also gives us a bit of a superpower. It's possible to create RenderNodes that can add functionality to the DOM nodes emitted by their children without knowing anything about their children. For example, you could build a RenderNode that listens to the focusin and focusout events on all of the DOM nodes that pass through it, so it can maintain knowledge if the user has focus contained within a specific subtree.

Also, if it wasn't clear, it's totally possible for different lifecycles to occur! A long-lived heavy component could be .retained() at the start of an application so it renders itself and its children in the background, but doesn't get attached until the user gets to a certain state. Then it could be attached, mounted, unmounted, detached; and then again attached (maybe to a different location in the DOM), mounted, unmounted, and detached any number of times until the end of the application. All while none of its owned DOM nodes change unless they're recreated.

I have no idea. Well, that's not true—I'm going to work on writing some proper documentation for Gooey, but after that, I'm not quite sure.

Hopefully someone will find the ideas here interesting. Please let me know if you do!

Update: There's now a website with documentation!