LIVE 2024

The Tenth Workshop on Live Programming

I strongly believe that visual programming can help software development in many ways. "a picture says more then a 1000 words"... especially if it's interactive. Thats why it's very important that a visual programming needs to embeddable in exisiting code bases and not be a silo. That's what I am focussing on and want to show in this talk.

The visual programming system that I am developing has a generic embeddable core that can be used to develop multiple visual programming languages in a live infinite canvas web environment. The execution engine is separated from the core so that different execution engines can be implemented and deployed with an application (even using different technologies then the visual programming design core itself, which is javascript/web). The results can be viewed and debugged live. How the results look and can be debugged, depend on the targetted application behavior.

In the current demo setups (see here at https://codeflowcanvas.io) there's a webgl shader and a web-client-app flow. In the web-client-app flow the debugging is done using a timeline slider which can be used to view all of the steps that the program executed including its state. In the webgl pixel shader the result is updated live and the timeline slider is used to scrub through the generated shaders when creating/modifying a visual flow.

Some nice features include:

• generic node composition
• replacing nodes with compatible nodes and seeing the impact of the change directly
• insert compatible nodes into connections

Snappets starts from the premise that humans have a huge amount of intuition about how the world works that is embodied in hand movement. The way animators work with conventional software, they will often make motions with their hands to help them imagine movements. Snappets tries to connect the computer directly with that, aiming to make animation more like puppetry or sculpting.

But the idea goes deeper: in programming and mathematics as well as animation, the idea of "transformation A followed by transformation B" is fundamental. To compose two existing transformations in snappets, you take your hand through the desired motion, and press a button - the system will recognize this as "transform composition" and create an entry in a spreadsheet corresponding to that object.

In "Projective Geometric Algebra" (PGA), the system on which Snappets is based, all geometric objects such as points and planes are also transforms (and therefore elements of a "group"). Snappets exposes PGA operations to the user to allow them to solve geometric operations using transform composition and variants of it.

A back-of-the-napkin sketch is one of the most popular ways to explore and share a rough idea. But if the idea is about something dynamic, like a clock or a dance, static sketches can only get you so far. In this video we introduce Inkling, a tablet-based “programmable napkin” that enables users to sketch and play with dynamic systems. The magic of Inkling comes from constraints you can draw and manipulate so quickly and fluidly that you could do it over a beer, in support of a conversation.

Is it effective to develop a live programming environment from within another live programming environment? We have been using Clojure and Clerk, a notebook-like live programming environment, to build reports for our users. We are now in the process of using Clerk to build an interactive report builder.

In the paper and videos below, we share our initial progress on going from a report to a report builder UI. We show a demo in which we use our prototype to build a data transformation pipeline while seeing intermediate results. We also share questions we want to answer and directions we want to explore in the future.

EYG is a programming language with the goal of dramatically reducing complexity around software deployment and software dependencies.

To enable this programs need to be completely predictable in their behavior and EYG has several features to support this predictability.

• Managed effects, all program semantics are independent of the environment the program runs in. As well it is possible to statically analyze any effects a given program will rely on.
• Hash references of AST fragments. Programs always fully described there dependencies via these hashes.
• Closure serialization generation of program fragments which can be sent to other location. This allows static analysis with the type system to extend over multiple execution locations, for example a build machine and web server.
• A minimal AST, it should be easy to re-implement interpreters or compilers in the future to run EYG programs

Debugging and testing compilers has always been a problem. Developers need to create a large number of test cases, invoke the compiler on them, and then verify the results against the expected output. Since compilers usually have many stages, developers might usually find it difficult to know what happens at a certain stage, let alone debugging the stage. Moreover, introducing new features to the compiler may affect existing components intrinsically in ways that traditional tests cannot reveal.

We propose a novel testing method. Each test file is divided into smaller test blocks by empty lines. The test suite sequentially executes each test block and appends the inferred type, evaluation results, diagnostic messages, and debugging information as comments to each block. Developers can prepend flags to each test block to selectively display debugging information from different stages. Most importantly, these comments are committed to the version control system to track changes to the compiler. By monitoring file changes, the test suite can rerun modified files and then write new results back to the file, displaying them to developers and making modifying tests as smooth as using an interactive notebook.

We name this testing method “diff-based testing” because it relies on the diff operation of version control systems. We have implemented this testing system on the MLscript compiler and developed an interactive editor plugin for exploring debugging information.

Manifold is an object model and runtime for quickly arranging and manipulating software components. Modeled after the Unity scene hierarchy, with the spirit of the Plan 9 filesystem, Manifold was made as a kernel for malleable, re-composable software.

This presentation will show examples made with Manifold across several domains of software. It will show how it can be used in authoring tools and editors to give software builders a live and direct connection to their creations. It will also show that when combined with components that take the Unix philosophy to heart, you can just ... throw together software.

A simple ClojureScript environment integrates: self-rewriting plain text, live inspection, inline visualizations, distributed tracing, deployment, declarative process reconciliation, and object capability security.

Programmers assign functions to nodes. ("What should run where?") A process is a function running on a node; it receives all platform capabilities as an argument. Each node reconciles its process state to match the latest specification. Top level forms set their liveness, and may evaluate across the system on every keypress. Stateful expressions preserve node-local values across code changes. The editor is also a node. Data passing through any expression, on any node, is observable as live text in the editor. Visualizations render inline with the code, through functions defined in the same code.

Examples range from structured personal notes through computational documents, up towards open-world integration with hardware, filesystems, servers, browsers, and third-party dependencies to build, run, inspect, and change small distributed systems from within a lightly enhanced text editor.

We discuss the claim that a live literate editor such as Ampleforth [1] can be used as an application builder and platform. We showcase two applications constructed using Ampleforth as existence proofs of this claim: Telescreen, a presentation tool, and Ozymandias, a computational notebook, both created in Ampleforth. Our experience indicates that a self-contained persistent representation for documents, customizable by the application, is essential. Document nesting (aka transclusion) impacts this persistence scheme, and is crucial in ensuring a composable document model that avoids the siloing of applications. The conclusion is that the initial claims made for Ampleforth are valid.

Over the past 34 years, we've grown a community with liveness as its rallying cry, and a dream of making programming direct, visible, tangible, and alive at its heart. But it's striking to realize that -- even as we come together to discuss liveness at LIVE -- we don't really know what liveness is. Definitions of liveness in the literature are often vague, more gestures towards a direction than falsifiable descriptions. When definitions do flirt with specificity, they sometimes contradict each other in interesting but unexamined ways.

Alongside the ambiguity of liveness's definition, we have a second problem: While it is clear that approaches to liveness range widely, varying across many axes, there has been little work to articulate the dimensions of variation which efforts towards liveness encompass. Searching out patterns and mapping out these dimensions might hint us towards unexplored realms of design space, or at least give us a clearer shared language.

This essay begins what I hope will be an ongoing, collaborative effort to figure out what liveness means to us, and to map out the space of possibility we are journeying through together.

Interactive spreadsheet programs can be constructed from small rulesets. ScrapSheets demonstrates a novel combination of (1) composable user-editable-rule behaviors (e.g. search/reduce) and (2) asynchronous rule evaluation (e.g. HTTP requests). Furthermore, its realtime rules-engine is implemented performantly via single-pass filtering/merging algorithms.

Live programming is uniquely suited to creative work. It can remove many of the creative blockers that individuals experience when trying to produce it. But we could place much more explicit emphasis on the removal of emotional blockers from the creative process, as opposed to only focusing on intellectual blockers. Arroost is a project that seeks to do that — an experimental live programming tool for making music.

Software systems should be explainable, that is, they should help us to answer questions while exploring, developing or using them. Textual documentation is a very weak form of explanation, since it is not causally connected to the code, so easily gets out of date. Tests, on the other hand, are causally connected to code, but they are also a weak form of explanation. Although some tests encode interesting scenarios that answer certain questions about how the system works, most tests tend to be uninteresting.

Examples are tests that are also factories for interesting system entities. Instead of simply succeeding or failing, an example returns the object under test so that it can be inspected, or reused to compose further tests. An example is causally connected to the system, is always live and tested, and can be embedded into live documentation. Although technically examples constitute just a tiny modification to test methods, their impact is potentially ground-breaking.

We show (i) how Example-Driven Development (EDD) enriches TDD with live programming, (ii) how examples can be molded with tiny tools to answer analysis questions, and (iii) how examples can be embedded within live documentation to make a system explainable.

Subsequently is an attempt at designing a live programming environment that is deeply visual, tangible and concrete. In Subsequently, you create programs by directly manipulating concrete, visual representations of data values, operations and control flow. All these different aspects are represented on screen as a diagrammatic comic strip. By simply manipulating concrete values step by step, the programmer creates a rich visual narrative that acts both as an executable program and a legible explanation of the algorithm. Because the program is represented concretely, there is no explicit “run” step or debugger. All program state is immediately visualised as you go.

Software for editing media (text, images, audio, video, etc.) faces an essential tension. Direct manipulation enables users to edit media fluidly, as the user can make changes directly to the final product rather than first describing each change programmatically. On the other hand, programming enables users to accomplish tasks that would be tedious or impossible via direct manipulation and can capture intent precisely. Problems arise when these two paradigms are combined, as in editors that support end-user programming. Such applications typically allow the user to go from direct to programmatic manipulation by creating some material directly and then transforming it programmatically (as in a spreadsheet formula). However, if the user then wishes to go back the other way by tweaking the computed output, they face a dilemma: either direct manipulation of the output is forbidden, in which case tweaks must be described programmatically by the user (sacrificing directness), or the computed output must be first copied before it can be changed, in which case the program is abandoned (sacrificing intention). TAPE, the Transformative Action-Preserving Editor, is a text-based prototype intended to suggest a general way out of the dilemma. By allowing direct edits and immediate actions in computed regions, which are automatically recorded as composable transformations, TAPE enables the user to go back and forth between direct and programmatic manipulation without sacrificing either.

Graphs, as a model of computation and as a means of interaction and authorship, have found success in specific domains such as shader programming and signal processing. In these systems, computation is often expressed on nodes of specific types, with edges representing the flow of information. This is a powerful and general-purpose model, but it incentivises a closed-world environment where both node and edge types are decided at design-time. By choosing an alternate topology where computation is represented by edges, the incentive for a closed environment disappears. Scoped Propagators are a programming model designed to be embedded within existing environments and user interfaces.