LIVE · Workshop
Sketchable Interaction - End-User Customization via Interactive
Regions
Part of the Proceedings of the Sixth Workshop on Live Programming (LIVE 2020).
Abstract
Sketchable Interaction (SI) is a concept for a generic graphical environment that facilitates end-user
customization of user interfaces and visual programming. Within an SI context, all graphical objects, including
canvas, icons, and the mouse pointer, are interactive regions with arbitrary shapes. Users can draw further
interactive regions with their mouse cursor, fingers or stylus, creating a sketch of a user interface. All
interactive regions may contain effects which are applied to other regions once those overlap or touch. These
effects can modify data, graphical representation, or both, of other regions. For example, a “Delete” region will
delete all documents that are dragged onto it. Additionally, regions have properties such as position, scale, or
color which can be linked to arbitrary properties of other region effects. For example, while dragging a region,
the position of that region is temporarily linked to the position of the cursor’s region. These two principles
allow users to iteratively construct user interfaces for a variety of different tasks, such as document
management, data processing, or window management. Sketchable Interaction can also function as a visual
programming language via special conveyor belt regions which transport other regions (e.g. documents) along their
path. While in transit, these regions can pass through other regions which apply their effects to the document.
Conveyor belts can also be used as conditional branches, allowing for simple application logic to be implemented
via SI concepts. A C++ runtime handles collision and linking. All interactive regions and their behaviors are
implemented via Python plugins.
We have completed a robust implementation of the runtime and are currently exploring novel application supported
by combinations of simple tools.