Add editor, codebase, and design documentation

2025-08-03 04:52:16 +00:00 · 2021-03-22 01:18:15 -07:00 · 2021-03-22 01:18:15 -07:00 · c4d3c339d1
commit c4d3c339d1
parent 51d966c573
7 changed files with 481 additions and 0 deletions
--- a/documentation/codebase/architecture-overview.nomnoml
+++ b/documentation/codebase/architecture-overview.nomnoml
@ -0,0 +1,165 @@
 #spacing: 120
 #arrowSize: 0.5
 #leading: 1.5
 #gravity: 1
 #direction: down
 #bendSize: 0.2
 #.dependencies: visual=note direction=right title=bold
 #.client: visual=roundrect fill=lightgreen title=bold
 #.editor: fill=lightblue title=bold
 #.document: fill=darkorange title=bold
 #.renderer: fill=gold title=bold
 // Hierarchy
 [<dependencies> Dependencies|
    [<client> Frontend Editor Client]
    [<client> Headless Editor Client]
    [<client> Application Client
    (like a game engine or game that renders .grd graphs at runtime)]
    [<editor> Editor Library]
    [<document> Document Library]
    [<renderer> Renderer Library]
    [Frontend Editor Client] Depends On -> [Editor Library]
    [Editor Library] Depends On -> [Document Library]
    [Headless Editor Client] Depends On -> [Document Library]
    [Document Library] Depends On -> [Renderer Library]
    [Application Client] Depends On -> [Renderer Library]
 ]
 // Client
 [<client> Frontend Implementation|
    One of many possible frontend implementations,
    such as the native or web client.
 ]
 [Frontend Implementation] -> Event [Editor Event Dispatcher]
 [<client> CLI Implementation|
    A possible text-based CLI program that
    wraps Graphite Document. Users can
    pass a .gdd document and operation to
    add to the document history by saving
    to the supplied .gdd file. Users can also
    pass a .gdd document and request a
    render. Caches are saved to disk.
 ]
 [CLI Implementation] -> [Design Document Maintainer]
 // Editor
 [<editor> Editor Event Dispatcher|
    Convert all events from the frontend into Actions
    and Operations. Actions are diffs to update state
    stores. Operations are history events in a .gdd
    file. Operations and document-specific Actions
    include the document ID in the call.
 ]
 [Editor Event Dispatcher] -> Action [Editor State Store]
 [Editor Event Dispatcher] -> Action [Open Documents State Store]
 [Editor Event Dispatcher] -> Update Bounds [Scheduler]
 [Editor Event Dispatcher] -> Operation [Design Document Maintainer]
 [<editor> Editor State Store|
    [<state> Panel Layout]|
    [<state> Working Colors]|
    [<state> Active Tool]|
    [<state> Tool Options]|
    [<state> Modifier Key Input]|
    [<state> Currently Hovered Panel]
 ]
 [Editor State Store] -> [Frontend Implementation]
 [Editor State Store] -> [Tool State Machine]
 [<editor> Open Documents State Store|
    For each open document:|
    [<state> Pan and zoom viewport bounds]|
    [<state> Selected layers]
 ]
 [Open Documents State Store] -> [Tool State Machine]
 [Open Documents State Store] -> [Frontend Implementation]
 [<editor> Tool State Machine|
    List possible operations in current context
 ]
 [Tool State Machine] -> [Frontend Implementation]
 // Document
 [<document> Design Document Maintainer|
    Process each operation and apply changes to the given document's
    operation chain. Consolidate changes where possible by modifying
    the previous operation if it's the same as this one and append partial
    updates to operations from in-progress interactive tools.
 ]
 [Design Document Maintainer] -> Design Document Diff [Render Graph Maintainer]
 [Design Document Maintainer] <-> Design Document [Open Documents File Store]
 [<document> Render Graph Maintainer|
    Bake operations into a complete graph including history references.
    Starting from the beginning of document history, each operation
    modifies the graph in a specific way. If an operation creates a node
    that references this point in history, duplicate the current graph pieces
    between the live graph and each additional history reference so
    changed nodes in history are "ghost variants" living in the live graph.
 ]
 [Render Graph Maintainer] -> Render Graph [Scheduler]
 [Render Graph Maintainer] <-> Render Graph [Open Documents File Store]
 [<document>Open Documents File Store|
    Kept in-memory or written to disk.|
    For each open document:|
    [<state> Design Document (.gdd)]|
    [<state> Render Graph (.grd)]
 ]
 // Renderer
 [<renderer> Scheduler|
    Manage a thread pool and GPU time. Perform progressive enhancement
    and speculative tile rendering. Request re-render of dirtied document within
    bounds and stitch them over dirty raster cache tiles. Maintain interactive
    responsiveness by targeting a minimum framerate and picking a requisite
    render quality level at all times. Precompute nodes downstream of changes
    when there is spare CPU thread pool and GPU time. Primary purpose is to
    keep the Precompute and Raster Tile caches up-to-date with minimal
    onscreen latency. Also notifies the Final Render Assembler when the cache
    tiles for the root document visible in bounds have finished being updated.
 ]
 [Scheduler] -> [Node Rasterizer] 
 [Scheduler] <-> [Node Cache Manager] 
 [Scheduler] -> [Final Render Assembler] 
 [<renderer> Node Cache Manager|
    Purges old or less useful cached data
    as necessary.|
    [<state> Precompute Node Cache|
        Prepare data for rasterization step.
        Evaluate algorithms (CPU or GPU).
        Changes when upstream changes.
    ]|
    [<state> Raster Tile Node Cache|
        Must be fast, fragment shader only.
        Only reevaluate downstream.
        Reuse when panning.
    ]
 ]
 [Node Cache Manager] -> [Final Render Assembler]
 [<renderer> Node Rasterizer|
    Rasterize and update the cache for a specified node
    given a preferred bounds (but it can give a different
    sized output), resolution, and quality level.
 ]
 [Node Rasterizer] <-> [Node Cache Manager]
 [<renderer> Final Render Assembler|
    Look up the rasterized cached tiles from
    the document root and assemble them into
    an image for live display by the frontend.
    Operates when woken up by the scheduler
    when root tiles or render bounds are updated.
 ]
 [Final Render Assembler] --> Rendered Viewport [Frontend Implementation]
--- a/documentation/codebase/index.md
+++ b/documentation/codebase/index.md
@ -0,0 +1,38 @@
 # Graphite codebase docs
 This is a great place to start learning about the Graphite codebase and its architecture and code structure.
 ## Core libraries
 Graphite's core rust codebase is split into three reusable libraries:
 - Graphite Editor Library
 - Graphite Document Library
 - Graphite Renderer Library
 Each depends on its successor in the list. These are used internally but also intended for usage by third parties through Rust or linked by a project in C, C++, or another language.
 ## Code structure
 The main modules of the project architecture are outlined below. Some parts describe future features and the directories don't exist yet. **Bold** modules are required for Graphite 0.1 which is purely an SVG editor.
 - **Web frontend**: `/web-frontend/`  
  Initial GUI for Graphite that will eventually be replaced by a native GUI implementation
 	- **Vue web app**: `src/`  
 	  Imports the WASM code and uses Vue props to customize and reuse most GUI components
 	- **Rust WebAssembly wrapper**: `wasm-wrapper/`  
 	  Wraps the Graphite Editor Library and provides an API for the web app to use unburdened by Rust's complex data types that are not supported by WASM
 - Native frontend: `/editor-client/`  
  The future official desktop client. Blocked on Rust's GUI ecosystem improving or dedicating the time to build a custom system that can nicely support editor extensions. The whole GUI should use WGPU for rendering and compile to WASM to make those calls to the WebGPU API.
 - CLI: `/packages/cli/`  
  A headless, command line GDD document editor (using the Graphite Document Library) and GRD render graph renderer (using the Graphite Renderer Library). Not the initial focus of development, but perhaps useful in testing for certain features throughout the development process. A future version of the CLI will probably redesign the command structure.
 - **Graphite Editor Library**: `/packages/graphite-editor/`  
  Used by a frontend editor client to maintain GUI state and dispatch user events. The official Graphite editor is the primary user, but others software like game engines could embed their own customized editor implementations. Depends on the Graphite Document Library.
 - Graphite Document Library: `/packages/graphite-document/`  
  A stateless library for updating Graphite design document (GDD) files. The official Graphite CLI and Graphite Editor Library are the primary users, but this library is intended to be useful to any application that wants to link the library for the purpose of updating GDD files by sending edit operations. Optionally depends on the Graphite Renderer Library if rendering is required.
 - Graphite Renderer Library: `/packages/graphite-renderer/`  
  A stateless library (with the help of in-memory and/or on-disk caches for performance) for rendering Graphite's render graph (GRD) files. The official Graphite CLI and Graphite Document Library are the primary users, but this library is intended to be useful to any application that wants to link the library for the purpose of rendering Graphite's render graphs. For example, games can link the library and render procedural textures with customizable parametric input values.
 ## Architecture diagram
 Take this [Architecture Overview](architecture-overview.nomnoml) diagram and paste it into the [Nomnoml](https://nomnoml.com/) diagram editor. We'll set up a better way to render Nomnoml diagrams when we have a proper home for this documentation.
--- a/documentation/design/feature-goals.md
+++ b/documentation/design/feature-goals.md
@ -0,0 +1,100 @@
 # Feature goals
 ## Possible use cases
 - Photography
 	- RAW photo editing/processing
 	- Batch processing pipeline
 - Motion graphics
 	- Title sequences/kinetic typography/etc.
 - Live broadcast/streaming
 	- Live video compositing/overlays
 	- Interactive exhibits (rendering live content for museum/art/festival exhibits)
 - Web
 	- SVG design
 	- Animated or interactive SVG design
 - Automation
 	- Batch multimedia editing/conversion
 - Graphic design
 	- Print design
 	- Web/digital-focused graphics (marketing, branding, infographics, ads, etc.)
 	- Templates filled with file/spreadsheet data (e.g. prototyping/iterating components of a board/card game)
 - Illustration
 - Digital painting
 - Icon design
 - Logo design
 - UI/UX design
 	- Mockups
 	- Wireframing
 - Desktop publishing
 	- Templates filled with Markdown/HTML content with export to PDF
 - Font design
 - Video compositing
 - Data Visualization
 	- Data-powered graphs/charts/etc.
 	- Automated rendering with live/often-updated data
 - Traditional Animation
 - 3D/Gamedev
 	- PBR procedural material authorship
 	- 3D model UV map texturing
 - AI-assisted tools
 - HDR processing
 - 360° and panoramic stitching and spherical editing
 ## User stories
 Contributions welcome! If you think of something Graphite would be great for, submit a pull request or send Keavon a message on Graphite's Discord server.
 ## Photography
 - Using a face detection node to sort photos into the correct folders upon export
 - Using a face detection node to place a watermark near every face to prevent customers from cropping out watermarks placed in less important areas of some photos
 - Shadow removal from part of an image by masking the location of a shadow and using the texture details of the darker area and the lighting and color context of the illuminated area
 - Lightening or contextually infilling all the areas where a camera sensor has dust specks
 - Isolating clipping highlights (that have expanded into neighboring pixels) in one or more color channels from point light sources like city lights and rendering smoother bright gradient point lights in place of them
 - Advanced, intuitive blending with a node that lets you create a "custom blend mode" by specifying something like "Anything that's white: display as is. Anything else further from white by luminosity: fade out the saturation and opacity."
 ## Image editing
 - Removing translucent watermarks that were applied in the same location to a batch of photos by finding their shared similarities and differences and using that as a subtraction diff
 ## Game development
 - Design a GUI for the game and use the Graphite Renderer Library to render the in-game GUI textures at runtime at the desired resolution without scaling problems, or even render it live as data updates its state
 - Authoring procedural noise-based textures and PBR materials
 ## Data visualization
 - Creating a chart from a CSV
 - Rendering an always-up-to-date chart powered by real-time updates from a database
 - Data-driven infographics like an org chart that can be updated with text instead of manual design work
 ## Digital painting
 - Creating a digital acrylic or oil painting using various brushes
 - Preventing mixing/smearing of previous wet paint layers by drying it with a hair dryer tool
 - Smearing wet paint colors together on a simulated paint palette and then sampling paint colors from that palette to paint with
 ## Education
 - Rendering a timelapse video of everything operation done in the history of a document
 ## Graphic design
 - Prototyping cards for board games fed with data in a spreadsheet which generates the cards from a template
 - Creating an image that has been shredded but pieced back together, where the image can be updated then return to the shredded one without having to redo the editing steps to shred it
 ## Broadcast and streaming
 - Rendering overlays for live streams or television broadcasts based on live input data, for example somebody donates and leaves a comment on a live stream and this web hook could trigger an animated display containing the user and their comment, or live telemetry for a rocket launch streams in and gets rendered as graphical overlays for a webcast.
 ## Interactive exhibits and digital signage
 - Rendering a custom live clockface with hour/minute/second hands based on an input of the current time, then showing them fullscreen on a display
 - Data from sensors can render interactive 2D graphics at a museum art installation
 - A storefront can have a monitor set up showing daily or hourly sales items based on web hooks or polling from the company’s website
 - Polling an API for content like Twitter or an RSS feed and displaying the tweet or headline when it arrives on screen, styled as desired
 ## Print and publishing
 - Formatting Markdown documents into PDF print layouts
 - Laying out book covers for proper PDF export to a printer
 - Typesetting and formatting all the interior pages of a book with manual control where needed
 ## Automation
 - Laser cutter artwork processing for automating custom Etsy orders
 - Running on a server to let users upload images for a custom T-shirt printing website, and it renders their graphic on the model’s shirt (or other custom printing online stores)
 - Generating a PDF invoice based on data in a pipeline in a server
 ## Industrial control
 - Factory line is examining its fruit for defects. In order to verify the quality, they need to enhance the contrast, automatically orient the image and correct the lighting. They then pass the results into a machine learning algorithm to classify, and sometimes need to snoop on the stream of data to manually do quality control (ImageMagick or custom Python libraries are often used for this right now)
--- a/documentation/design/index.md
+++ b/documentation/design/index.md
@ -0,0 +1,7 @@
 # Graphite design notes
 Notes and details about the product design, workflow, user experience, and UI for the Graphite editor and systems. More to come as scattered outdated notes get cleaned up and moved here.
 - [Feature goals](feature-goals.md)
 	- [Possible use cases](feature-goals.md#user-stories)
 	- [User stories](feature-goals.md#user-stories)
--- a/documentation/editor/1-overview.md
+++ b/documentation/editor/1-overview.md
@ -0,0 +1,58 @@
 # Overview
 ## Key concepts
 TODO
 ## Glossary of terminology
 TODO: Add more to make a comprehensive list, finish writing definitions, separate into categories, alphabetize
 - Asset  
  A *GDD* or *GRD* file. Can be shared and *embedded* in another *layer graph*. Useful for providing custom *nodes* that perform some useful functionality. Tangible examples include custom procedural effects, shape generators, and image filters. Many of the Graphite editor's built-in *layers* are also assets that provide useful functionality through a group of nodes rather than being implemented directly in code. The *Asset Manager* panel helps maintain these assets from various sources. The *Asset Store* can be used to share and sell assets for easily inclusion in projects.
 - Document  
  A design source file created and edited in the Graphite editor. When saved to disk as *GDD files* (Graphite Design Document), they are one of the two types of *assets*. Documents can be included as *layers* inside other documents, and in doing so they take the form of *groups*. The *layer graph* contents of a *group* actually belong to the *embedded* document's *subgraph*. Because a document is a *group* which is a *layer* in the *layer graph*, documents have *properties* such as the (optional) *pixel bounds* of the *canvas*. Documents are composed of a layer graph, a defined set of properties of set *data types* that are *imported* and *exported*, and the *properties* of the root *layer*.
 - Render graph  
  A read-only "compiled" *document* in a format that is immediately ready for rendering. When saved to disk as *GRD files* (Graphite Render Data), they are one of the two types of *assets*. The Graphite editor internally maintains a render graph based on the open document in order to display it live in the *viewport*, but this can also be saved to disk for the purposes of sharing as an *asset*.
 - GDD file  
  Graphite Design Document. A binary serialization of a *document* source file. The format includes a chain of *operations* that describe changes to the *layer graph* and the *properties* of *layers* throughout the history of the document since its creation. It also stores certain metadata and *embedded* file data. GDD files, along with *GRD files*, represent *assets* when shared. Because GDD files are editable (unlike *GRD files*), the *layers* of GDD *assets* may be expanded in the layer graph to reveal and modify their contents using a copy-on-write scheme stored to the *asset's* *layer*.
 - GRD file  
  Graphite Render Data. A binary serialization of a *render graph* file. The format includes a single directed acyclic graph (DAG) compiled from the *layer graph* of a *document* as well as certain *properties* of set *data types* that are *imported* and *exported*. GRD files, along with *GDD files*, represent *assets* when shared. Because GRD files are read-only and can't be edited (unlike *GDD files*), the *layers* of GRD *assets* do not offer an ability to be expanded in the layer graph. GRD files are useful for sharing *assets* when their authors do not wish to provide the source *documents* used for their authoring. They are also the input format included in games that utilize the *Graphite Renderer Library* to render graphical content at runtime, as well as similar applications like headless renderers on web servers and image processing pipelines.
 - Window
 - Main window
 - Popout window
 - Title bar
 - Status bar
 - Workspace  
  The part of the Graphite editor's UI that houses the *panels* in a *window*. The workspace occupies the large space below the *title bar* and above the *status bar* of the *main window*. It occupies the entirety of *popout windows* (window buttons are added in the *tab bar*).
 - Workspace layout  
  The specific configuration of panels in the *main window* and any *popout windows*. Workspace layout presets are provided by the Graphite editor and users may customize and save their own.
 - Tab bar  
  The bar at the top of a *panel group* which includes a clickable tab for each panel that is docked there. Each tab bar has at least one tab and one active tab.
 - Active tab  
  The one tab in a *tab bar* that is currently active. The user can click any inactive tab to make it become the active tab. The active tab shows the *panel content* beneath it unless it is a *folded panel*.
 - Folded panel  
  A shrunken *panel* showing only the *tab bar*. A *panel* consists of the *tab bar* and *panel body* except when the latter is folded away. The user may click the *active tab* to fold and restore a panel, however a panel cannot be folded if there are no other unfolded panels in its column.
 - Panel
 - Panel body
 - Panel content
 - Editor
 - Layer graph  
  A (directed acyclic) graph structure composed of *layers* with *connections* between their input and output *ports*. This is commonly referred to as a "node graph" in other software, but Graphite's layer graph is more suited towards layer-based compositing compared to traditional compositor node graphs.
 - Layer  
  Any instance of a *node* that lives in the *layer graph*. Layers (usually) take input data, then they transform it or synthesize new data, then they provide it as output. Layers have *properties* as well as exposed input and output *ports* for sending and receiving data.
 - Node  
  A definition of a 
 - Group
 - Raster
 - Vector
 - Mask
 - Data type
 - Subgraph
 - Port
 - Connection
 - Pixel bounds
 - Canvas
 - Graphite Editor Library
 - Graphite Document Library
 - Graphite Renderer Library
--- a/documentation/editor/index.md
+++ b/documentation/editor/index.md
@ -0,0 +1,100 @@
 # Graphite editor manual
 Work in progress.
 - [Overview](1-overview.md)
 	- [Key concepts](1-overview.md#key-concepts)
 	- [Glossary of terminology](1-overview.md#glossary-of-terminology)
 - Interface
 	- Title bar
 	- Workspace
 		- Panel interface (tab, pin, options bar, left menu)
 		- Arrangement and docking
 	- Status bar
 	- Multiple windows
 - Panels
 	- Viewport
 		- Canvas
 		- Rulers
 		- Tool menu
 		- Options bar
 	- Properties
 		- Blending
 		- Origin
 		- Transform
 		- Node-specific properties
 	- Layers
 		- Interface
 		- Compositing flow
 		- Groups
 		- Masks
 		- Isolation
 	- Graph
 		- Interface
 		- Layer/node equivalence
 		- Compositing flow
 		- Groups
 		- Masks
 		- Isolation
 	- Connection Matrix
 	- Spreadsheet
 	- Node Catalog
 	- Asset Manager
 		- Embedded assets
 		- Linked assets
 		- Local assets
 		- Remote assets
 		- Store assets
 	- Palettes
 		- Each color palette is an asset
 	- Minimap
 	- Histogram
 	- Timeline
 - Documents and assets
 	- Node groups, layer groups, and document tabs are equivalent
 	- Groups/documents are assets
 	- Assets are embedded or linked
 	- The document canvas is a group output
 	- Art boards and dimensions
 	- Reusable assets
 - Tools
 	- Overview
 		- Tools add and update assets
 	- Layout tool group
 	- Parametric tool group
 	- Raster tool group
 	- Vector tool group
 - Vector editing
 	- Data types
 	- Data flow
 	- Rasterization
 - Raster editing
 	- Resolution-agnostic sampling
 	- Compositing and data flow
 	- Caching
 	- Predictive caching
 	- Progressive enhancement
 	- Generator nodes
 	- Rasterizer nodes
 	- Adjustment nodes
 	- Filter nodes
 	- Sample transformer nodes
 	- Historical sampling
 - Text editing
 - Data types
 - Node types
 - Node library
 - File format interoperability
 	- Import/export: JPG, PNG, APNG, GIF, TIFF, TGA, BMP, JPEG 2000, WebP, HEIF, ICO
 	- Import/export: EXR
 	- Import: digital camera raw formats
 	- Import/export: SVG
 	- Import/export: EPS
 	- Import/export: PDF
 	- Import: PSD/PSB, AI, INDD?
 	- Export: G-code formats for laser and vinyl cutters?
 - Extensions
 - Color management
 - Units, measurement, and scale
 - Headless and integrations
 - Animation
--- a/documentation/index.md
+++ b/documentation/index.md
@ -0,0 +1,13 @@
 # Graphite project documentation
 ## [Graphite editor manual](editor/index.md)
 Click above for the Graphite editor manual. This will evolve into the official manual for the Graphite editor. It contains documentation for each feature of Graphite focused on both usage instructions and technical details.
 ## [Graphite codebase docs](codebase/index.md)
 Click above for the Graphite codebase docs. This is meant as a good starting point for new developers to jump into the Graphite codebase. It aims to explain the architecture, code structure, and other useful details.
 ## [Graphite design notes](design/index.md)
 Click above for the Graphite design notes. This is a collection of notes and ideas pertaining to the user experience, workflow, and UI design of Graphite. It should aim to be mostly up-to-date and reflect the current state of understanding of the design for relatively finalized features, but in-progress feature designs may contain ideas, concepts, and proposals.