mirror of
https://github.com/GraphiteEditor/Graphite.git
synced 2025-12-23 10:11:54 +00:00
Update references to the latest tech stack plans
This commit is contained in:
Keavon Chambers
parent
00cfa073b8
commit
4391f88d03
14 changed files with 15 additions and 51 deletions
4
.github/workflows/build-dev-and-ci.yml
vendored
4
.github/workflows/build-dev-and-ci.yml
vendored
|
|
@ -48,11 +48,9 @@ jobs:
|
|||
rustc --version
|
||||
|
||||
- name: ✂ Replace template in <head> of index.html
|
||||
if: github.ref != 'refs/heads/master'
|
||||
env:
|
||||
INDEX_HTML_HEAD_REPLACEMENT: ""
|
||||
run: |
|
||||
# Remove the INDEX_HTML_HEAD_REPLACEMENT environment variable for build links (not master deploys)
|
||||
git rev-parse --abbrev-ref HEAD | grep master > /dev/null || export INDEX_HTML_HEAD_REPLACEMENT=""
|
||||
sed -i "s|<!-- INDEX_HTML_HEAD_REPLACEMENT -->|$INDEX_HTML_HEAD_REPLACEMENT|" frontend/index.html
|
||||
|
||||
- name: 🌐 Build Graphite web code
|
||||
|
|
|
|||
|
|
@ -1,14 +1,12 @@
|
|||
# This is a helper file for people using NixOS as their operating system.
|
||||
# If you don't know what this file does, you can safely ignore it.
|
||||
# This file defines both the development environment for the project.
|
||||
# This file defines the reproducible development environment for the project.
|
||||
#
|
||||
# Development Environment:
|
||||
# - Provides all necessary tools for Rust/WASM development
|
||||
# - Includes Tauri dependencies for desktop app development
|
||||
# - Provides all necessary tools for Rust/Wasm development
|
||||
# - Sets up profiling and debugging tools
|
||||
# - Configures mold as the default linker for faster builds
|
||||
#
|
||||
#
|
||||
# Usage:
|
||||
# - Development shell: `nix develop`
|
||||
# - Run in dev shell with direnv: add `use flake` to .envrc
|
||||
|
|
@ -16,7 +14,6 @@
|
|||
description = "Development environment and build configuration";
|
||||
|
||||
inputs = {
|
||||
# This url should be changed to match your system packages if you work on tauri because you need to use the same graphics library versions as the ones used by your system
|
||||
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
|
||||
nixpkgs-unstable.url = "github:nixos/nixpkgs/nixos-unstable";
|
||||
rust-overlay = {
|
||||
|
|
@ -96,7 +93,6 @@
|
|||
pkgs.pkg-config
|
||||
pkgs.git
|
||||
pkgs.gobject-introspection
|
||||
pkgs-unstable.cargo-tauri
|
||||
pkgs-unstable.cargo-about
|
||||
|
||||
# Linker
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
use super::*;
|
||||
use std::sync::mpsc::{Receiver, Sender};
|
||||
|
||||
/// Handles communication with the NodeRuntime, either locally or via Tauri
|
||||
/// Handles communication with the NodeRuntime
|
||||
#[derive(Debug)]
|
||||
pub struct NodeRuntimeIO {
|
||||
// Send to
|
||||
|
|
|
|||
|
|
@ -1,8 +1,6 @@
|
|||
# Overview of `/frontend/`
|
||||
|
||||
The Graphite frontend is a web app that provides the presentation for the editor. It displays the GUI based on state from the backend and provides users with interactive widgets that send updates to the backend, which is the source of truth for state information. The frontend is built out of reactive components using the [Svelte](https://svelte.dev/) framework. The backend is written in Rust and compiled to WebAssembly (WASM) to be run in the browser alongside the JS code.
|
||||
|
||||
For lack of other options, the frontend is currently written as a web app. Maintaining web compatibility will always be a requirement, but the long-term plan is to port this code to a Rust-based native GUI framework, either written by the Rust community or created by our project if necessary. As a medium-term compromise, we may wrap the web-based frontend in a desktop webview windowing solution like Electron (probably not) or [Tauri](https://tauri.app/) (probably).
|
||||
The Graphite frontend is a web app that provides the presentation for the editor. It displays the GUI based on state from the backend and provides users with interactive widgets that send updates to the backend, which is the source of truth for state information. The frontend is built out of reactive components using the [Svelte](https://svelte.dev/) framework. The backend is written in Rust and compiled to WebAssembly (Wasm) to be run in the browser alongside the JS code.
|
||||
|
||||
## Bundled assets: `assets/`
|
||||
|
||||
|
|
@ -18,15 +16,15 @@ Source code for the web app in the form of Svelte components and [TypeScript](ht
|
|||
|
||||
## WebAssembly wrapper: `wasm/`
|
||||
|
||||
Wraps the editor backend codebase (`/editor`) and provides a JS-centric API for the web app to use unburdened by Rust's complex data types that are incompatible with JS data types. Bindings (JS functions that call into the WASM module) are provided by [wasm-bindgen](https://rustwasm.github.io/docs/wasm-bindgen/) in concert with [wasm-pack](https://github.com/rustwasm/wasm-pack).
|
||||
Wraps the editor backend codebase (`/editor`) and provides a JS-centric API for the web app to use as an entry point, unburdened by Rust's complex data types that are incompatible with JS data types. Bindings (JS functions that call into the Wasm module) are provided by [wasm-bindgen](https://rustwasm.github.io/docs/wasm-bindgen/) in concert with [wasm-pack](https://github.com/rustwasm/wasm-pack).
|
||||
|
||||
## ESLint configurations: `.eslintrc.js`
|
||||
## ESLint configurations: `.eslintrc.cjs`
|
||||
|
||||
[ESLint](https://eslint.org/) is the tool which enforces style rules on the JS, TS, and Svelte files in our frontend codebase. As it is set up in this config file, ESLint will complain about bad practices and often help reformat code automatically when (in VS Code) the file is saved or `npm run lint` is executed. (If you don't use VS Code, remember to run this command before committing!) This config file for ESLint sets our style preferences and configures our usage of extensions/plugins for Svelte support and [Prettier](https://prettier.io/)'s role as a code formatter.
|
||||
|
||||
## npm ecosystem packages: `package.json`
|
||||
|
||||
While we don't use Node.js as a JS-based server, we do have to rely on its wide ecosystem of packages for our build system toolchain. If you're just getting started, make sure to install the latest LTS copy of Node.js. Our project's philosophy on third-party packages is to keep our dependency tree as light as possible, so adding anything new to our `package.json` should have overwhelming justification. Most of the packages are just development tooling (TypeScript, Vite, ESLint, Prettier, wasm-pack, and [Sass](https://sass-lang.com/)) that run in your console during the build process.
|
||||
While we don't use Node.js as a JS-based server, we do rely on its ecosystem of packages for our build system toolchain. If you're just getting started, make sure to install the latest LTS copy of [Node.js](https://nodejs.org/en/download). Our project's philosophy on third-party packages is to keep our dependency tree as light as possible, so adding anything new to our `package.json` should have overwhelming justification. Most of the packages are just development tooling (TypeScript, Vite, ESLint, Prettier, and [Sass](https://sass-lang.com/)) that run in your terminal during the build process.
|
||||
|
||||
## npm package installed versions: `package-lock.json`
|
||||
|
||||
|
|
@ -36,6 +34,6 @@ Specifies the exact versions of packages installed in the npm dependency tree. W
|
|||
|
||||
Basic configuration options for the TypeScript build tool to do its job in our repository.
|
||||
|
||||
## Vite configurations: `vite.config.js`
|
||||
## Vite configurations: `vite.config.ts`
|
||||
|
||||
We use the [Vite](https://vitejs.dev/) bundler/build system. This file is where we configure Vite to set up plugins (like the third-party license checker/generator). Part of the license checker plugin setup includes some functions to format web package licenses, as well as Rust package licenses provided by [cargo-about](https://github.com/EmbarkStudios/cargo-about), into a text file that's distributed with the application to provide license notices for third-party code.
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ export function textInputCleanup(text: string): string {
|
|||
// <https://github.com/WICG/keyboard-map/issues/26>
|
||||
// In the desktop version of VS Code, this is achieved with this Electron plugin:
|
||||
// <https://github.com/Microsoft/node-native-keymap>
|
||||
// We may be able to port that (it's a relatively small codebase) to Rust for use with Tauri.
|
||||
// We may be able to port that (it's a relatively small codebase) to Rust for use with our desktop application.
|
||||
// But on the web, just like VS Code, we're limited by the shortcomings of the spec.
|
||||
// A collection of further insights:
|
||||
// <https://docs.google.com/document/d/1p17IBbYGsZivLIMhKZOaCJFAJFokbPfKrkB37fOPXSM/edit>
|
||||
|
|
|
|||
|
|
@ -59,13 +59,6 @@ in
|
|||
samply
|
||||
cargo-flamegraph
|
||||
|
||||
# For Tauri
|
||||
openssl
|
||||
glib
|
||||
gtk3
|
||||
libsoup
|
||||
webkitgtk
|
||||
|
||||
# For Rawkit tests
|
||||
libraw
|
||||
|
||||
|
|
|
|||
|
|
@ -70,14 +70,6 @@ pub struct WasmApplicationIo {
|
|||
static WGPU_AVAILABLE: std::sync::atomic::AtomicI8 = std::sync::atomic::AtomicI8::new(-1);
|
||||
|
||||
pub fn wgpu_available() -> Option<bool> {
|
||||
// Always enable wgpu when running with Tauri
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
if let Some(window) = web_sys::window() {
|
||||
if js_sys::Reflect::get(&window, &wasm_bindgen::JsValue::from_str("__TAURI__")).is_ok() {
|
||||
return Some(true);
|
||||
}
|
||||
}
|
||||
|
||||
match WGPU_AVAILABLE.load(Ordering::SeqCst) {
|
||||
-1 => None,
|
||||
0 => Some(false),
|
||||
|
|
|
|||
|
|
@ -166,7 +166,7 @@ And then from a development perspective, I am looking forward to accomplishing t
|
|||
|
||||
- Restoring several previous features that were removed during refactors in the past year to a fully working state including Imaginate, snapping, folder bounding boxes, transform pivots, and vector shape boolean operations
|
||||
- Deploying GPU-based rendering by default and moving from an experimental to a production-ready hardware-accelerated compositing system using [Vello](https://github.com/linebender/vello) to unify the currently separate raster and vector pipelines
|
||||
- Shipping desktop apps for Windows, Mac, and Linux by integrating [Tauri](https://tauri.app/) and bundling built-in AI models to run Imaginate and other upcoming features directly on user hardware
|
||||
- Shipping desktop apps for Windows, Mac, and Linux and bundling built-in AI models to run Imaginate and other upcoming features directly on user hardware
|
||||
- Designing a new vector graphics data format suitable for advanced procedural editing and rendering, plus the associated procedural workflow features
|
||||
- Remaking the Brush tool with the GPU-accelerated pipeline and the adaptive resolution system so digital painting in Graphite becomes practical
|
||||
- Implementing the Mask Mode feature for Magic Wand tool marquee selections, which will dramatically improve Graphite's utility as a raster graphics editor
|
||||
|
|
|
|||
|
|
@ -173,7 +173,7 @@ There are so many plans that I'm eager to carry out to improve the clarity and c
|
|||
|
||||
Starting out with our most in-demand request: a desktop app. This has been on our roadmap from the start but only recently it's begun making sense putting it at the front of the roadmap priorities. We hoped to complete it by the end of 2024, but that wasn't in the cards due to developer availability and the specialized skills needed to complete the task.
|
||||
|
||||
Now to get technical, the lazy option exists: chucking the whole web app in an unaltered Electron wrapper, but this is a technological dead end that I believe offers no value compared to a PWA. The value comes from offering an actual native app where the editor Rust code and GPU-accelerated rendering runs on a user's Windows, Mac, or Linux machine without browser overhead. <a href="https://tauri.app/" target="_blank">Tauri</a> provides this capability, but our use case presents some unsupported challenges that our team has to overcome— individually on each platform. The natively rendered viewport content must be composited with the rest of the Tauri window's editor GUI. When investigating this earlier in the year, this isn't properly supported and we will have to develop solutions to fix Tauri's use of the OS platform APIs. If you have experience with native development on Windows, Mac, and/or Linux, please get involved to speed up this effort! With our current resources, I am anticipating this will be ready for release around spring.
|
||||
Now to get technical, the lazy option exists: chucking the whole web app in an unaltered Electron wrapper, but this is a technological dead end that I believe offers no value compared to a PWA. The value comes from offering an actual native app where the editor Rust code and GPU-accelerated rendering runs on a user's Windows, Mac, or Linux machine without browser overhead. Our use case of combining the web-rendered editor interface with the user's native-rendered artwork presents several unique challenges that our team has to overcome— individually on each platform. If you have experience with native development on Windows, Mac, and/or Linux, please get involved to speed up this effort! With our current resources, I am anticipating this will be ready for release around spring.
|
||||
|
||||
### Animation
|
||||
|
||||
|
|
|
|||
|
|
@ -41,7 +41,7 @@ Out of the [full list of project opportunities](/volunteer/guide/student-project
|
|||
- Creating an efficient brush rendering system for large painted path datasets
|
||||
- Managing colors through flexible use of color spaces, color models, and color profiles
|
||||
- Researching an assortment of algorithms for image processing operations
|
||||
- Engineering solutions for combining Tauri's webview and Graphite's WGPU renderer in the same window across Windows, Mac, and Linux desktop applications
|
||||
- Engineering solutions for combining our web-based editor GUI and native WGPU renderer in the same window across Windows, Mac, and Linux desktop applications
|
||||
|
||||
These recently added projects are pending an extended description, but we'll be happy to discuss the details if you pop into our [Discord server](https://discord.graphite.rs) and introduce yourself with links to related experience and projects.
|
||||
|
||||
|
|
|
|||
|
|
@ -15,17 +15,6 @@ The best introduction for getting up-to-speed with Graphite contribution comes f
|
|||
<img data-youtube-embed="vUzIeg8frh4" src="https://static.graphite.rs/content/volunteer/guide/workshop-intro-to-coding-for-graphite-youtube.avif" onerror="this.onerror = null; this.src = this.src.replace('.avif', '.png')" alt="Workshop: Intro to Coding for Graphite" />
|
||||
</div>
|
||||
|
||||
<!-- ## Tech stack -->
|
||||
<!-- - rustc: Compiler for node graph generics and custom nodes -->
|
||||
<!-- - rust-gpu: Compiler backend to generate compute shaders from Rust source code -->
|
||||
<!-- - wgpu: Portable graphics API for running compute shaders on desktop and web -->
|
||||
<!-- - Tauri: lightweight desktop web UI shell while the backend runs natively (experimental) -->
|
||||
<!-- - Vello: GPU-accelerated vector graphics renderer -->
|
||||
<!-- - COSMIC Text: Text shaping and typesetting -->
|
||||
<!-- - Wasmer or Wasmtime: Portable, sandboxed runtime for custom nodes -->
|
||||
<!-- - Tokio: parallelized job execution in the node graph pipeline -->
|
||||
<!-- - Xilem: High-performance native UI framework, to replace Tauri when ready -->
|
||||
|
||||
## Codebase structure
|
||||
|
||||
Graphite is built from several main software components. New developers may choose to specialize in one or more area without having to attain a working knowledge of the full codebase.
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@ The fully compiled regime is used only when the user exports the procedural artw
|
|||
|
||||
### Compile server
|
||||
|
||||
The three regimes have thus far been only a description of the eventual architecture direction. The interpreted regime is currently the only mode implemented in Graphene. The other two will require access to `rustc` which will necessitate the compile server that we will finish building and then publicly host for Graphite users in the future. Users of the desktop version of Graphite, utilizing [Tauri](https://tauri.app/), will be able to use an embedded `rustc` if the user has opted to download the Rust toolchain while installing Graphite.
|
||||
The three regimes have thus far been only a description of the eventual architecture direction. The interpreted regime is currently the only mode implemented in Graphene. The other two will require access to `rustc` which will necessitate the compile server that we will finish building and then publicly host for Graphite users in the future. Users of the desktop version of Graphite will be able to use an embedded `rustc` if the user has opted to download the Rust toolchain while installing Graphite.
|
||||
|
||||
Without a compile server, all the nodes are precompiled when Graphite is built. The node registry (in the file `node_registry.rs`) currently exists to allow the interpreted executor to find the Rust functions that correspond to each node with its appropriate type signature. Nodes support generics, so it's currently necessary to list every forseeable concrete type signature in the registry until the compile server can generate bytecode for less common type combinations on-the-fly.
|
||||
|
||||
|
|
|
|||
|
|
@ -26,8 +26,6 @@ cargo install -f wasm-bindgen-cli@0.2.100
|
|||
|
||||
Regarding the last one: you'll likely get faster build times if you manually install that specific version of `wasm-bindgen-cli`. It is supposed to be installed automatically but a version mismatch causes it to reinstall every single recompilation. It may need to be manually updated periodically to match the version of the `wasm-bindgen` dependency in [`Cargo.toml`](https://github.com/GraphiteEditor/Graphite/blob/master/Cargo.toml).
|
||||
|
||||
Lastly, if you intend to develop using the Tauri desktop app build target, obtain [Tauri's dependencies](https://v2.tauri.app/start/prerequisites/). This is not the usual setup for most contributors, so you will know if you need it.
|
||||
|
||||
## Repository
|
||||
|
||||
Clone the project to a convenient location:
|
||||
|
|
|
|||
|
|
@ -83,7 +83,7 @@ AI/ML is filling a rapidly growing role as a tool in the creative process. Graph
|
|||
<details>
|
||||
<summary>For additional technical details: click here</summary>
|
||||
|
||||
The approach should be extensible to future models. It needs to run fast and natively on the assorted hardware of local user machines with hardware acceleration. It should be a one-click installation process for users to download and run models without requiring dependencies or environment setup. Ideally, it should allow the more lightweight models to run locally in browsers with WebGPU. It needs to also be deployable to servers in a scalable, cost-viable manner that reuses most of the same code that runs locally. Runtime overhead, cold start times, and memory usage should be minimized for quick, frequent switching between models in a node graph pipeline. The tech stack also needs to be permissively licensed and, as much as possible, Rust-centric so it doesn't add complexity to our Wasm and Tauri build processes. For Stable Diffusion, we need the flexability to track the latest research and extensions to the ecosystem like new base models, checkpoint training, DreamBooth, LoRA, ControlNet, IP-Adapter, etc. and expose these functionalities through modular nodes.
|
||||
The approach should be extensible to future models. It needs to run fast and natively on the assorted hardware of local user machines with hardware acceleration. It should be a one-click installation process for users to download and run models without requiring dependencies or environment setup. Ideally, it should allow the more lightweight models to run locally in browsers with WebGPU. It needs to also be deployable to servers in a scalable, cost-viable manner that reuses most of the same code that runs locally. Runtime overhead, cold start times, and memory usage should be minimized for quick, frequent switching between models in a node graph pipeline. The tech stack also needs to be permissively licensed and, as much as possible, Rust-centric so it doesn't add complexity to our Wasm and desktop build processes. For Stable Diffusion, we need the flexability to track the latest research and extensions to the ecosystem like new base models, checkpoint training, DreamBooth, LoRA, ControlNet, IP-Adapter, etc. and expose these functionalities through modular nodes.
|
||||
|
||||
To meet most of these criteria, our current thinking is to distribute and run our models using the [ONNX](https://onnx.ai/) format. This would integrate ONNX runtimes for WebGPU, native (DirectML, CUDA), and GPU cloud providers. The issue with this approach is that these models (particularly Stable Diffusion) aren't available in an ONNX format.
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue