## Overview
This PR enhances documentation in the expression analysis system to
clarify how goto_definition works and when the `RefExpr` fields are set
versus `None`. This addresses feedback from @BlueQuantumx and @jo3-l who
requested better documentation to understand the expression analysis
flow.
## Problem
The existing documentation was insufficient for developers trying to
understand:
1. How the goto_definition feature works through the expression analysis
system
2. When the `root`, `step`, and `term` fields in `RefExpr` are `Some`
versus `None`
3. The overall flow of expression analysis in `expr_of`
The brief one-line comments didn't provide enough context about the
resolution chain concept or concrete examples of when fields are
populated.
## Changes
### RefExpr Structure Documentation
(`crates/tinymist-analysis/src/syntax/def.rs`)
Added comprehensive documentation including:
- Explanation of the resolution chain concept: `root` -> `step` ->
`decl`
- Three concrete examples showing different use cases:
- Simple identifier reference (`let y = x`)
- Module field access (`mod.field`)
- Import with rename (`import: old as new`)
- Detailed field documentation explaining when each is set:
- `step`: Set for imports, field access, chained references, renamed
imports
- `root`: Set for module imports, field selection, propagated in chains
- `term`: Set when type inference succeeds, `None` when type unknown or
deferred
### Expression Analysis Documentation
(`crates/tinymist-query/src/syntax/expr.rs`)
Enhanced `expr_of` function documentation to explain:
- Two-pass analysis architecture (init_stage for forward references,
then full resolution)
- How it builds the resolves map that powers goto_definition
- Caching strategy based on source content and import hashes
Added detailed documentation for helper functions:
- `resolve_ident_`: Resolution process and reference chain building
- `eval_ident`: Lexical scope lookup order and type availability
- `extract_ref`: Reference chain propagation mechanism
Added inline comments at all RefExpr creation sites explaining the
context and field values for:
- Module imports
- Import/include paths
- Named imports with rename
- Module field selection
## Example
**Before:** The RefExpr struct had minimal documentation
```rust
/// The intermediate step in resolution (if any).
pub step: Option<Expr>,
```
**After:** Clear explanation of when the field is set
```rust
/// The intermediate expression in the resolution chain.
///
/// Set in the following cases:
/// - **Import/include**: The module expression being imported
/// - **Field access**: The selected field's expression
/// - **Chained references**: When an identifier resolves to another reference
/// - **Renamed imports**: The original name before renaming
///
/// `None` when the identifier is a direct definition (not a reference).
pub step: Option<Expr>,
```
## Testing
- ✅ All core tests pass (expr module, tinymist-analysis package)
- ✅ Clippy passes with no warnings
- ✅ Build succeeds
- ✅ Code formatted with `yarn fmt`
Note: Package-related test failures (cetz, fletcher, tidy, touying,
import_package) are expected in environments without network access per
project guidelines.
> [!WARNING]
>
> <details>
> <summary>Firewall rules blocked me from connecting to one or more
addresses (expand for details)</summary>
>
> #### I tried to connect to the following addresses, but was blocked by
firewall rules:
>
> - `packages.typst.org`
> - Triggering command:
`/home/REDACTED/work/tinymist/tinymist/target/debug/deps/tinymist_query-1cacdd437723df09
--skip=e2e` (dns block)
> - Triggering command:
`/home/REDACTED/work/tinymist/tinymist/target/debug/deps/tinymist_query-1cacdd437723df09
goto_definition::tests::test --exact` (dns block)
>
> If you need me to access, download, or install something from one of
these locations, you can either:
>
> - Configure [Actions setup
steps](https://gh.io/copilot/actions-setup-steps) to set up my
environment, which run before the firewall is enabled
> - Add the appropriate URLs or hosts to the custom allowlist in this
repository's [Copilot coding agent
settings](https://github.com/Myriad-Dreamin/tinymist/settings/copilot/coding_agent)
(admins only)
>
> </details>
<!-- START COPILOT CODING AGENT SUFFIX -->
<details>
<summary>Original prompt</summary>
>
> ----
>
> *This section details on the original issue you should resolve*
>
> <issue_title>Improve code comments in expr.rs</issue_title>
> <issue_description>> I'd like to push this forward, but I need some
tips/docs about how the goto_definition works. I've walked through `fn
def_of_syntax_or_dyn`, `fn definition` ... and finally focused on
`crates/tinymist-query/src/syntax/expr.rs -> fn expr_of`. Maybe there
could have some concise code comments?
>
> _Originally posted by @BlueQuantumx in
[#1960](https://github.com/Myriad-Dreamin/tinymist/issues/1960#issuecomment-3294723078)_
>
> > I read those comments when I was originally writing the code, but
I'm mainly interested in when `type` and `root` are set (or, conversely,
when they are `None`.) The comments don't really go into that detail
AFAICS?
>
> _Originally posted by @jo3-l in
https://github.com/Myriad-Dreamin/tinymist/pull/2065#discussion_r2315906981_</issue_description>
>
> ## Comments on the Issue (you are @copilot in this section)
>
> <comments>
> </comments>
>
</details>
FixesMyriad-Dreamin/tinymist#2122
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/Myriad-Dreamin/tinymist/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Myriad-Dreamin <35292584+Myriad-Dreamin@users.noreply.github.com>
Co-authored-by: Myriad-Dreamin <camiyoru@gmail.com>
fix https://github.com/Myriad-Dreamin/tinymist/issues/1845
Fallback for tables with block-level cells now converts the Typst
`HtmlElement` tree into a `cmark_writer::ast::Node::HtmlElement` and
lets `HtmlWriter` render it.
---------
Co-authored-by: Myriad-Dreamin <camiyoru@gmail.com>
## Features
- Support multi-page export in the server export command. Added an extra
arg to decide whether to write to the file or only return data for
client use (page preview in exporter). VSCode users can use this feature
via the quick-export command or code lens.
- (refactor) Move most export logic from tinymist to tinymist-task,
excluding typlite-related, which already depends on tinymist-task.
- Added relevant export tests. The export e2e test now includes hash
checking for all targets.
## Not done
- Support new args in `TypstExtraArgs` for CLI compilation.
---------
Co-authored-by: Myriad-Dreamin <camiyoru@gmail.com>
Further fix and refactor may base on this pr.
also refactor: centralize CommonMark spacing and fix inline custom node
handling
- add `FormatPolicy` to manage trailing newline / blank-line rules and
expose prepare_block_sequence
- route document, blockquote, and list content through the policy so
nested blocks get consistent spacing
- adjust `Node::is_block` / `is_inline` to respect
`CustomNode::is_block`, preventing inline custom nodes from forcing HTML
fallbacks
- ensure table rendering stays in Markdown by recognizing inline custom
cells correctly
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
When a file, for example a rust source file, is sent to the language
server by `didOpen`, it is implicitly focused and starts a typst
compilation with the file. However, this is not expected. The error is
rarely triggered because vscode extension also explicitly send focus
events which prevents the implicit focus happens. So in conclusion this
bug was triggered stably by following steps:
- activate tinymist for some reason, and no typst source file is opened.
- open a rust source file and tinymist reports typst syntax error in the
file.
Add a lint warning for unknown math variables[^1], powered by Tinymist's
existing reference analysis. This allows users to see all such undefined
variables at once if they have the linter enabled, instead of just the
first error from the compiler. The autofix from #2062 is also applicable
to these warnings.
Example:
(The first error is pre-existing and emitted by the Typst compiler; the
second warning is new from this PR and emitted by us.)
Implementation notes:
- The generated diagnostic tries to closely match the corresponding
Typst compiler error, with one deliberate change: to differentiate the
Tinymist warning and the compiler error, we use the message `potentially
unknown variable: ...` instead of `unknown variable: ...`. I am not the
biggest fan of this choice, but I think it is very important that users
don't blame the Typst compiler for warnings that we generate; changing
the message so that it isn't an exact clone is the best way I thought of
for now.
- We avoid duplicating a warning if the compiler has already generated
an error for a given identifier by computing a set of `KnownLintIssues`
from the compiler diagnostics and threading this information through the
lint routines.
[^1]: If users like this warning, we can extend it later to apply to
undefined variables outside of math as well.
---------
Co-authored-by: Myriad-Dreamin <camiyoru@gmail.com>
This PR fixes#1796 by correcting the parent_last_loc parameter passed
to child elements when processing heading folding ranges.
This is because heading's `lsp_range` only covers the line it is on, but
it has children. This causes the `parent_last_loc` passed when
recursively calling the `calc_folding_range` function to be incorrect:
it is expected to be the end of the line before the next sibling
heading, but actually it is the end of the current heading's line. This
further prevents the correction of the heading's `folding_range` from
working properly, ultimately causing the returned `folding_range` to
retain the original `lsp_range`, which is just the heading line itself.
The fix uses the already-calculated `folding_range.end_line` for
headings when determining the `parent_last_loc` for child elements,
ensuring consistency between the parent's actual folding boundary and
the boundary passed to its children.
Hierarchy of comment group is always wrong previously, causing #2021.
This PR fixed it and added a test case.
BTW, the changes in the lexical hierarchy module influenced a folding
range test case, but it's just changed the order of items, which doesn't
affect the correctness (the order is even more correct).
Previously, Tinymist preview does not work on VS Code web (code-server)
because it tries to connect to the `localhost`, which is impossible
since we are doing remote development.
Even if we fix this error, the HTTP origin check will also fail because
the domain of our VS Code web is not trusted by Tinymist. One possible
fix is to grab the `VSCODE_PROXY_URI` environment variable and check if
the incoming request matches the URI. This will fix the issue for most
standard code-server installations.
However, GitHub Codespace is an exception. I did some logging, and found
that GitHub Codespace used different origins:
- `<random>.assets.github.dev`
- `https://localhost:<port>` (note HTTP**S**)
- `None`
This is very weird and difficult to debug. Moreover, the ports forwarded
by GitHub Codespaces require authentication before visiting, which the
VS Code webview does not support. One have to open the URL in another
browser tab to authenticate. This is frustrating and is difficult to
workaround. Therefore, currently, GitHub Codespaces are not supported.
(I asked LLM to generate some of the Rust code. It works, but may need
improvements.)
Fixes#625
- feat: move symbol svg generation logic from client to client, and
directly display svg elements instead of mask-image. This significantly
boosts loading speed.
- fix: use font ascender and descender to bound view box, avoid
clipping.
- Restructured files.
- Render non-printable symbols, following typst documentation's
convention.
- Add a toggle to show details: symbol name and unicode, both of which
can be copied to clipboard on click.
- Make detypify import async, so that it will not block symbol
rendering.
- Show detypify canvas only when available.
<img width="1289" height="421" alt="image"
src="https://github.com/user-attachments/assets/ed6469c4-57a0-4d17-a4be-206875a03e43"
/>
<img width="1238" height="632" alt="image"
src="https://github.com/user-attachments/assets/e6bba758-7836-4230-9ab9-f9550454b4de"
/>
---------
Co-authored-by: Myriad-Dreamin <camiyoru@gmail.com>
This PR improves the help message of the CLL. Also, we change the verbs
in the help message from plural (e.g. `compiles`) to singular (e.g.
`compile`). This matches the style of `clap-rs`.
Added tests about critical help messages.
> Hm. Sorry about the lint failures here, they slipped my attention
locally since they show up as warnings, not errors, in my editor. I'm
happy to put a PR fixing them later unless you get to it first.
>
> Not too sure how to deal with the unused DiagWorker::check method,
since it is used, just in a test-only module. We can certainly mark it
as #[cfg(test)] but that feels a little nasty.
https://github.com/Myriad-Dreamin/tinymist/pull/2062#issuecomment-3237812684
I ended up inlining the logic of `DiagWorker::check` into the test.
Examples:
| Input | Suggested fix |
| -- | -- |
| `$xyz$` | `$x y z$` |
| `$a_ij$` | `$a_(i j)$` |
| `$a_(ij)$` | `$a_(i j)$` |
| `$xy/z$` | `$(x y)/z$` |
If the unknown identifier appears as a subscript or as the
numerator/denominator of a fraction, we parenthesize the suggested fix.
For example, `a_ij` turns into `a_(i j)`, not `a_i j`, because the
latter is unlikely to be what the user intended.
This PR fixes a latent cache invalidation issue with project snapshots.
I recommend reviewing commit-by-commit, even though this is a small
change. Only the last commit changes behavior; the rest are cleaning up
code to make it easier to see what is happening.
---
Here's what I think is the problem.
Project instance states (`ProjectInsState`) provide access to a snapshot
of the latest compilation result. This snapshot is cached for
performance:
```rust
match self.snapshot.as_ref() {
Some(snap) if snap.world().revision() == self.verse.revision => snap.clone(),
// otherwise recompute
}
```
Unfortunately, it turns out that the cached snapshot may be outdated
even if the revision matches. (In practice, it will usually just lag one
compilation behind, and I think this is why it hasn't caused any
observable issues--one compilation behind is not that much, especially
if the user is continuously typing.)
To see the issue, consider the following sequence of steps. Assume
initially all revisions are `0`.
1. User triggers a recompilation, say by typing into the file.
- This triggers an `Interrupt::Memory` or similar.
2. The changes are applied to the vfs and recompilation begins.
- `verse.revision` is incremented to `verse.revision = 1`.
3. Before compilation finishes, a project snapshot is requested.
- The project compiler checks the cached snapshot first, which still has
`snap.world().revision() == 0`.
- Since `verse.revision = 1` was already incremented before the
compilation finished, a new snapshot is created and cached--even though
nothing has actually been recompiled yet.
- The cached snapshot now has `snap.world().revision() == 1`.
4. Compilation finishes, and some data from the resulting compilation
artifact is recorded.
5. Another project snapshot is requested.
- Since the snapshot was recomputed already in (3), the revisions appear
to match and the outdated, cached snapshot is returned.
The fix here is to just clear the cached snapshot in (4) after
recompilation finishes.
---
I would add a regression test, but it didn't seem easy to write an
automated test that can differentiate the change here. I tested this
locally by adding some logging around when the cached snapshot is used
and when it is recomputed.
As mentioned on Discord, I noticed this problem when adding some more
data to project snapshots on a local fork of Tinymist, which caused the
slightly outdated state to become a little more obvious.
---------
Co-authored-by: Myriad-Dreamin <camiyoru@gmail.com>
This PR fixes the release script to remove the `tinymist-analysis` crate
from the release process.
This PR also fixes the `crityp` crate to include the `system` feature in
the `cli` feature.
This PR adds comprehensive documentation for all public items in
`crates/tinymist-query/src/syntax/expr.rs` following Rust documentation
conventions.
## Changes Made
The following public items now have proper documentation comments:
- **`ExprRoute`** type alias - Documents its purpose as a mapping
structure for file identifiers to lexical scopes during expression
analysis routing
- **`expr_of`** function - Describes its role in analyzing expressions
within source files and producing expression information including
resolves, imports, docstrings, and lexical scoping data
- **`ExprWorker`** struct - Explains its functionality as a worker for
processing expressions during source file analysis, handling expression
checking, scope management, symbol resolution, and docstring collection
## Documentation Style
All documentation follows Rust conventions by:
- Using third person singular verbs
- Focusing on the purpose and functionality rather than implementation
details
- Providing clear, concise descriptions of what each item does
The code has been formatted with `cargo fmt` and compiles successfully
after the changes.
Fixes#2048.
<!-- START COPILOT CODING AGENT TIPS -->
---
💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Myriad-Dreamin <35292584+Myriad-Dreamin@users.noreply.github.com>
Co-authored-by: Myriad-Dreamin <camiyoru@gmail.com>
This PR adds comprehensive documentation comments to all public items in
`crates/tinymist-analysis/src/syntax/def.rs`. The file contained
numerous public structs, enums, functions, and fields that lacked
documentation, making it difficult for contributors to understand the
purpose and usage of these types in the syntax analysis system.
## Changes Made
- **Public Types**: Added documentation to 40+ public types including
core expression types (`ExprInfo`, `ExprInfoRepr`, `Expr`), declaration
types (`Decl`, `DefKind`), and expression-specific structs
- **Public Functions**: Added documentation to 60+ public functions
including:
- Constructor methods for all declaration types (`Decl::func`,
`Decl::var`, etc.)
- Utility methods across all major types (`get_def`, `is_exported`,
`span`, etc.)
- Expression and operation constructors (`SelectExpr::new`,
`UnInst::new`, etc.)
- **Public Fields**: Added documentation to all public fields in 35+
structs, explaining their purpose and usage context
- **Type Aliases**: Documented utility types like `LexicalScope`,
`DeclExpr`, `ExportMap`
All documentation follows the existing codebase style using triple-slash
comments (`///`) and focuses on explaining the purpose, usage context,
and behavior of each item within the syntax analysis pipeline.
## Verification
- Code formatting applied using `cargo fmt`
- No functional changes made - purely documentation additions
- All public API items in the file now have appropriate documentation
This significantly improves the developer experience for anyone working
with the syntax analysis components of tinymist.
Fixes#2023.
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/Myriad-Dreamin/tinymist/issues/new?title=✨Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Myriad-Dreamin <35292584+Myriad-Dreamin@users.noreply.github.com>
Co-authored-by: Myriad-Dreamin <camiyoru@gmail.com>
