## Overview
This PR enhances documentation in the expression analysis system to
clarify how goto_definition works and when the `RefExpr` fields are set
versus `None`. This addresses feedback from @BlueQuantumx and @jo3-l who
requested better documentation to understand the expression analysis
flow.
## Problem
The existing documentation was insufficient for developers trying to
understand:
1. How the goto_definition feature works through the expression analysis
system
2. When the `root`, `step`, and `term` fields in `RefExpr` are `Some`
versus `None`
3. The overall flow of expression analysis in `expr_of`
The brief one-line comments didn't provide enough context about the
resolution chain concept or concrete examples of when fields are
populated.
## Changes
### RefExpr Structure Documentation
(`crates/tinymist-analysis/src/syntax/def.rs`)
Added comprehensive documentation including:
- Explanation of the resolution chain concept: `root` -> `step` ->
`decl`
- Three concrete examples showing different use cases:
- Simple identifier reference (`let y = x`)
- Module field access (`mod.field`)
- Import with rename (`import: old as new`)
- Detailed field documentation explaining when each is set:
- `step`: Set for imports, field access, chained references, renamed
imports
- `root`: Set for module imports, field selection, propagated in chains
- `term`: Set when type inference succeeds, `None` when type unknown or
deferred
### Expression Analysis Documentation
(`crates/tinymist-query/src/syntax/expr.rs`)
Enhanced `expr_of` function documentation to explain:
- Two-pass analysis architecture (init_stage for forward references,
then full resolution)
- How it builds the resolves map that powers goto_definition
- Caching strategy based on source content and import hashes
Added detailed documentation for helper functions:
- `resolve_ident_`: Resolution process and reference chain building
- `eval_ident`: Lexical scope lookup order and type availability
- `extract_ref`: Reference chain propagation mechanism
Added inline comments at all RefExpr creation sites explaining the
context and field values for:
- Module imports
- Import/include paths
- Named imports with rename
- Module field selection
## Example
**Before:** The RefExpr struct had minimal documentation
```rust
/// The intermediate step in resolution (if any).
pub step: Option<Expr>,
```
**After:** Clear explanation of when the field is set
```rust
/// The intermediate expression in the resolution chain.
///
/// Set in the following cases:
/// - **Import/include**: The module expression being imported
/// - **Field access**: The selected field's expression
/// - **Chained references**: When an identifier resolves to another reference
/// - **Renamed imports**: The original name before renaming
///
/// `None` when the identifier is a direct definition (not a reference).
pub step: Option<Expr>,
```
## Testing
- ✅ All core tests pass (expr module, tinymist-analysis package)
- ✅ Clippy passes with no warnings
- ✅ Build succeeds
- ✅ Code formatted with `yarn fmt`
Note: Package-related test failures (cetz, fletcher, tidy, touying,
import_package) are expected in environments without network access per
project guidelines.
> [!WARNING]
>
> <details>
> <summary>Firewall rules blocked me from connecting to one or more
addresses (expand for details)</summary>
>
> #### I tried to connect to the following addresses, but was blocked by
firewall rules:
>
> - `packages.typst.org`
> - Triggering command:
`/home/REDACTED/work/tinymist/tinymist/target/debug/deps/tinymist_query-1cacdd437723df09
--skip=e2e` (dns block)
> - Triggering command:
`/home/REDACTED/work/tinymist/tinymist/target/debug/deps/tinymist_query-1cacdd437723df09
goto_definition::tests::test --exact` (dns block)
>
> If you need me to access, download, or install something from one of
these locations, you can either:
>
> - Configure [Actions setup
steps](https://gh.io/copilot/actions-setup-steps) to set up my
environment, which run before the firewall is enabled
> - Add the appropriate URLs or hosts to the custom allowlist in this
repository's [Copilot coding agent
settings](https://github.com/Myriad-Dreamin/tinymist/settings/copilot/coding_agent)
(admins only)
>
> </details>
<!-- START COPILOT CODING AGENT SUFFIX -->
<details>
<summary>Original prompt</summary>
>
> ----
>
> *This section details on the original issue you should resolve*
>
> <issue_title>Improve code comments in expr.rs</issue_title>
> <issue_description>> I'd like to push this forward, but I need some
tips/docs about how the goto_definition works. I've walked through `fn
def_of_syntax_or_dyn`, `fn definition` ... and finally focused on
`crates/tinymist-query/src/syntax/expr.rs -> fn expr_of`. Maybe there
could have some concise code comments?
>
> _Originally posted by @BlueQuantumx in
[#1960](https://github.com/Myriad-Dreamin/tinymist/issues/1960#issuecomment-3294723078)_
>
> > I read those comments when I was originally writing the code, but
I'm mainly interested in when `type` and `root` are set (or, conversely,
when they are `None`.) The comments don't really go into that detail
AFAICS?
>
> _Originally posted by @jo3-l in
https://github.com/Myriad-Dreamin/tinymist/pull/2065#discussion_r2315906981_</issue_description>
>
> ## Comments on the Issue (you are @copilot in this section)
>
> <comments>
> </comments>
>
</details>
FixesMyriad-Dreamin/tinymist#2122
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/Myriad-Dreamin/tinymist/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Myriad-Dreamin <35292584+Myriad-Dreamin@users.noreply.github.com>
Co-authored-by: Myriad-Dreamin <camiyoru@gmail.com>
Add a lint warning for unknown math variables[^1], powered by Tinymist's
existing reference analysis. This allows users to see all such undefined
variables at once if they have the linter enabled, instead of just the
first error from the compiler. The autofix from #2062 is also applicable
to these warnings.
Example:
(The first error is pre-existing and emitted by the Typst compiler; the
second warning is new from this PR and emitted by us.)
Implementation notes:
- The generated diagnostic tries to closely match the corresponding
Typst compiler error, with one deliberate change: to differentiate the
Tinymist warning and the compiler error, we use the message `potentially
unknown variable: ...` instead of `unknown variable: ...`. I am not the
biggest fan of this choice, but I think it is very important that users
don't blame the Typst compiler for warnings that we generate; changing
the message so that it isn't an exact clone is the best way I thought of
for now.
- We avoid duplicating a warning if the compiler has already generated
an error for a given identifier by computing a set of `KnownLintIssues`
from the compiler diagnostics and threading this information through the
lint routines.
[^1]: If users like this warning, we can extend it later to apply to
undefined variables outside of math as well.
---------
Co-authored-by: Myriad-Dreamin <camiyoru@gmail.com>
This PR adds comprehensive documentation comments to all public items in
`crates/tinymist-analysis/src/syntax/def.rs`. The file contained
numerous public structs, enums, functions, and fields that lacked
documentation, making it difficult for contributors to understand the
purpose and usage of these types in the syntax analysis system.
## Changes Made
- **Public Types**: Added documentation to 40+ public types including
core expression types (`ExprInfo`, `ExprInfoRepr`, `Expr`), declaration
types (`Decl`, `DefKind`), and expression-specific structs
- **Public Functions**: Added documentation to 60+ public functions
including:
- Constructor methods for all declaration types (`Decl::func`,
`Decl::var`, etc.)
- Utility methods across all major types (`get_def`, `is_exported`,
`span`, etc.)
- Expression and operation constructors (`SelectExpr::new`,
`UnInst::new`, etc.)
- **Public Fields**: Added documentation to all public fields in 35+
structs, explaining their purpose and usage context
- **Type Aliases**: Documented utility types like `LexicalScope`,
`DeclExpr`, `ExportMap`
All documentation follows the existing codebase style using triple-slash
comments (`///`) and focuses on explaining the purpose, usage context,
and behavior of each item within the syntax analysis pipeline.
## Verification
- Code formatting applied using `cargo fmt`
- No functional changes made - purely documentation additions
- All public API items in the file now have appropriate documentation
This significantly improves the developer experience for anyone working
with the syntax analysis components of tinymist.
Fixes#2023.
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/Myriad-Dreamin/tinymist/issues/new?title=✨Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Myriad-Dreamin <35292584+Myriad-Dreamin@users.noreply.github.com>
Co-authored-by: Myriad-Dreamin <camiyoru@gmail.com>
using clippy 0.1.91 (898aff704d 2025-08-14), but some warnings already
exist in the latest stable version
mostly about elided lifetime and if-chain
only remaining warnings:
```
warning: struct `HashRepr` is never constructed
--> crates\tinymist-query\src\tests.rs:462:12
|
462 | pub struct HashRepr<T>(pub T);
| ^^^^^^^^
|
= note: `#[warn(dead_code)]` on by default
warning: a method with this name may be added to the standard library in the future
--> crates\tinymist\src\actor\editor.rs:103:30
|
103 | ... .map_or_default(|fid| unix_slash(fid.vpath().as_rooted_path()));
| ^^^^^^^^^^^^^^
|
= warning: once this associated item is added to the standard library, the ambiguity may cause an error or change in behavior!
= note: for more information, see issue #48919 <https://github.com/rust-lang/rust/issues/48919>
= help: call with fully qualified syntax `typst::typst_utils::OptionExt::map_or_default(...)` to keep using the current method
= note: `#[warn(unstable_name_collisions)]` on by default
help: add `#![feature(result_option_map_or_default)]` to the crate attributes to enable `std::option::Option::<T>::map_or_default`
--> crates\tinymist\src\lib.rs:3:1
|
3 + #![feature(result_option_map_or_default)]
|
```
* dev: init markdown file
* dev: typlite element derive
* feat: pass tests
refactor lib.rs to separated files (#1692)
feat(typlite): Docx export and export markdown in cmark-writer (#1698)
* feat: docx export support
* refactor: simplify DocxConverter structure and improve content handling
* tests: add binary insta for docx
* feat: add MathBlock style and improve frame rendering in DocxConverter
* fix: enhance paragraph creation(silly method)
* fix: enhance math equation rendering
* use md5 instead of docx binary
* feat: enhance list numbering and paragraph handling in DocxConverter
* feat: add all_elements test
* refactor
* reimpl md export in cmark-writer
* feat: add support for highlight tag in MarkdownConverter
* feat: refactor LaTeXConverter to improve element processing and add new methods
* fmt
* Refactor DOCX converter to improve list handling and document structure
- Introduced separate methods for creating ordered and unordered list numbering.
- Enhanced list management by tracking next numbering IDs.
- Consolidated paragraph and run management within the DocxConverter.
- Improved image processing with better error handling and placeholder support.
- Streamlined the handling of various HTML elements, including headings, lists, and images.
- Added functionality for processing captions and preformatted blocks.
- Updated methods for processing inline styles and links.
* feat: update cmark-writer to version 0.2.0
* feat: refactor code block handling in DOCX converter for improved readability
* refactor: refactor DOCX converter to enhance document structure
* refactor docx to separated files
* update instas
* fmt
* chore: update cmark-writer version to 0.3.0
* fix: ol custom value
* feat: table and grid processing
* use cmark-writer's ast node for consistency
* fix: update snapshot hashes for document generation tests
* fix: add preamble
* update snapshot hashes
* refactor DOCX conversion: Split writer functionality into separate module, enhance image processing, and clean up utility functions
* update comments in LaTeX and Markdown converters for clarity and consistency
* fmt
* delete utils
* feat: support figure node by custom node in cmark-writer
* fix
* fix: frame
* feat: enhance table conversion logic in MarkdownConverter
* refactor: simplify FigureNode implementation by removing CustomNode trait
* chore: update cmark-writer to version 0.5.0
* fix: update figure and raw inline snapshots for consistency
* fix: update snapshot hashes and correct caption reference in markdown.typ
* refactor proj structure
* feat: update CompileArgs to support multiple output files and remove debug option
* docs: update README to clarify usage of multiple output formats and comment out feature section
* remove DocxConverter module
* feat: impl assets-path feature and add ExternalFrameNode for handling external frames and update writers to support it
* feat: enhance HTML element conversion to include attributes and children handling
* feat: update cmark-writer to 0.6.1 and refactor related code
* fix: update snapshots for figure caption, list, outline, and docx generation
* feat: refactor HTML element conversion to use create_html_element method and enhance table processing
* fix
* feat: add HighlightNode for highlighted text and integrate with HTML to AST parser and LaTeX writer
* refactoor
* update tests
Co-Authored-By: Hong Jiarong <me@jrhim.com>
* feat: revert latex/docx conversions
* fix: warnings
* bad: convert docs
* build: remove other cargo deps
* build: update cargo.lock
* test: update snapshot
* chore: remove useless parser trait
* feat: annotate v1
* feat: annotate v2
* test: update snapshot
* question: is it a bug?
* test: update bad snapshot
---------
Co-authored-by: Hong Jiarong <me@jrhim.com>
* feat: add path conversion actions for absolute and relative paths in special function call
* feat: implement matchers
* docs: edit comment
* fix: path on windows
* feat: add a comment
* dev: edit a bit
* dev: use `resolved`
* refactor: simplify path rewriting logic using `diff`
* feat: add absolute path import fixture
* fix: update path check for absolute paths to use `starts_with` to work with windows
* feat: add path expression import fixture
---------
Co-authored-by: Myriad-Dreamin <camiyoru@gmail.com>
