mirror of
https://github.com/typst/typst
synced 2025-05-14 04:56:26 +08:00
41 lines
2.0 KiB
Markdown
41 lines
2.0 KiB
Markdown
# typst-syntax
|
|
|
|
Welcome to the Typst Syntax crate! This crate manages the syntactical structure
|
|
of Typst by holding some core abstractions like assigning source file ids,
|
|
parsing Typst syntax, creating an Abstract Syntax Tree (AST), initializing
|
|
source "spans" (for linking AST elements to their outputs in a document), and
|
|
syntax highlighting.
|
|
|
|
Below are quick descriptions of the files you might be editing if you find
|
|
yourself here :)
|
|
|
|
- `lexer.rs`: The lexical foundation of the parser, which converts a string of
|
|
characters into tokens.
|
|
- `parser.rs`: The main parser definition, preparing a Concrete Syntax Tree made
|
|
of nested vectors of `SyntaxNode`s.
|
|
- `reparser.rs`: The algorithm for reparsing the minimal required amount of
|
|
source text for efficient incremental compilation.
|
|
- `ast.rs`: The conversion layer between the Concrete Syntax Tree of the parser
|
|
and the Abstract Syntax Tree used for code evaluation.
|
|
- `node.rs` & `span.rs`: The underlying data structure for the Concrete Syntax
|
|
Tree and the definitions of source spans used for efficiently pointing to a
|
|
syntax node in things like diagnostics.
|
|
- `kind.rs` & `set.rs`: An enum with all syntactical tokens and nodes and
|
|
bit-set data structure for sets of `SyntaxKind`s.
|
|
- `highlight.rs`: Extracting of syntax highlighting information out of the
|
|
Concrete Syntax Tree (and outputting as HTML).
|
|
- `path.rs`, `file.rs`, `package.rs`: The system for interning project and
|
|
package paths as unique file IDs and resolving them in a virtual filesystem
|
|
(not actually for _opening_ files).
|
|
|
|
The structure of the parser is largely adapted from Rust Analyzer. Their
|
|
[documentation][ra] is a good reference for a number of the design decisions
|
|
around the parser and AST.
|
|
|
|
The reparsing algorithm is explained in Section 4 of [Martin's thesis][thesis]
|
|
(though it changed a bit since).
|
|
|
|
[ra]: https://github.com/rust-lang/rust-analyzer/blob/master/docs/dev/syntax.md
|
|
[thesis]:
|
|
https://www.researchgate.net/publication/364622490_Fast_Typesetting_with_Incremental_Compilation
|