This basically reverts the earlier change from parbreaks to par nodes because:
- It is simpler and less nested
- It works way better with functions that layout their body inline like `font`, which where buggy before, previously
The original reasons for changing to par nodes were:
- the envisioned design of the layouter at that time (based on dynamic nodes etc.), which is not relevant anymore
- possibly existing benefits with regards to incremental compilation, which are unsure and outweighed by the immediate benefits of the parbreak-representation
- Refactors the tokenizer to be lazy: It does not emit pre-parsed function tokens, but instead allows it's mode to be changed. The modes are tracked on a stack to allow nested compute/typesetting (pop/push).
- Introduces delimited groups into the parser, which make it easy to parse delimited expressions without handling the delimiters in the parsing code for the group's content. A group is started with `start_group`. When reaching the group's end (matching delimiter) the eat and peek methods will simply return `None` instead of the delimiter, stopping the content parser and bubbling up the call stack until `end_group` is called to clear up the situation.
- In addition to syntax trees there are now `Value`s, which syntax trees can be evaluated into (e.g. the tree is `5+5` and the value is `10`)
- Parsing is completely pure, function calls are not parsed into nodes, but into simple call expressions, which are resolved later
- Functions aren't dynamic nodes anymore, but simply functions which receive their arguments as a table and the layouting context
- Functions may return any `Value`
- Layouting is powered by functions which return the new `Commands` value, which informs the layouting engine what to do
- When a function returns a non-`Commands` value, the layouter simply dumps the value into the document in monospace
Previously they were passed as strings to the function parser, now they are parsed and then passed as trees to the function. This allows making bodies sugar for a last content argument. While it removes some flexibility allowing function to parse arbitrary syntaxes in their bodies, these can be modelled as (raw) string arguments.
- Problems -> Diagnostics
- Position -> Pos
- offset_spans -> Offset trait
- Size -> Length (and some more size types renamed)
- Paper into its own module
- scope::Parser -> parsing::CallParser
- Create `Decorations` alias
- Remove lots of double newlines
- Switch from f32 to f64
- Forced line breaks with backslash followed by whitespace
- (Multline) raw text in backticks
- Set font class fallbacks with [font.family] (e.g. [font.family: monospace=("CMU Typewriter Text")])
- More sophisticated procedure to find end of function, which accounts for comments, strings, raw text and nested functions (this is a mix of a feature and a bug fix)
