HTML tests (#5594)

Co-authored-by: Laurenz <laurmaedje@gmail.com>

CONTRIBUTING.md

@@ -37,8 +37,8 @@ Below are some signs of a good PR:
 - Adds/changes as little code and as few interfaces as possible. Should changes
   to larger-scale abstractions be necessary, these should be discussed
   throughout the implementation process.
-- Adds tests if appropriate (with reference images for visual tests). See the
+- Adds tests if appropriate (with reference output for visual/HTML tests). See
-  [testing] readme for more details.
+  the [testing] readme for more details.
 - Contains documentation comments on all new Rust types.
 - Comes with brief documentation for all new Typst definitions
   (elements/functions), ideally with a concise example that fits into ~5-10

Cargo.lock

@@ -3076,6 +3076,7 @@ dependencies = [
  "typst",
  "typst-assets",
  "typst-dev-assets",
+ "typst-html",
  "typst-library",
  "typst-pdf",
  "typst-render",

tests/.gitattributes

@@ -0,0 +1 @@
+*.html text eol=lf

tests/Cargo.toml

@@ -19,6 +19,7 @@ default = [
     "typst",
     "typst-assets",
     "typst-dev-assets",
+    "typst-html",
     "typst-library",
     "typst-pdf",
     "typst-render",
@@ -33,6 +34,7 @@ typst-syntax = { workspace = true }
 typst = { workspace = true, optional = true }
 typst-assets = { workspace = true, features = ["fonts"], optional = true }
 typst-dev-assets = { workspace = true, optional = true }
+typst-html = { workspace = true, optional = true }
 typst-library = { workspace = true, optional = true }
 typst-pdf = { workspace = true, optional = true }
 typst-render = { workspace = true, optional = true }

tests/README.md

@@ -6,8 +6,8 @@ Top level directory structure:
 - `suite`: Input files. Mostly organized in parallel to the code. Each file can
            contain multiple tests, each of which is a section of Typst code
            following `--- {name} ---`.
-- `ref`: Reference images which the output is compared with to determine whether
+- `ref`: References which the output is compared with to determine whether a
-         a test passed or failed.
+         test passed or failed.
 - `store`: Store for PNG, PDF, and SVG output files produced by the tests.
 
 ## Running the tests
@@ -54,18 +54,29 @@ You may find more options in the help message:
 testit --help
 ```
 
-To make the integration tests go faster they don't generate PDFs by default.
+To make the integration tests go faster they don't generate PDFs or SVGs by
-Pass the `--pdf` flag to generate those. Mind that PDFs are not tested
+default. Pass the `--pdf` or `--svg` flag to generate those. Mind that PDFs and
-automatically at the moment, so you should always check the output manually when
+SVGs are **not** tested automatically at the moment, so you should always check
-making changes.
+the output manually when making changes.
 ```bash
 testit --pdf
 ```
 
 ## Writing tests
-The syntax for an individual test is `--- {name} ---` followed by some Typst
+The syntax for an individual test is `--- {name} {attr}* ---` followed by some
-code that should be tested. The name must be globally unique in the test suite,
+Typst code that should be tested. The name must be globally unique in the test
-so that tests can be easily migrated across files.
+suite, so that tests can be easily migrated across files. A test name can be
+followed by space-separated attributes. For instance, `--- my-test html ---`
+adds the `html` modifier to `my-test`, instructing the test runner to also
+test HTML output. The following attributes are currently defined:
+
+- `render`: Tests paged output against a reference image (the default, only
+  needs to be specified when `html` is also specified to enable both at the
+  same)
+- `html`: Tests HTML output against a reference HTML file. Disables the `render`
+  default.
+- `large`: Permits a reference image size exceeding 20 KiB. Should be used
+  sparingly.
 
 There are, broadly speaking, three kinds of tests:
 
@@ -80,35 +91,42 @@ There are, broadly speaking, three kinds of tests:
   below. If the code span is in a line further below, you can write ranges
   like `3:2-3:7` to indicate the 2-7 column in the 3rd non-comment line.
 
-- Tests that ensure certain visual output is produced: Those render the result
+- Tests that ensure certain output is produced:
-  of the test with the `typst-render` crate and compare against a reference
+
-  image stored in the repository. The test runner automatically detects whether
+  - Visual output: By default, the compiler produces paged output, renders it
-  a test has visual output and requires a reference image in this case.
+    with the `typst-render` crate, and compares it against a reference image
+    stored in the repository. The test runner automatically detects whether a
+    test has visual output and requires a reference image in this case.
 
     To prevent bloat, it is important that the test images are kept as small as
     possible. To that effect, the test runner enforces a maximum size of 20 KiB.
-  If you're updating a test and hit `reference image size exceeds`, see the
+    If you're updating a test and hit `reference output size exceeds`, see the
     section on "Updating reference images" below. If truly necessary, the size
-  limit can be lifted by adding `// LARGE` as the first line of a test, but this
+    limit can be lifted by adding a `large` attribute after the test name, but
-  should be the case very rarely.
+    this should be the case very rarely.
+
+  - HTML output: When a test has the `html` attribute, the compiler produces
+    HTML output and compares it against a reference file stored in the
+    repository. By default, this enables testing of paged output, but you can
+    test both at once by passing both `render` and `html` as attributes.
 
 If you have the choice between writing a test using assertions or using
 reference images, prefer assertions. This makes the test easier to understand
 in isolation and prevents bloat due to images.
 
 ## Updating reference images
-If you created a new test or fixed a bug in an existing test, you need to update
+If you created a new test or fixed a bug in an existing test, you may need to
-the reference image used for comparison. For this, you can use the `--update`
+update the reference output used for comparison. For this, you can use the
-flag:
+`--update` flag:
 ```bash
 testit --exact my-test-name --update
 ```
 
-This will generally generate compressed reference images (to remain within the
+For visual tests, this will generally generate compressed reference images (to
-above size limit).
+remain within the size limit).
 
 If you use the VS Code test helper extension (see the `tools` folder), you can
-alternatively use the save button to update the reference image.
+alternatively use the save button to update the reference output.
 
 ## Making an alias
 If you want to have a quicker way to run the tests, consider adding a shortcut

tests/ref/html/link-basic.html

@@ -0,0 +1,21 @@
+<!DOCTYPE html>
+<html>
+  <head>
+    <meta charset="utf-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+  </head>
+  <body>
+    <p>
+      <a href="https://example.com/">https://example.com/</a>
+    </p>
+    <p>
+      <a href="https://typst.org/">Some text text text</a>
+    </p>
+    <p>
+      This link appears <a href="https://google.com/">in the middle of</a> a paragraph.
+    </p>
+    <p>
+      Contact <a href="mailto:hi@typst.app">hi@typst.app</a> or call <a href="tel:123">123</a> for more information.
+    </p>
+  </body>
+</html>

tests/src/args.rs

@@ -23,7 +23,7 @@ pub struct CliArguments {
     /// Lists what tests will be run, without actually running them.
     #[arg(long, group = "action")]
     pub list: bool,
-    /// Updates the reference images of non-passing tests.
+    /// Updates the reference output of non-passing tests.
     #[arg(short, long, group = "action")]
     pub update: bool,
     /// The scaling factor to render the output image with.
@@ -100,6 +100,6 @@ impl CliArguments {
 pub enum Command {
     /// Clears the on-disk test artifact store.
     Clean,
-    /// Deletes all dangling reference images.
+    /// Deletes all dangling reference output.
     Undangle,
 }

tests/src/collect.rs

@@ -23,9 +23,9 @@ pub fn collect() -> Result<(Vec<Test>, usize), Vec<TestParseError>> {
 pub struct Test {
     pub pos: FilePos,
     pub name: EcoString,
+    pub attrs: Vec<Attr>,
     pub source: Source,
     pub notes: Vec<Note>,
-    pub large: bool,
 }
 
 impl Display for Test {
@@ -57,6 +57,14 @@ impl Display for FilePos {
     }
 }
 
+/// A test attribute, given after the test name.
+#[derive(Clone, Debug, PartialEq)]
+pub enum Attr {
+    Html,
+    Render,
+    Large,
+}
+
 /// The size of a file.
 pub struct FileSize(pub usize);
 
@@ -109,8 +117,7 @@ impl Display for NoteKind {
 struct Collector {
     tests: Vec<Test>,
     errors: Vec<TestParseError>,
-    seen: HashMap<EcoString, FilePos>,
+    seen: HashMap<EcoString, (FilePos, Vec<Attr>)>,
-    large: HashSet<EcoString>,
     skipped: usize,
 }
 
@@ -121,7 +128,6 @@ impl Collector {
             tests: vec![],
             errors: vec![],
             seen: HashMap::new(),
-            large: HashSet::new(),
             skipped: 0,
         }
     }
@@ -156,7 +162,7 @@ impl Collector {
         }
     }
 
-    /// Walks through all reference images and ensure that a test exists for
+    /// Walks through all reference output and ensures that a test exists for
     /// each one.
     fn walk_references(&mut self) {
         for entry in walkdir::WalkDir::new(crate::REF_PATH).sort_by_file_name() {
@@ -169,20 +175,20 @@ impl Collector {
             let stem = path.file_stem().unwrap().to_string_lossy();
             let name = &*stem;
 
-            let Some(pos) = self.seen.get(name) else {
+            let Some((pos, attrs)) = self.seen.get(name) else {
                 self.errors.push(TestParseError {
                     pos: FilePos::new(path, 0),
-                    message: "dangling reference image".into(),
+                    message: "dangling reference output".into(),
                 });
                 continue;
             };
 
             let len = path.metadata().unwrap().len() as usize;
-            if !self.large.contains(name) && len > crate::REF_LIMIT {
+            if !attrs.contains(&Attr::Large) && len > crate::REF_LIMIT {
                 self.errors.push(TestParseError {
                     pos: pos.clone(),
                     message: format!(
-                        "reference image size exceeds {}, but the test is not marked as `// LARGE`",
+                        "reference output size exceeds {}, but the test is not marked as `large`",
                         FileSize(crate::REF_LIMIT),
                     ),
                 });
@@ -218,6 +224,7 @@ impl<'a> Parser<'a> {
 
         while !self.s.done() {
             let mut name = EcoString::new();
+            let mut attrs = Vec::new();
             let mut notes = vec![];
             if self.s.eat_if("---") {
                 self.s.eat_while(' ');
@@ -228,8 +235,8 @@ impl<'a> Parser<'a> {
                     self.error("expected test name");
                 } else if !is_ident(&name) {
                     self.error(format!("test name `{name}` is not a valid identifier"));
-                } else if !self.s.eat_if("---") {
+                } else {
-                    self.error("expected closing ---");
+                    attrs = self.parse_attrs();
                 }
             } else {
                 self.error("expected opening ---");
@@ -247,7 +254,7 @@ impl<'a> Parser<'a> {
             self.test_start_line = self.line;
 
             let pos = FilePos::new(self.path, self.test_start_line);
-            self.collector.seen.insert(name.clone(), pos.clone());
+            self.collector.seen.insert(name.clone(), (pos.clone(), attrs.clone()));
 
             while !self.s.done() && !self.s.at("---") {
                 self.s.eat_until(is_newline);
@@ -257,10 +264,6 @@ impl<'a> Parser<'a> {
             }
 
             let text = self.s.from(start);
-            let large = text.starts_with("// LARGE");
-            if large {
-                self.collector.large.insert(name.clone());
-            }
 
             if !selected(&name, self.path.canonicalize().unwrap()) {
                 self.collector.skipped += 1;
@@ -285,10 +288,33 @@ impl<'a> Parser<'a> {
                 }
             }
 
-            self.collector.tests.push(Test { pos, name, source, notes, large });
+            self.collector.tests.push(Test { pos, name, source, notes, attrs });
         }
     }
 
+    fn parse_attrs(&mut self) -> Vec<Attr> {
+        let mut attrs = vec![];
+        while !self.s.eat_if("---") {
+            let attr = match self.s.eat_until(char::is_whitespace) {
+                "large" => Attr::Large,
+                "html" => Attr::Html,
+                "render" => Attr::Render,
+                found => {
+                    self.error(format!(
+                        "expected attribute or closing ---, found `{found}`"
+                    ));
+                    break;
+                }
+            };
+            if attrs.contains(&attr) {
+                self.error(format!("duplicate attribute {attr:?}"));
+            }
+            attrs.push(attr);
+            self.s.eat_while(' ');
+        }
+        attrs
+    }
+
     /// Skips the preamble of a test.
     fn skip_preamble(&mut self) {
         let mut errored = false;

tests/src/logger.rs

@@ -9,8 +9,8 @@ pub struct TestResult {
     pub errors: String,
     /// The info log for this test.
     pub infos: String,
-    /// Whether the image was mismatched.
+    /// Whether the output was mismatched.
-    pub mismatched_image: bool,
+    pub mismatched_output: bool,
 }
 
 /// Receives status updates by individual test runs.
@@ -19,7 +19,7 @@ pub struct Logger<'a> {
     passed: usize,
     failed: usize,
     skipped: usize,
-    mismatched_image: bool,
+    mismatched_output: bool,
     active: Vec<&'a Test>,
     last_change: Instant,
     temp_lines: usize,
@@ -34,7 +34,7 @@ impl<'a> Logger<'a> {
             passed: 0,
             failed: 0,
             skipped,
-            mismatched_image: false,
+            mismatched_output: false,
             active: vec![],
             temp_lines: 0,
             last_change: Instant::now(),
@@ -73,7 +73,7 @@ impl<'a> Logger<'a> {
             self.failed += 1;
         }
 
-        self.mismatched_image |= result.mismatched_image;
+        self.mismatched_output |= result.mismatched_output;
         self.last_change = Instant::now();
 
         self.print(move |out| {
@@ -102,8 +102,8 @@ impl<'a> Logger<'a> {
         eprintln!("{passed} passed, {failed} failed, {skipped} skipped");
         assert_eq!(selected, passed + failed, "not all tests were executed successfully");
 
-        if self.mismatched_image {
+        if self.mismatched_output {
-            eprintln!("  pass the --update flag to update the reference images");
+            eprintln!("  pass the --update flag to update the reference output");
         }
 
         self.failed == 0

tests/src/run.rs

@@ -1,16 +1,17 @@
 use std::fmt::Write;
 use std::ops::Range;
-use std::path::Path;
+use std::path::PathBuf;
 
 use ecow::eco_vec;
 use tiny_skia as sk;
 use typst::diag::{SourceDiagnostic, Warned};
-use typst::layout::{Abs, Frame, FrameItem, Page, PagedDocument, Transform};
+use typst::html::HtmlDocument;
+use typst::layout::{Abs, Frame, FrameItem, PagedDocument, Transform};
 use typst::visualize::Color;
-use typst::WorldExt;
+use typst::{Document, WorldExt};
 use typst_pdf::PdfOptions;
 
-use crate::collect::{FileSize, NoteKind, Test};
+use crate::collect::{Attr, FileSize, NoteKind, Test};
 use crate::logger::TestResult;
 use crate::world::TestWorld;
 
@@ -50,7 +51,7 @@ impl<'a> Runner<'a> {
             result: TestResult {
                 errors: String::new(),
                 infos: String::new(),
-                mismatched_image: false,
+                mismatched_output: false,
             },
             not_annotated: String::new(),
         }
@@ -62,6 +63,23 @@ impl<'a> Runner<'a> {
             log!(into: self.result.infos, "tree: {:#?}", self.test.source.root());
         }
 
+        let html = self.test.attrs.contains(&Attr::Html);
+        let render = !html || self.test.attrs.contains(&Attr::Render);
+        if render {
+            self.run_test::<PagedDocument>();
+        }
+        if html {
+            self.run_test::<HtmlDocument>();
+        }
+
+        self.handle_not_emitted();
+        self.handle_not_annotated();
+
+        self.result
+    }
+
+    /// Run test specific to document format.
+    fn run_test<D: OutputType>(&mut self) {
         let Warned { output, warnings } = typst::compile(&self.world);
         let (doc, errors) = match output {
             Ok(doc) => (Some(doc), eco_vec![]),
@@ -72,8 +90,8 @@ impl<'a> Runner<'a> {
             log!(self, "no document, but also no errors");
         }
 
-        self.check_custom(doc.as_ref());
+        D::check_custom(self, doc.as_ref());
-        self.check_document(doc.as_ref());
+        self.check_output(doc.as_ref());
 
         for error in &errors {
             self.check_diagnostic(NoteKind::Error, error);
@@ -82,11 +100,6 @@ impl<'a> Runner<'a> {
         for warning in &warnings {
             self.check_diagnostic(NoteKind::Warning, warning);
         }
-
-        self.handle_not_emitted();
-        self.handle_not_annotated();
-
-        self.result
     }
 
     /// Handle errors that weren't annotated.
@@ -113,86 +126,42 @@ impl<'a> Runner<'a> {
         }
     }
 
-    /// Run custom checks for which it is not worth to create special
-    /// annotations.
-    fn check_custom(&mut self, doc: Option<&PagedDocument>) {
-        let errors = crate::custom::check(self.test, &self.world, doc);
-        if !errors.is_empty() {
-            log!(self, "custom check failed");
-            for line in errors.lines() {
-                log!(self, "  {line}");
-            }
-        }
-    }
-
     /// Check that the document output is correct.
-    fn check_document(&mut self, document: Option<&PagedDocument>) {
+    fn check_output<D: OutputType>(&mut self, document: Option<&D>) {
-        let live_path = format!("{}/render/{}.png", crate::STORE_PATH, self.test.name);
+        let live_path = D::live_path(&self.test.name);
-        let ref_path = format!("{}/{}.png", crate::REF_PATH, self.test.name);
+        let ref_path = D::ref_path(&self.test.name);
-        let has_ref = Path::new(&ref_path).exists();
+        let ref_data = std::fs::read(&ref_path);
 
         let Some(document) = document else {
-            if has_ref {
+            if ref_data.is_ok() {
                 log!(self, "missing document");
-                log!(self, "  ref       | {ref_path}");
+                log!(self, "  ref       | {}", ref_path.display());
             }
             return;
         };
 
-        let skippable = match document.pages.as_slice() {
+        let skippable = match D::is_skippable(document) {
-            [] => {
+            Ok(skippable) => skippable,
+            Err(()) => {
                 log!(self, "document has zero pages");
                 return;
             }
-            [page] => skippable(page),
-            _ => false,
         };
 
-        // Tests without visible output and no reference image don't need to be
+        // Tests without visible output and no reference output don't need to be
         // compared.
-        if skippable && !has_ref {
+        if skippable && ref_data.is_err() {
             std::fs::remove_file(&live_path).ok();
             return;
         }
 
-        // Render the live version.
+        // Render and save live version.
-        let pixmap = render(document, 1.0);
+        let live = document.make_live();
-
+        document.save_live(&self.test.name, &live);
-        // Save live version, possibly rerendering if different scale is
-        // requested.
-        let mut pixmap_live = &pixmap;
-        let slot;
-        let scale = crate::ARGS.scale;
-        if scale != 1.0 {
-            slot = render(document, scale);
-            pixmap_live = &slot;
-        }
-        let data = pixmap_live.encode_png().unwrap();
-        std::fs::write(&live_path, data).unwrap();
-
-        // Write PDF if requested.
-        if crate::ARGS.pdf() {
-            let pdf_path = format!("{}/pdf/{}.pdf", crate::STORE_PATH, self.test.name);
-            let pdf = typst_pdf::pdf(document, &PdfOptions::default()).unwrap();
-            std::fs::write(pdf_path, pdf).unwrap();
-        }
-
-        // Write SVG if requested.
-        if crate::ARGS.svg() {
-            let svg_path = format!("{}/svg/{}.svg", crate::STORE_PATH, self.test.name);
-            let svg = typst_svg::svg_merged(document, Abs::pt(5.0));
-            std::fs::write(svg_path, svg).unwrap();
-        }
-
-        // Compare against reference image if available.
-        let equal = has_ref && {
-            let ref_data = std::fs::read(&ref_path).unwrap();
-            let ref_pixmap = sk::Pixmap::decode_png(&ref_data).unwrap();
-            approx_equal(&pixmap, &ref_pixmap)
-        };
 
+        // Compare against reference output if available.
         // Test that is ok doesn't need to be updated.
-        if equal {
+        if ref_data.as_ref().map_or(false, |r| D::matches(&live, r)) {
             return;
         }
 
@@ -201,36 +170,37 @@ impl<'a> Runner<'a> {
                 std::fs::remove_file(&ref_path).unwrap();
                 log!(
                     into: self.result.infos,
-                    "removed reference image ({ref_path})"
+                    "removed reference output ({})", ref_path.display()
                 );
             } else {
-                let opts = oxipng::Options::max_compression();
+                let ref_data = D::make_ref(live);
-                let data = pixmap.encode_png().unwrap();
+                if !self.test.attrs.contains(&Attr::Large)
-                let ref_data = oxipng::optimize_from_memory(&data, &opts).unwrap();
+                    && ref_data.len() > crate::REF_LIMIT
-                if !self.test.large && ref_data.len() > crate::REF_LIMIT {
+                {
-                    log!(self, "reference image would exceed maximum size");
+                    log!(self, "reference output would exceed maximum size");
                     log!(self, "  maximum   | {}", FileSize(crate::REF_LIMIT));
                     log!(self, "  size      | {}", FileSize(ref_data.len()));
                     log!(self, "please try to minimize the size of the test (smaller pages, less text, etc.)");
-                    log!(self, "if you think the test cannot be reasonably minimized, mark it as `// LARGE`");
+                    log!(self, "if you think the test cannot be reasonably minimized, mark it as `large`");
                     return;
                 }
                 std::fs::write(&ref_path, &ref_data).unwrap();
                 log!(
                     into: self.result.infos,
-                    "updated reference image ({ref_path}, {})",
+                    "updated reference output ({}, {})",
+                    ref_path.display(),
                     FileSize(ref_data.len()),
                 );
             }
         } else {
-            self.result.mismatched_image = true;
+            self.result.mismatched_output = true;
-            if has_ref {
+            if ref_data.is_ok() {
-                log!(self, "mismatched rendering");
+                log!(self, "mismatched output");
-                log!(self, "  live      | {live_path}");
+                log!(self, "  live      | {}", live_path.display());
-                log!(self, "  ref       | {ref_path}");
+                log!(self, "  ref       | {}", ref_path.display());
             } else {
-                log!(self, "missing reference image");
+                log!(self, "missing reference output");
-                log!(self, "  live      | {live_path}");
+                log!(self, "  live      | {}", live_path.display());
             }
         }
     }
@@ -242,6 +212,10 @@ impl<'a> Runner<'a> {
         if diag.span.id().is_some_and(|id| id != self.test.source.id()) {
             return;
         }
+        // TODO: remove this once HTML export is stable
+        if diag.message == "html export is under active development and incomplete" {
+            return;
+        }
 
         let message = diag.message.replace("\\", "/");
         let range = self.world.range(diag.span);
@@ -349,6 +323,153 @@ impl<'a> Runner<'a> {
     }
 }
 
+/// An output type we can test.
+trait OutputType: Document {
+    /// The type that represents live output.
+    type Live;
+
+    /// The path at which the live output is stored.
+    fn live_path(name: &str) -> PathBuf;
+
+    /// The path at which the reference output is stored.
+    fn ref_path(name: &str) -> PathBuf;
+
+    /// Whether the test output is trivial and needs no reference output.
+    fn is_skippable(&self) -> Result<bool, ()> {
+        Ok(false)
+    }
+
+    /// Produces the live output.
+    fn make_live(&self) -> Self::Live;
+
+    /// Saves the live output.
+    fn save_live(&self, name: &str, live: &Self::Live);
+
+    /// Produces the reference output from the live output.
+    fn make_ref(live: Self::Live) -> Vec<u8>;
+
+    /// Checks whether the live and reference output match.
+    fn matches(live: &Self::Live, ref_data: &[u8]) -> bool;
+
+    /// Runs additional checks.
+    #[expect(unused_variables)]
+    fn check_custom(runner: &mut Runner, doc: Option<&Self>) {}
+}
+
+impl OutputType for PagedDocument {
+    type Live = tiny_skia::Pixmap;
+
+    fn live_path(name: &str) -> PathBuf {
+        format!("{}/render/{}.png", crate::STORE_PATH, name).into()
+    }
+
+    fn ref_path(name: &str) -> PathBuf {
+        format!("{}/{}.png", crate::REF_PATH, name).into()
+    }
+
+    fn is_skippable(&self) -> Result<bool, ()> {
+        /// Whether rendering of a frame can be skipped.
+        fn skippable_frame(frame: &Frame) -> bool {
+            frame.items().all(|(_, item)| match item {
+                FrameItem::Group(group) => skippable_frame(&group.frame),
+                FrameItem::Tag(_) => true,
+                _ => false,
+            })
+        }
+
+        match self.pages.as_slice() {
+            [] => Err(()),
+            [page] => Ok(page.frame.width().approx_eq(Abs::pt(120.0))
+                && page.frame.height().approx_eq(Abs::pt(20.0))
+                && page.fill.is_auto()
+                && skippable_frame(&page.frame)),
+            _ => Ok(false),
+        }
+    }
+
+    fn make_live(&self) -> Self::Live {
+        render(self, 1.0)
+    }
+
+    fn save_live(&self, name: &str, live: &Self::Live) {
+        // Save live version, possibly rerendering if different scale is
+        // requested.
+        let mut pixmap_live = live;
+        let slot;
+        let scale = crate::ARGS.scale;
+        if scale != 1.0 {
+            slot = render(self, scale);
+            pixmap_live = &slot;
+        }
+        let data: Vec<u8> = pixmap_live.encode_png().unwrap();
+        std::fs::write(Self::live_path(name), data).unwrap();
+
+        // Write PDF if requested.
+        if crate::ARGS.pdf() {
+            let pdf_path = format!("{}/pdf/{}.pdf", crate::STORE_PATH, name);
+            let pdf = typst_pdf::pdf(self, &PdfOptions::default()).unwrap();
+            std::fs::write(pdf_path, pdf).unwrap();
+        }
+
+        // Write SVG if requested.
+        if crate::ARGS.svg() {
+            let svg_path = format!("{}/svg/{}.svg", crate::STORE_PATH, name);
+            let svg = typst_svg::svg_merged(self, Abs::pt(5.0));
+            std::fs::write(svg_path, svg).unwrap();
+        }
+    }
+
+    fn make_ref(live: Self::Live) -> Vec<u8> {
+        let opts = oxipng::Options::max_compression();
+        let data = live.encode_png().unwrap();
+        oxipng::optimize_from_memory(&data, &opts).unwrap()
+    }
+
+    fn matches(live: &Self::Live, ref_data: &[u8]) -> bool {
+        let ref_pixmap = sk::Pixmap::decode_png(ref_data).unwrap();
+        approx_equal(live, &ref_pixmap)
+    }
+
+    fn check_custom(runner: &mut Runner, doc: Option<&Self>) {
+        let errors = crate::custom::check(runner.test, &runner.world, doc);
+        if !errors.is_empty() {
+            log!(runner, "custom check failed");
+            for line in errors.lines() {
+                log!(runner, "  {line}");
+            }
+        }
+    }
+}
+
+impl OutputType for HtmlDocument {
+    type Live = String;
+
+    fn live_path(name: &str) -> PathBuf {
+        format!("{}/html/{}.html", crate::STORE_PATH, name).into()
+    }
+
+    fn ref_path(name: &str) -> PathBuf {
+        format!("{}/html/{}.html", crate::REF_PATH, name).into()
+    }
+
+    fn make_live(&self) -> Self::Live {
+        // TODO: Do this earlier to be able to process export errors.
+        typst_html::html(self).unwrap()
+    }
+
+    fn save_live(&self, name: &str, live: &Self::Live) {
+        std::fs::write(Self::live_path(name), live).unwrap();
+    }
+
+    fn make_ref(live: Self::Live) -> Vec<u8> {
+        live.into_bytes()
+    }
+
+    fn matches(live: &Self::Live, ref_data: &[u8]) -> bool {
+        live.as_bytes() == ref_data
+    }
+}
+
 /// Draw all frames into one image with padding in between.
 fn render(document: &PagedDocument, pixel_per_pt: f32) -> sk::Pixmap {
     for page in &document.pages {
@@ -397,23 +518,6 @@ fn render_links(canvas: &mut sk::Pixmap, ts: sk::Transform, frame: &Frame) {
     }
 }
 
-/// Whether rendering of a frame can be skipped.
-fn skippable(page: &Page) -> bool {
-    page.frame.width().approx_eq(Abs::pt(120.0))
-        && page.frame.height().approx_eq(Abs::pt(20.0))
-        && page.fill.is_auto()
-        && skippable_frame(&page.frame)
-}
-
-/// Whether rendering of a frame can be skipped.
-fn skippable_frame(frame: &Frame) -> bool {
-    frame.items().all(|(_, item)| match item {
-        FrameItem::Group(group) => skippable_frame(&group.frame),
-        FrameItem::Tag(_) => true,
-        _ => false,
-    })
-}
-
 /// Whether two pixel images are approximately equal.
 fn approx_equal(a: &sk::Pixmap, b: &sk::Pixmap) -> bool {
     a.width() == b.width()

tests/src/tests.rs

@@ -37,13 +37,13 @@ const STORE_PATH: &str = "tests/store";
 /// The directory where syntax trees are stored.
 const SYNTAX_PATH: &str = "tests/store/syntax";
 
-/// The directory where the reference images are stored.
+/// The directory where the reference output is stored.
 const REF_PATH: &str = "tests/ref";
 
 /// The file where the skipped tests are stored.
 const SKIP_PATH: &str = "tests/skip.txt";
 
-/// The maximum size of reference images that aren't marked as `// LARGE`.
+/// The maximum size of reference output that isn't marked as `large`.
 const REF_LIMIT: usize = 20 * 1024;
 
 fn main() {
@@ -62,7 +62,7 @@ fn setup() {
     std::env::set_current_dir("..").unwrap();
 
     // Create the storage.
-    for ext in ["render", "pdf", "svg"] {
+    for ext in ["render", "html", "pdf", "svg"] {
         std::fs::create_dir_all(Path::new(STORE_PATH).join(ext)).unwrap();
     }
 
@@ -156,10 +156,10 @@ fn clean() {
 
 fn undangle() {
     match crate::collect::collect() {
-        Ok(_) => eprintln!("no danging reference images"),
+        Ok(_) => eprintln!("no danging reference output"),
         Err(errors) => {
             for error in errors {
-                if error.message == "dangling reference image" {
+                if error.message == "dangling reference output" {
                     std::fs::remove_file(&error.pos.path).unwrap();
                     eprintln!("✅ deleted {}", error.pos.path.display());
                 }
@@ -188,7 +188,7 @@ fn run_parser_test(
     let mut result = TestResult {
         errors: String::new(),
         infos: String::new(),
-        mismatched_image: false,
+        mismatched_output: false,
     };
 
     let syntax_file = live_path.join(format!("{}.syntax", test.name));

tests/src/world.rs

@@ -19,7 +19,7 @@ use typst::syntax::{FileId, Source, Span};
 use typst::text::{Font, FontBook, TextElem, TextSize};
 use typst::utils::{singleton, LazyHash};
 use typst::visualize::Color;
-use typst::{Library, World};
+use typst::{Feature, Library, World};
 
 /// A world that provides access to the tests environment.
 #[derive(Clone)]
@@ -180,7 +180,9 @@ fn library() -> Library {
     // Set page width to 120pt with 10pt margins, so that the inner page is
     // exactly 100pt wide. Page height is unbounded and font size is 10pt so
     // that it multiplies to nice round numbers.
-    let mut lib = Library::default();
+    let mut lib = Library::builder()
+        .with_features([Feature::Html].into_iter().collect())
+        .build();
 
     // Hook up helpers into the global scope.
     lib.global.scope_mut().define_func::<test>();

tests/suite/layout/inline/justify.typ

@@ -7,8 +7,7 @@ forms a "block" with flush edges at both sides.
 
 First line indents and hyphenation play nicely with justified text.
 
---- justify-knuth-story ---
+--- justify-knuth-story large ---
-// LARGE
 #set page(width: auto, height: auto)
 #set par(leading: 4pt, justify: true)
 #set text(font: "New Computer Modern")

tests/suite/model/link.typ

@@ -1,6 +1,6 @@
 // Test hyperlinking.
 
---- link-basic ---
+--- link-basic render html ---
 // Link syntax.
 https://example.com/
 

tests/suite/text/coma.typ

@@ -1,5 +1,4 @@
---- coma ---
+--- coma large ---
-// LARGE
 #set page(width: 450pt, margin: 1cm)
 
 *Technische Universität Berlin* #h(1fr) *WiSe 2019/2020* \

tools/test-helper/README.md

@@ -6,7 +6,7 @@ Code Lens buttons will appear above every test's name:
 
 - View: Opens the output and reference image of a test to the side.
 - Run: Runs the test and shows the results to the side.
-- Save: Runs the test with `--update` to save the reference image.
+- Save: Runs the test with `--update` to save the reference output.
 - Terminal: Runs the test in the integrated terminal.
 
 In the side panel opened by the Code Lens buttons, there are a few menu buttons
@@ -14,7 +14,7 @@ at the top right:
 
 - Refresh: Reloads the panel to reflect changes to the images.
 - Run: Runs the test and shows the results.
-- Save: Runs the test with `--update` to save the reference image.
+- Save: Runs the test with `--update` to save the reference output.
 
 ## Installation
 In order for VS Code to run the extension with its built-in

tools/test-helper/package.json

@@ -28,7 +28,7 @@
       },
       {
         "command": "typst-test-helper.saveFromPreview",
-        "title": "Run and save reference image",
+        "title": "Run and save reference output",
         "category": "Typst Test Helper",
         "icon": "$(save)",
         "enablement": "typst-test-helper.runButtonEnabled"
@@ -102,4 +102,3 @@
     "vscode": "^1.88.0"
   }
 }
-

tools/test-helper/src/extension.ts

@@ -121,7 +121,7 @@ class TestHelper {
     const lenses = [];
     for (let nr = 0; nr < document.lineCount; nr++) {
       const line = document.lineAt(nr);
-      const re = /^--- ([\d\w-]+) ---$/;
+      const re = /^--- ([\d\w-]+)( [\d\w-]+)* ---$/;
       const m = line.text.match(re);
       if (!m) {
         continue;
@@ -143,7 +143,7 @@ class TestHelper {
         }),
         new vscode.CodeLens(line.range, {
           title: "Save",
-          tooltip: "Run and view the test and save the reference image",
+          tooltip: "Run and view the test and save the reference output",
           command: "typst-test-helper.saveFromLens",
           arguments: [name],
         }),