Meziantou.Framework.WebPlatform 1.0.10

Meziantou.Framework.WebPlatform

A spec-compliant, high-performance implementation of the web platform for .NET. It parses HTML, XML, SVG, MathML and CSS exactly as a browser does, and exposes the browser DOM API (the WebIDL surface of the WHATWG and W3C specifications) with .NET naming: queries (QuerySelector, XPath, live collections), mutations, events, the CSSOM and the computed styles, forms, custom elements, accessibility, and more. A managed layout engine computes the geometry exposed by the web APIs (CSSOM View, the observers...), and the documents are painted into a display list that a rendering backend (a managed rasterizer, in a separate package) rasterizes for the screenshots and the canvases. Documents can be loaded in a browsing context that fetches their resources over the network and runs their scripts with a pluggable JavaScript engine (Meziantou.Framework.JavaScriptEngine is provided).

Every public member mapping a WebIDL member is annotated with [IdlMember(name, specUrl)]. The WebIDL names follow the .NET naming guidelines: acronyms of 3 letters or more are PascalCased and acronyms of 2 letters stay upper case (HTMLElement.innerHTML → HtmlElement.InnerHtml, toJSON → ToJson, CDATASection → CDataSection, UIEvent and HTMLBRElement → HtmlBRElement). MathML and XPath keep their casing.

Examples

Parse HTML, query the nodes and the styles

HtmlDocumentParser is in the Meziantou.Framework.WebPlatform namespace, and the DOM types (BrowsingContext, Document, HtmlAnchorElement...) are in Meziantou.Framework.WebPlatform.Dom.

var document = HtmlDocumentParser.ParseDocument("""
    <style>.important { color: red; font-weight: bold }</style>
    <ul>
      <li><a href="https://example.com/a" class="important">A</a>
      <li><a href="/b" style="color: blue">B</a>
    </ul>
    """);

// CSS selectors, with typed overloads and LINQ
var links = document.QuerySelectorAll<HtmlAnchorElement>("li > a");
var external = links.Where(a => a.Href.StartsWith("https:", StringComparison.Ordinal)).Select(a => a.TextContent);
var first = document.GetElementsByTagName("li")[0];
Assert.Equal(["A"], external);
Assert.Same(first, links[0].ParentElement);

// XPath
var result = document.Evaluate("//a[@class='important']", document, type: XPathResult.FirstOrderedNodeType);
var important = (Element?)result.SingleNodeValue;
Assert.Equal("A", important?.TextContent);

// Inline style (style attribute)
var color = links[1].Style.GetPropertyValue("color");
Assert.Equal("blue", color);

The computed styles (cascade of the style sheets, inheritance, media queries) need a browsing context, which provides the window and the viewport:

var html = """
    <style>.important { color: red; font-weight: bold }</style>
    <p class="important">Hello</p>
    """;

using var context = new BrowsingContext(new BrowsingContextOptions
{
    Viewport = new ViewportOptions { Width = 1280, Height = 720 }, // optional, 800x600 by default
});

var document = context.Navigate("https://example.com/", _ => html);
var style = context.Window.GetComputedStyle(document.QuerySelector(".important")!);
Assert.Equal("rgb(255, 0, 0)", style.Color);
Assert.Equal("700", style.GetPropertyValue("font-weight"));

See docs/dom-helpers.md, docs/selectors.md, docs/computed-style.md and docs/css-typed-om.md.

Update the DOM

The formatters are in the Meziantou.Framework.WebPlatform.Formatting namespace.

var document = HtmlDocumentParser.ParseDocument("""
    <ul>
      <li><a href="/a">A</a>
      <li><a href="/b">B</a>
    </ul>
    """);
var list = document.QuerySelector("ul")!;

// Create elements
var item = document.CreateElement<HtmlLIElement>();
var link = document.CreateElement<HtmlAnchorElement>();
link.Href = "/c";
link.TextContent = "C";
link.ClassList.Add("important");
link.SetAttribute("data-id", "3");
item.Append(link);
list.Append(item);

// Parse markup into an element, edit the styles, remove nodes
list.InsertAdjacentHtml("beforeend", "<li><a href=/d>D</a>");
link.Style.SetProperty("text-decoration", "underline");
document.QuerySelector("a[href='/b']")?.Remove();

// Serialize the document
var html = document.GetHtml();
var indented = document.ToHtml(new HtmlPrettyFormatter());
var minified = document.ToHtml(new HtmlMinifyFormatter());
Assert.StartsWith("<html><head></head><body><ul>", html, StringComparison.Ordinal);
Assert.Equal("""
    <html>
      <head></head>
      <body>
        <ul>
          <li>
            <a href="/a">A</a>
          </li>
          <li></li>
          <li>
            <a href="/c" class="important" data-id="3" style="text-decoration: underline;">C</a>
          </li>
          <li>
            <a href="/d">D</a>
          </li>
        </ul>
      </body>
    </html>
    """, indented);
Assert.Equal("""<ul><li><a href=/a>A</a><li><li><a href=/c class=important data-id=3 style="text-decoration: underline;">C</a><li><a href=/d>D</a></ul>""", minified);

Element.SetHtml() and Document.ParseHtml() insert untrusted markup with the HTML Sanitizer API. See docs/html-formatting.md and docs/css-formatting.md for the serialization options.

Allow network requests

Nothing is fetched unless a DocumentLoader is configured. With a loader, the browsing context loads the documents of the navigations and the iframe elements, the style sheets (<link rel=stylesheet>, @import), the images and, when a script engine is configured, the scripts:

using var context = new BrowsingContext(new BrowsingContextOptions
{
    DocumentLoader = DocumentLoader.Create(),  // HTTP GET requests with a shared HttpClient (or pass your own)
    CookieJar = new CookieJar(),               // optional: the cookies (RFC 6265bis, document.cookie, cookieStore)
});

context.Navigate("https://example.com/");
await context.EventLoop.RunUntilIdleAsync(); // runs the tasks until the document and its resources are loaded

var document = context.Document;

DocumentLoader.Create() requests any URL. A custom loader can filter the requests (to prevent server-side request forgery with untrusted documents), serve them from a cache or from the disk:

internal sealed class SameHostLoader(string host) : DocumentLoader
{
    private static readonly DocumentLoader Http = DocumentLoader.Create();

    public override ValueTask<DocumentResource?> LoadAsync(DocumentRequest request, CancellationToken cancellationToken)
    {
        if (new Uri(request.Url).Host != host)
            return ValueTask.FromResult<DocumentResource?>(null); // network error

        return Http.LoadAsync(request, cancellationToken);
    }
}

See docs/browsing-context.md.

Execute scripts

Scripts only run when a JavaScript engine is plugged into the browsing context. The Meziantou.Framework.WebPlatform.JavaScript package provides one, based on Meziantou.Framework.JavaScriptEngine, in the Meziantou.Framework.WebPlatform.JavaScript namespace:

using var engine = new JavaScriptScriptEngine(new JavaScriptScriptEngineOptions
{
    Timeout = TimeSpan.FromSeconds(5), // per script or callback
});

using var context = new BrowsingContext(new BrowsingContextOptions
{
    ScriptEngine = engine,
    DocumentLoader = DocumentLoader.Create(), // optional: external scripts, modules, fetch, XMLHttpRequest, WebSocket, EventSource...
    UnhandledScriptError = (_, error) => Console.Error.WriteLine(error),
    ConsoleMessage = (_, message) => Console.WriteLine(message.Text), // console.log(), console.count()...
});

var document = await context.NavigateAsync("https://example.com/", _ => """
    <title>Example</title>
    <ul id=list></ul>
    <script>
      setTimeout(() => document.getElementById('list').innerHTML = '<li>a<li>b', 100);
    </script>
    """);

// Run the tasks and the timers until there is nothing left
do { await context.EventLoop.RunUntilIdleAsync(); } while (await context.EventLoop.WaitForNextTimerAsync());

Assert.Equal("<li>a</li><li>b</li>", document.GetElementById("list")!.InnerHtml);

// Evaluate code in the realm of a window
var title = engine.Evaluate(context.Window, "document.title");
Assert.Equal("Example", title.AsString());

The scripts run in the process: read the threat model in docs/scripting.md before running untrusted documents.

Take screenshots

The documents are painted by the library into a backend-agnostic display list; the Meziantou.Framework.WebPlatform.Imaging package draws them, and decodes and encodes the images, with Meziantou.Framework.Imaging, a fully managed library (no native dependency: the same pixels on every platform). ImagingRenderingBackend and the screenshot methods are in the Meziantou.Framework.WebPlatform.Imaging namespace:

using var context = new BrowsingContext(new BrowsingContextOptions
{
    Viewport = new ViewportOptions { Width = 1280, Height = 720, DevicePixelRatio = 2 },
    DocumentLoader = DocumentLoader.Create(),
    RenderingBackend = new ImagingRenderingBackend(),
});

context.Navigate("https://example.com/");
await context.EventLoop.RunUntilIdleAsync();

await File.WriteAllBytesAsync("page.png", context.TakeScreenshotAsPng());            // the viewport: 2560x1440 pixels
using var title = context.Document.QuerySelector("h1")!.TakeScreenshot();           // an element, as an Image<Rgba32>

// Print the document: the print media type, the @page rules, one image per page
var pages = context.RenderPagesAsPng(new PrintOptions { PageWidth = 794, PageHeight = 1123 }); // A4 at 96 dpi

See docs/rendering.md for what is painted, and Printing for the pages.

Limitations

The limitations of the library are listed in the documentation.

Specifications

Building

./eng/webplatform/fetch-tests.sh
dotnet test --project tests/Meziantou.Framework.WebPlatform.Tests
dotnet test --project tests/Meziantou.Framework.WebPlatform.Imaging.Tests
dotnet test --project tests/Meziantou.Framework.WebPlatform.Wpt

The web-platform-tests (testharness.js tests and reftests, the reftests being rendered by the Imaging backend) run with the script engine of Meziantou.Framework.WebPlatform.JavaScript: see docs/web-platform-tests.md.

See docs/updating-specs.md to update the specification data, docs/fuzzing.md to fuzz the parsers, and docs/performance.md for the benchmarks.