Emit generated output artifacts

To emit a byte artifact — robots.txt, a sitemap variant, a social-image .png, a sidecar .json index — that is not a routed page, not in navigation, and not an xref target, implement IArtifactContentService. The interface has three members and one rule: the same resolver produces the bytes for a live dev request and for the static build, so the two surfaces can never drift.

• Claims declares the URL territory the service owns (an exact path, a prefix, or a path suffix). Claims derive from options alone — they are consulted on every request and must never trigger expensive work.
• ResolveAsync turns one claimed path into bytes plus a content type, or returns null to decline so the request falls through to content routing.
• DiscoverAsync enumerates the routes the static build writes — each one resolved through ResolveAsync and written to its output file.

Pennington's own search shards (/search/**.json), llms.txt files, and book PDFs ship through this interface; RobotsTxtContentService below is the smallest possible example.

For the opposite case — a service that contributes routed pages, TOC entries, and xrefs from a non-markdown source — see Source content from outside the markdown pipeline.

Before you begin

• A working Pennington site on bare AddPennington (see Create your first Pennington site if not).
• Familiarity with the four-stage pipeline at a conceptual level (The content pipeline and union types).

Write the service

namespace ExtensibilityLabExample;
  
using System.Collections.Immutable;
using System.Text;
using Pennington.Artifacts;
using Pennington.Pipeline;
using Pennington.Routing;
  
/// <summary>
/// Demonstrates the artifact tier — <see cref="IArtifactContentService"/> for byte outputs
/// (robots, search-index sidecars, social-image generators) that are not routed pages, not in
/// navigation, and not xref targets. <see cref="Claims"/> declares the URL territory,
/// <see cref="ResolveAsync"/> produces the bytes (served live in dev by the artifact router),
/// and <see cref="DiscoverAsync"/> enumerates the routes the static build writes — one byte
/// path for both surfaces.
/// <para>
/// Backs how-to <c>/how-to/extensibility/emit-generated-artifacts</c>.
/// </para>
/// </summary>
public sealed class RobotsTxtContentService : IArtifactContentService
{
    private const string Body = """
        User-agent: *
        Allow: /
        Sitemap: /sitemap.xml
        """;
  
    /// <summary>The one URL this service owns.</summary>
    public ImmutableList<ArtifactClaim> Claims { get; } =
        [new ArtifactClaim("robots", new ExactClaim(new UrlPath("/robots.txt")), "robots.txt")];
  
    /// <summary>
    /// Produces the robots.txt bytes — for a live dev request and for the static build alike.
    /// Returning null declines the request so it falls through to content routing.
    /// </summary>
    public Task<ArtifactContent?> ResolveAsync(string relativePath, CancellationToken cancellationToken)
        => Task.FromResult(relativePath.Equals("robots.txt", StringComparison.OrdinalIgnoreCase)
            ? new ArtifactContent(Encoding.UTF8.GetBytes(Body), "text/plain; charset=utf-8")
            : null);
  
    /// <summary>Enumerates the single robots.txt route for the static build.</summary>
    public async IAsyncEnumerable<DiscoveredItem> DiscoverAsync()
    {
        await Task.CompletedTask;
        yield return new DiscoveredItem(
            new ContentRoute
            {
                CanonicalPath = new UrlPath("/robots.txt"),
                OutputFile = new FilePath("robots.txt"),
            },
            new GeneratedSource("text/plain"));
    }
}

The pieces:

• ArtifactClaim carries an owner name, a shape, and a description. The shape is a union: ExactClaim (one path), PrefixClaim (everything under a prefix, optionally narrowed by extension — /search/ + .json), or SuffixClaim (a path ending at any depth — this is how {section}/llms.txt works, a territory no endpoint route template can express). diag routes lists every registered claim.
• ResolveAsync receives the request path without its leading slash (robots.txt, search/en/index.json). Returning null declines: the request continues into content routing, so a real page under a claimed prefix keeps working.
• DiscoverAsync yields DiscoveredItems with a GeneratedSource. Routes that should exist only in dev are resolvable without being enumerated — the book package serves its live /book-preview/ this way while enumerating only the PDFs.
• The resolver may do expensive work on demand (build an index, fold over ISiteProjection, run a headless browser). The claims must not.

Register the service

Register on the artifact tier — never as IContentService, which would put the service in every request-path discovery walk:

builder.Services.AddTransient<IArtifactContentService, RobotsTxtContentService>();

Result

The dev server answers /robots.txt live, and the static build writes the same bytes to the output root:

User-agent: *
Allow: /
Sitemap: /sitemap.xml

Verify

• Fetch /robots.txt from the dev server and expect the body above — same bytes both surfaces.
• Run dotnet run --project examples/ExtensibilityLabExample -- build output and confirm output/robots.txt exists with the expected body.
• Run dotnet run -- diag routes and confirm the claim appears under "Artifact territories".

Related

• Reference: Content pipeline interfaces
• How-to: Source content from outside the file system
• Background: The content pipeline and union types