Detect Markdown fallback to plain text

Compare the requested and returned formats

To detect Markdown fallback, retain the format you requested and inspect the response's format and stats.fallback. A request for Markdown can succeed while returning plain text. Route the returned representation deliberately; do not give it a .md extension simply because the request asked for Markdown.

Scraipe's current reader labels failed or over-budget Markdown conversion as format: "text" and includes a nonempty stats.fallback explanation. A request that explicitly asks for text normally returns text without that fallback field. The API reference documents the conversion budget and the separate overall read refusal; these are different outcomes.

Do not detect fallback by searching the content for headings, backticks or links. Ordinary prose can be valid Markdown, and plain text can contain those same symbols literally. Format metadata and the original request context answer the representation question. They still do not prove that the selected content is complete, correct or the intended page.

Separate representation, completeness and acceptance

The downloadable classifier uses this explicit policy:

“Accept” means this consumer policy permits the representation. It does not mean a file has been written or the source has been verified. The policy is intentionally stricter than simply accepting every successful response. A legacy or modified capture missing fallback metadata may be usable after investigation, but this example does not silently assume why it changed format.

stats.truncated is independent of fallback. The reader applies its character limit after choosing a representation, so a text fallback can also be partial. Explicitly allowing text does not waive the requirement for stats.truncated: false. Missing completeness evidence is refused as well.

Reproduce the decision locally

Download these five files into a new directory:

• Representation classifier
• Fourteen tests
• Local fallback capture and strict policy
• Bounded capture loader
• Loader's existing acceptance-helper dependency

Use Node 22 or later. No package installation, API key or network request is needed:

node --test classify-read-format.test.mjs
node classify-read-format.mjs read-format.fixture.json

The tests should report fourteen passes. The second command intentionally exits with code 1 because the fixture requests Markdown and sets acceptTextFallback: false. Its report is:

{
  "accepted": false,
  "representation": "text-fallback",
  "extension": null,
  "hasFallbackReason": true,
  "issues": ["text_fallback_not_accepted"]
}

This is an expected policy refusal, not a broken installation. The classifier does not export content or modify the capture. Exit 0 means the policy accepted it; exit 1 means a classification was made but refused; exit 2 means usage, input parsing, configuration or local-file validation failed.

The input file combines requestedFormat, the explicit boolean acceptTextFallback, and a captured response object. Both policy fields are required. Store the requested format with the original job; do not infer it from the response after the fact. This example covers Markdown and text requests. HTML needs a separate workflow.

Allow text only when the downstream task can use it

If your downstream task needs words but not Markdown link destinations, code fences or table structure, text may be sufficient. Make that a deliberate policy choice. Create a separate policy file without changing the original fixture:

node --input-type=module -e '
import fs from "node:fs";
const input = JSON.parse(fs.readFileSync("read-format.fixture.json", "utf8"));
input.acceptTextFallback = true;
fs.writeFileSync("allow-text.json", JSON.stringify(input, null, 2), { flag: "wx" });
'
node classify-read-format.mjs allow-text.json

The creation command refuses an existing allow-text.json. The classifier now exits 0 with accepted: true, representation: "text-fallback" and extension: ".txt". The returned content is unchanged. The extension is a routing hint for your storage code, not a file the classifier creates.

If the application requires Markdown structure, retain the strict refusal and inspect the source and conversion reason. A smaller intended source region or a different representation may solve the task. Blindly repeating the same request does not guarantee a different result, and wrapping the text in a code fence does not restore lost structure.

What the fixture actually demonstrates

The fixture was produced from the invented HTML used in the conversion guide by the repository reader on 27 October 2026. A fresh local subprocess used a deliberately tiny one-millisecond Markdown conversion budget to exercise the fallback branch. The ok and url fields wrap that local output for a consumer test; no hosted request or receipt was fabricated.

With the normal local configuration, this source produced Markdown. An explicit text request produced text without fallback metadata. Under the artificial tiny budget, the reader produced text with a fallback reason; its 507 UTF-16 code units matched the explicit-text output. Applying a forty-unit content limit also set truncation to true. These are observations of this fixture and induced test condition, not production latency measurements or a claim that this page normally times out.

The public classifier tests cover both strict and permissive policies, explicit text requests, Markdown without punctuation, text with Markdown-looking punctuation, missing or contradictory metadata, truncation, invalid Unicode, unsupported formats and CLI behavior. They test consumer decisions against captures. They do not run the hosted conversion service.

Keep diagnostic text out of control flow

The classifier treats a nonempty fallback string as available diagnostic evidence; it does not match the exact wording to decide whether a timeout or conversion exception occurred. Human-readable reasons can change. Preserve the original capture securely for investigation and use stable representation fields for routing.

Blank, null, boolean and object fallback values are rejected as malformed metadata. A fallback field on an explicitly requested text response is also treated as inconsistent in this example. If your upstream contract intentionally changes, review and version the policy rather than quietly accepting a new shape.

Reports contain classification, booleans, issue codes and an optional extension. They omit content, URLs, receipts and the raw fallback explanation. Input files are bounded to 1 MiB of valid UTF-8 JSON. Empty content and unpaired Unicode surrogates are refused before any extension is selected.

Apply the decision before export or ingestion

For a real captured response, first confirm that the request succeeded and that the source matches your intended origin and page. The classifier checks representation and completeness, not source authorization or content truth. Pair it with source preservation and the conversion acceptance pack when required sections, links or code matter.

The strict Markdown exporter in that conversion guide refuses text by design. Keep a separate text storage path when text fallback is approved; do not change the response's format field just to pass an exporter. Preserve the original requested format, returned format and actual receipt so later investigations can reconstruct the decision.

A hosted Read call requires the read scope. The current API contract has no direct-network conversion credit charge and separately meters residential networking; inspect the actual request receipt for a hosted call's cost. This local classifier adds no API request or credit charge. Once the representation is accepted, budget the exact content sent to an agent rather than assuming a format decision also guarantees model-context fit.