# Read-or-Extract example pack v1

An offline request planner for Node 22+. No API key, npm install, network requests,
account changes or credit spend. The example.com paths are invented placeholders.
This pack tests planning, not extraction, accessibility of a website or service cost.

## Reproduce from a fresh directory

Keep these four files together: README.md, requirements.json, plan-requests.mjs,
and plan-requests.test.mjs. Then run:

```sh
node --test plan-requests.test.mjs
node plan-requests.mjs requirements.json > plan.json
```

Expect 12 passing tests and this summary in plan.json:

```json
{"inputUrls":3,"uniqueUrls":2,"duplicateUrls":1,"requests":4}
```

The output has two POST /v1/read recipes (Markdown, maxChars 10000) and two
POST /v1/scrape recipes (title, price, currency; maxTokens 2000). All select
direct egress. Nothing dispatches them. The planner fixes these output limits
as teaching defaults; adapt and test them for your consumer.

Set pageContent to false for fields only, or fields to [] for page content only.
Both empty is an error. Lower maxRequests to 3 and rerun: the command exits 1,
writes a diagnostic to stderr and emits no plan to stdout. Shell redirection
still creates/truncates plan.json on failure, so check the exit code before
consuming it. Input is limited to 1000 URLs, 100 unique URLs, 200 requests and
20 simple field names. These are this sample's limits, not Scraipe quotas.

## URL identity and limitations

Node's URL parser normalises host case and default ports; fragments are removed.
Paths, query values, query order and trailing slashes remain significant.
This does not discover canonical URLs, follow redirects, remove tracking
parameters or deduplicate page content. All input URLs must be valid, even if
the unique-URL limit would already be exceeded.

Only absolute HTTP(S) URLs without embedded credentials are accepted. This is
syntax validation, not a public-network or permission check: private hosts,
DNS rebinding, signed query strings and page access are not assessed. Do not
commit plans containing private URL parameters. Before dispatch, enforce your
trusted URL policy and use only sources you are permitted to access.

## Connect deliberately

The output is a local recipe, not a Scraipe batch endpoint. A separate server-side
runner must supply the trusted API origin, scoped credentials, timeouts, bounded
concurrency and response validation. Use read scope for Read, scrape scope for
Extract. The planner supplies no idempotency key and performs no retries.

Do not equate four planned API calls with four origin fetches, four credits,
complete coverage or four successes. Compilation, fallback and source behaviour
affect execution. Direct egress opts out of residential fallback; compilation
can still cost credits. Inspect the receipt for each executed request.

For Read, check ok, returned format, stats.truncated and expected sections.
Markdown may fall back to text. For Extract, check ok, truncated, required
fields, types and domain-specific meaning. Preserve source URL and receipt
alongside each accepted result; neither a successful call nor a passing shape
check proves correctness. A truncated output needs an explicit continuation
decision. This pack does not crawl or follow a website's next-page links.

Contract: https://scraipe.co/docs/api
Guide: https://scraipe.co/learn/choose-read-or-extract-for-a-web-data-task
