Extraction foundations
Choose Read or Extract for a web-data task
Choose Read or Extract, bound your request plan and validate content, truncation and the final source origin before ingestion.
Practical guides for structured extraction, agent tools and repeatable workflows. Start with the job you need to finish.
36 guides
Extraction foundations
Choose Read or Extract, bound your request plan and validate content, truncation and the final source origin before ingestion.
Extraction foundations
Extract product titles and prices with a Python client, validate JSON, retain currency and handle rate limits, truncation and retries.
Extraction foundations
Use native Node.js fetch to extract quote records, validate JSON and handle cancellation, incomplete results and retries with a tested client.
Extraction foundations
Distinguish missing, null, blank and wrongly typed fields with a tested JSON validator that preserves valid zero and false values.
Extraction foundations
Read compile and network charges, recognise idempotent replays, and diagnose missing or inconsistent receipt fields with a tested offline checker.
MCP clients
Connect Scraipe MCP to Claude Code, verify the selected configuration scope, diagnose approval conflicts with a local lab and inspect one bounded Read call.
MCP clients
Connect Cursor to Scraipe, check workspace and global MCP entries without printing secrets, and inspect a bounded extraction with its receipt.
MCP clients
Configure Scraipe MCP in Codex with a bearer environment variable, a read-only tool allowlist and a bounded first-call checklist.
MCP clients
Configure Scraipe MCP in VS Code, align the file format with your agent harness and verify a bounded Read workflow.
MCP clients
Configure a pinned local MCP bridge for Claude Desktop, keep the bearer key out of process arguments and reproduce the transport with a local fixture.
Structured data
Design a product JSON Schema, compare contract changes against a regression corpus, and reject contradictory price fields with tested conditional rules.
Structured data
Preserve parent and child associations, distinguish empty lists from selector drift, and test a checked local HTML extractor before importing nested data.
Structured data
Preserve raw prices and currency while producing exact decimal strings with explicit format policies, runnable Node.js code and ambiguity checks.
Structured data
Validate calendar dates and explicit-offset timestamps without timezone guesses, with a strict Node.js example, real failure cases and fifteen tests.
Structured data
Preserve missing, blank, null, zero and false values in partial updates with an explicit field policy, a runnable local planner and sixteen tests.
Reader workflows
Convert a web page to Markdown, export it safely, and test preserved content and parsed tables, code, headings and links with reproducible examples.
Reader workflows
Preserve requested and returned URLs for a Read capture, then replay an exact quote against its source using tested Node.js examples.
Reader workflows
Measure a complete document payload with a pinned tokenizer, reserve model context and expose omitted sources with a tested local packer.
Reader workflows
Distinguish requested text, Markdown conversion fallback and partial output with a tested classifier and an explicitly reproduced local fallback fixture.
Reader workflows
Compare main-content and whole-page reader output with reproducible fixtures that catch missing sections, unwanted noise and content neither mode can restore.
Reusable extraction
Learn when Scraipe reuses an extractor or a cached response, why forceBrowser can return a placeholder, and how to test a fresh render without forcing compilation.
Reusable extraction
Test an extractor on held-out HTML pages, compare candidates case by case, and spot records or layouts that regress even when the total pass count rises.
Reusable extraction
Quarantine suspect captures, distinguish source failures from layout drift, and evaluate a versioned repair with a tested local diagnostic.
Reusable extraction
Choose diverse extraction samples, audit risk coverage and split leakage, and keep compile checks separate from independent evaluation.
Reusable extraction
Measure extraction cost across pages and layouts with a tested ledger that includes pre-warming, network charges, rejected results and unknown receipts.
Discovery and crawl
Map a site before crawling, review URL scope, compare saved inventories and check modification timestamps with tested offline planning tools.
Discovery and crawl
Set crawl depth and page limits, understand sitemap seeds and failed-page budgets, and check every returned URL against the saved include/exclude rules.
Discovery and crawl
Test crawl include and exclude rules against labelled URLs. Catch seed mismatches, query variants and overbroad globs before submitting a job.
Discovery and crawl
Deduplicate Markdown from saved crawl jobs while preserving source URLs, changed revisions and receipts. Reproduce the workflow with two local Node.js fixtures.
Discovery and crawl
Review page errors in a completed Scraipe crawl, reconcile missing URLs against an inventory and stage successful pages with a tested offline Node.js example.
Cost control
Estimate extraction credits before a batch runs. Use sample quotes, layout counts and paid fetch assumptions to size a workload against a credit budget.
Cost control
Forecast repair credits for a recurring extraction job. Use single-page receipts and a tested Node calculator to compare average and high-repair weeks.
Cost control
Budget residential HTTP and browser fetches with current credit rates, a tested offline calculator and explicit page-size and routing scenarios.
Cost control
Reconcile Scraipe request records with daily credit usage using matching UTC windows, complete pagination and a tested account-export checker.
Cost control
Choose an observation-age limit, schedule cache-bypassed refreshes and hold stale results with a tested Node.js freshness policy checker.
Monitoring
Build a one-product price monitor with Scraipe Watches, signed webhooks, exact USD comparisons, durable receipt checks and missed-event reconciliation.