Skip to content

Menu

Build with web data.

Practical guides for structured extraction, agent tools and repeatable workflows. Start with the job you need to finish.

36 guides

Extraction foundations

Extract website data with Python

Extract product titles and prices with a Python client, validate JSON, retain currency and handle rate limits, truncation and retries.

Extraction foundations

Extract website data with Node.js

Use native Node.js fetch to extract quote records, validate JSON and handle cancellation, incomplete results and retries with a tested client.

Extraction foundations

Understand a Scraipe request receipt

Read compile and network charges, recognise idempotent replays, and diagnose missing or inconsistent receipt fields with a tested offline checker.

MCP clients

Connect Claude Code to Scraipe

Connect Scraipe MCP to Claude Code, verify the selected configuration scope, diagnose approval conflicts with a local lab and inspect one bounded Read call.

MCP clients

Connect Cursor to Scraipe

Connect Cursor to Scraipe, check workspace and global MCP entries without printing secrets, and inspect a bounded extraction with its receipt.

MCP clients

Connect Codex to Scraipe

Configure Scraipe MCP in Codex with a bearer environment variable, a read-only tool allowlist and a bounded first-call checklist.

MCP clients

Connect VS Code to Scraipe

Configure Scraipe MCP in VS Code, align the file format with your agent harness and verify a bounded Read workflow.

MCP clients

Connect Claude Desktop to Scraipe

Configure a pinned local MCP bridge for Claude Desktop, keep the bearer key out of process arguments and reproduce the transport with a local fixture.

Structured data

Normalize prices without losing currency

Preserve raw prices and currency while producing exact decimal strings with explicit format policies, runnable Node.js code and ambiguity checks.

Structured data

Validate dates from web pages

Validate calendar dates and explicit-offset timestamps without timezone guesses, with a strict Node.js example, real failure cases and fifteen tests.

Reader workflows

Convert a website to Markdown

Convert a web page to Markdown, export it safely, and test preserved content and parsed tables, code, headings and links with reproducible examples.

Reader workflows

Detect Markdown fallback to plain text

Distinguish requested text, Markdown conversion fallback and partial output with a tested classifier and an explicitly reproduced local fallback fixture.

Reusable extraction

Test an extractor on held-out pages

Test an extractor on held-out HTML pages, compare candidates case by case, and spot records or layouts that regress even when the total pass count rises.

Reusable extraction

Choose representative sample pages

Choose diverse extraction samples, audit risk coverage and split leakage, and keep compile checks separate from independent evaluation.

Discovery and crawl

Map a site before crawling

Map a site before crawling, review URL scope, compare saved inventories and check modification timestamps with tested offline planning tools.

Discovery and crawl

Limit crawl depth and page count

Set crawl depth and page limits, understand sitemap seeds and failed-page budgets, and check every returned URL against the saved include/exclude rules.

Discovery and crawl

Include and exclude URL patterns

Test crawl include and exclude rules against labelled URLs. Catch seed mismatches, query variants and overbroad globs before submitting a job.

Discovery and crawl

Deduplicate crawl results

Deduplicate Markdown from saved crawl jobs while preserving source URLs, changed revisions and receipts. Reproduce the workflow with two local Node.js fixtures.

Discovery and crawl

Handle partial crawl failures

Review page errors in a completed Scraipe crawl, reconcile missing URLs against an inventory and stage successful pages with a tested offline Node.js example.

Cost control

Model repair costs in a recurring job

Forecast repair credits for a recurring extraction job. Use single-page receipts and a tested Node calculator to compare average and high-repair weeks.

Cost control

Budget residential network requests

Budget residential HTTP and browser fetches with current credit rates, a tested offline calculator and explicit page-size and routing scenarios.

Monitoring

Build a price monitor with webhooks

Build a one-product price monitor with Scraipe Watches, signed webhooks, exact USD comparisons, durable receipt checks and missed-event reconciliation.