Validate missing fields in extracted JSON

Validate presence separately from the value

A successful extraction can still be unusable for your application. A product may have a title but no price, an explicit null, an empty string or a value with the wrong type. Those cases need different decisions. Treating them all as “missing” makes it harder to diagnose the source and can discard legitimate values such as a zero price or inStock: false.

This guide supplies a small Node.js validator for a specific product-record contract. It audits a captured Scraipe response without fetching a website, changing a record or making another API request. The result tells you which rows passed the contract and whether the whole response can proceed to the next import check.

Decide what the consumer actually requires

The example's application requires four fields: title, price, currency and inStock. Its optional description can be absent, a string or null. These are chosen application rules, not universal requirements for every website.

The title must contain a non-whitespace character, and price must be a finite, nonnegative number. Currency must be one of AUD, USD, GBP or EUR. That four-value allowlist belongs to this example application; it is not a complete list of currencies. Unexpected record fields are reported for review instead of silently removed.

Adapt these choices to your actual consumer before using the validator. A price label such as "£12.50" may be a perfectly useful extraction result, but it fails this numeric-price contract. Keep the raw value, then use a separately tested currency-aware conversion if your application needs a number. Do not infer “free” from an unavailable price.

Run the local example

Download these four files into the same directory:

• Validation module and CLI
• Runnable tests
• Invented response fixture
• Product-record JSON Schema

Use Node 22 or later; the example was checked with Node 22.23.2 on 5 October 2026. The validator and tests use Node's built-in APIs and require no npm installation.

node --test validate-extracted-json.test.mjs
node validate-extracted-json.mjs missing-fields.fixture.json

The first command passes 17 tests. The second intentionally exits with code 1: the fixture contains four rows and three invalid fields. Its report has readyToImport: false, rowsChecked: 4, validRowIndexes: [0], and these issues:

[
  { "rowIndex": 1, "field": "price", "code": "missing" },
  { "rowIndex": 2, "field": "title", "code": "null_not_allowed" },
  { "rowIndex": 3, "field": "price", "code": "wrong_type" }
]

Indexes start at zero, so index 0 refers to the first row. The valid row deliberately has a zero price and false availability. All records and request identifiers in this fixture are invented test inputs. The example is evidence of local validation behaviour, not a live extraction result or an accuracy benchmark.

Avoid a truthiness check

A check such as if (!row.price) combines absent values, nulls, empty strings and the valid number zero. Use an ownership check for presence, followed by a field-specific value check:

if (!Object.hasOwn(row, 'price')) {
  // Missing property: investigate the extraction or consumer requirement.
} else if (row.price === null) {
  // Present, but explicitly null: apply the field's nullability rule.
} else if (typeof row.price !== 'number' || !Number.isFinite(row.price)) {
  // Present, but not a finite number: do not silently convert it.
} else if (row.price < 0) {
  // Valid number type, but outside this application's allowed range.
}

The downloadable module applies that separation to every required field. It also checks that each row is an object, so an array, string or null row does not accidentally pass. Its diagnostics contain the row index, field name and issue code rather than copying full record values into logs.

No normalization happens during the audit. Whitespace checks do not trim the stored title, missing fields do not receive defaults, and numeric or boolean strings remain strings. Tests verify that auditing the fixture leaves the original input unchanged.

Express the contract in JSON Schema

In JSON Schema, listing a field in properties does not require it to exist. Put mandatory names in required. A present null is a different value from an absent property, and a string type alone permits an empty string. The downloaded schema adds explicit presence, type and content constraints for this application.

Its title rule combines minLength: 1 with the pattern \S to require a non-whitespace character. Its optional description uses both string and null types. additionalProperties: false rejects new fields until the consumer's contract is intentionally changed.

The schema documents record rules. The JavaScript module is a small implementation of those specific rules, not a general JSON Schema engine. During review, the schema compiled with a JSON Schema 2020-12 validator and agreed with the module on ten representative records, including the downloadable fixture. That comparison does not claim complete standards conformance.

A schema's default annotation does not, by itself, fill absent values during validation. If another tool offers default insertion or coercion, treat those as separate transformations with their own audit trail. Supplying a fabricated price or title can hide the problem this guide is trying to detect.

Check the response before checking individual rows

This example expects a Scraipe envelope containing ok: true, truncated: false and an array under data. If any of those conditions is missing, it refuses to approve individual rows. An unsuccessful request or incomplete response is a collection problem first, not a reason to import whichever fields happen to be present.

The API contract describes the response cursor used when an output budget truncates an extraction. Resolve that continuation before treating the result as complete. A complete response still covers only the collection scope you requested; it does not prove that every page or product on a website was visited.

An empty array receives empty_result_requires_decision by default. That does not mean empty output is always a defect: a known source can legitimately contain no records. After checking that your use case allows an empty result, enable the explicit option:

node validate-extracted-json.mjs response.json --allow-empty

This option changes only the empty-list decision. It does not waive field rules or approve a failed or truncated response.

Connect the audit to an import job

The CLI prints a JSON report and uses three exit codes: 0 when its contract checks pass, 1 when a report contains validation issues, and 2 when the command cannot read valid JSON or its arguments are invalid. It reads the captured file into memory, so use this example for small response captures; a large ingestion pipeline needs its own streaming and resource controls.

The module also exports auditExtraction(payload, { allowEmpty }) for an existing Node application. Gate the import on readyToImport, then apply your remaining source, freshness and business checks. validRowIndexes is useful for diagnosis, but it does not authorise silently importing a partial batch when readyToImport is false.

Preserve the original response, source URL, retrieval context and server request ID in appropriately restricted storage. Keep rejected rows separate from accepted data and record the reason. A report with many missing prices may indicate a changed page layout, an inappropriate field request or a legitimate source limitation; inspect representative rows against the page before selecting a remedy.

Repair the cause without inventing data

For a required field, confirm whether the value exists on the source page and whether the requested field means what the consumer expects. If it is absent at the source, change the consumer's requirement or reject the record. If it is present but was not captured, inspect extraction provenance and the API's field/schema options. Passing a consumer-side schema is not proof of factual correctness or a promise that every schema keyword is enforced by an extraction service.

These local checks make no network requests and spend no API credits. Re-extracting or recompiling is a separate operation: inspect its receipt and current pricing rather than repeatedly retrying unchanged input. Do not turn a validation failure into an uncontrolled request loop.

Use the JSON tool or playground to inspect a small permitted source, then compare the captured result with your consumer contract. For choosing the source representation first, see Read or Extract.