Extract nested lists without losing parent records

Keep the product as the unit of extraction

To extract nested lists without losing their parent records, select one parent container first, then collect its children inside that container. Keep a stable parent key on every child row you later flatten. A product with no variants must still remain in the parent collection.

This guide demonstrates that workflow with invented product HTML and a runnable Node.js extractor. It preserves two products, including one with no tags or variants, and keeps a missing variant price attached to the correct variant. The code runs locally with Cheerio; it does not fetch a website, call Scraipe or execute page scripts. It was tested on 18 October 2026 with Node 22.23.2 and Cheerio 1.2.0.

Choose the grain of each record

A catalog can contain several related collections. Decide what one record represents before choosing selectors or designing a schema.

An array index is an order, not a stable identity. Reordering two cards must not change which product owns a variant. The example requires source-provided product IDs and variant SKUs, rejects duplicate parent IDs and rejects duplicate SKUs within one parent. The same SKU text may appear under a different parent because the example uses the pair as its key. Adapt this identity rule to your actual source; it is not a universal SKU convention.

Why separate global lists can misalign values

The fixture's notebook has two variants. “Plain” has no displayed price; “Ruled” has AUD 14.50. Collecting every variant name and every price into two separate arrays produces:

{
  "names": ["Plain", "Ruled"],
  "prices": ["AUD 14.50"]
}

Zipping these arrays by index assigns the Ruled price to Plain. Padding the shorter list afterward cannot recover the missing association. The downloadable test suite reproduces this mistake and checks the parent-scoped result instead:

[
  { "sku": "PLAIN", "name": "Plain", "priceText": null },
  { "sku": "RULED", "name": "Ruled", "priceText": "AUD 14.50" }
]

Extract the name and price while still inside each variant element. An absent price becomes an explicit null on that variant. It never shifts a later variant's price and never becomes zero. The raw currency text is preserved.

Run the complete local example

Download these five files into one directory:

• package.json
• package-lock.json
• products.fixture.html
• extract-nested-products.mjs
• extract-nested-products.test.mjs

Use Node 22 or later:

npm ci --ignore-scripts --no-fund --no-audit
npm test
node extract-nested-products.mjs products.fixture.html

The install downloads the pinned parser and its dependencies. The tests and extractor then operate entirely on local HTML and spend no API credits. Expect twelve passing tests and a JSON result whose summary is:

{ "parents": 2, "variants": 2 }

The output contains both products, with the child arrays still nested, and variantRows, with an explicit productId on every child. The parent with ID P002 remains in products with tags: [] and variants: []. It contributes no child row. Keep the parent collection when importing the child table; reconstructing products from variant rows alone would erase P002.

The CLI prints the complete result only after extraction succeeds. It exits with code 1 on a missing required value, an ambiguous structure or an unreadable file. It does not print a partial result. Shell redirection can still create or truncate an output file on failure, so check the exit code before importing it.

Scope selectors at every level

The extractor begins with direct product children of the catalog container:

const roots = $('#catalog').children('article.product');

For each root, it selects that product's direct tag and variant lists. For each variant, it reads the name and optional price from that variant alone. The example expects one direct container at each declared level. This deliberate layout contract keeps nested recommendation cards and unrelated promotional prices out of the result.

Cheerio's children selects direct children; find selects matching descendants. Neither method is universally correct. If the real page inserts a wrapper around the title or variants, update the selector contract and tests rather than broadening every query to all descendants. A broad selector that produces plausible data can still mix neighboring products.

The module removes script, style and noscript descendants when reading text and collapses whitespace in labels. It reads a saved HTML document; it does not render JavaScript, click pagination controls, resolve network redirects or establish permission to collect a site. Start from a permitted capture and keep its source URL, capture time and retrieval context outside the product fields.

Distinguish empty from unobserved

The invented folder has an explicit, empty variant-list container. That supports the local fixture's empty list. Removing the container is different: the example throws instead of pretending it observed zero variants. A changed selector or incomplete capture can look like “no children” unless you check the structure.

These choices belong to this captured-layout example. In a rendered application, an empty container might be waiting for JavaScript or a network response. The local parser cannot establish that loading finished. Confirm the capture state before using an empty list as business evidence.

The tests cover the association error, parent retention, recommendation noise, reordering, missing containers, duplicate identifiers, multiple prices and actual CLI behavior. The function rejects HTML above 1 MiB before parsing. The CLI reads the file into memory first, so this is a small-fixture example, not a streaming resource-control solution.

Apply the pattern to Scraipe carefully

For repeated scalar values such as tag labels, the API reference supports a [] suffix in field requests, for example fields: ["title", "tags[]"]. The equivalent schema uses an array of strings for tags. A list with one tag should remain an array; no tags can be an empty array. Test that the chosen repeating root corresponds to one product and compare the values with the source.

A nested array of objects is a different capability. The current compiled selector interpreter collects extracted field values from matching elements; it does not recursively build a separate object schema inside every matched child. A local source probe against this fixture preserved parent-scoped tag arrays and an empty child list, but returned variant text strings that failed a nested-object contract. The local Cheerio example in this guide constructs those objects explicitly; it is not a Scraipe API parameter or evidence of a hosted nested-object extraction.

Do not assume that supplying items: { "type": "object" } guarantees a reusable compiled extractor for every layout. Inspect the returned shape and receipt. If a workflow uses a model extraction path, validate the associations separately and account for that path's charges. Another application design is to collect a parent inventory and child rows carrying source-provided parent IDs, then join them while preserving parents with no children. Neither approach should join independent name and price arrays by position.

Validate relationships before importing

JSON Schema can describe nested arrays and child record fields; the product schema guide shows a reproducible full-result validator. Add the nested properties and required fields your consumer needs. Cross-record rules such as unique parent IDs and parent/child references still require application checks; a schema pass alone does not establish correct ownership.

Before import, compare parent counts, child counts and representative associations with the capture. Reject a failed or truncated API response before treating any child list as complete. API continuation and following a website's next-page link are separate operations; a complete response for one page does not prove catalog coverage.

Use the playground for a small permitted live request, then inspect its receipt. Local tests cost no Scraipe credits, while extraction compilation, repair and network operations may. Missing children should trigger investigation of capture completeness and selectors, not an uncontrolled retry loop or invented product data.