Context.dev: Contextdev Post Web Extract

Crawl a website and extract structured data matching a JSON schema and your instructions.

Tool ID: contextdev_post_web_extract

Run this action

Use the TypeScript SDK for one call. Use ctx.tools.execute(...) inside a Play for durable calls, scheduled calls, or CSV processing.

import { Deepline } from 'deepline';

const deepline = await Deepline.connect();
const result = await deepline.tools.execute(
  'contextdev_post_web_extract',
  {
    "url": "https://example.com/resource",
    "schema": {}
  },
);

console.log(result.toolResponse.raw);

CLI

deepline tools execute contextdev_post_web_extract --input '{
  "url": "https://example.com/resource",
  "schema": {}
}' --json

Example response

The SDK returns this shape at result.toolResponse.raw. The example values illustrate the response.

{
  "data": {
    "status": "example",
    "url": "https://example.com",
    "urls_analyzed": [
      "https://example.com"
    ],
    "data": {},
    "metadata": {
      "numUrls": 123,
      "maxCrawlDepth": 123,
      "numSucceeded": 123,
      "numFailed": 123,
      "numSkipped": 123,
      "numBlocked": 123
    }
  }
}

Use deepline describe contextdev_post_web_extract --json for the latest machine-readable contract.

Input reference

Crawl a website, use the provided JSON Schema and instructions to prioritize relevant internal links, and extract structured data from the selected pages.

Name	Type	Required	Default	Details
`payload.url`	`string`	Yes	—	The starting website URL to crawl and extract from. Must include http:// or https://. Format: `uri`.
`payload.schema`	`object`	Yes	—	JSON Schema for the returned data object. TypeScript Zod users can pass a JSON Schema generated from a Zod object. Python users can pass the equivalent JSON Schema object.
`payload.instructions`	`string`	No	—	Optional extraction guidance, such as which facts to prioritize or how to interpret fields in the schema. Maximum length: 2000.
`payload.factCheck`	`boolean`	No	`false`	When true, every returned value must be grounded in facts stated on the page. fields that cannot be supported by the page are returned as null/empty. When false (default), the model may make reasonable inferences and derivations from the page content (e.g. ideal customer, competitor analysis, recommendations) while keeping verifiable specifics (names, quotes, URLs, dates, metrics) faithful to the source.
`payload.followSubdomains`	`boolean`	No	`false`	When true, follow links on subdomains of the starting URL’s domain.
`payload.maxPages`	`integer`	No	`5`	Maximum number of pages to analyze for extraction. Hard cap: 50. Defaults to 5. Minimum: 1. Maximum: 50.
`payload.maxDepth`	`integer`	No	—	Optional maximum link depth from the starting URL (0 = only the starting page). If omitted, there is no crawl depth limit. Minimum: 0.
`payload.pdf`	`object`	No	`{"shouldParse":true}`	—
`payload.includeFrames`	`boolean`	No	`false`	When true, iframe contents are included in Markdown before extraction.
`payload.maxAgeMs`	`integer`	No	`604800000`	Return cached scrape results if a prior scrape for the same parameters is younger than this many milliseconds. Defaults to 7 days (604800000 ms). Minimum: 0. Maximum: 2592000000.
`payload.waitForMs`	`integer`	No	—	Optional browser wait time in milliseconds after initial page load for each crawled page. Minimum: 0. Maximum: 30000.
`payload.settleAnimations`	`boolean`	No	`false`	When true, waits briefly for CSS and transition animations to settle before extracting each crawled page. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
`payload.stopAfterMs`	`integer`	No	`80000`	Soft time budget for the crawl in milliseconds. Min: 10000 (10s). Max: 110000 (110s). Default: 80000 (80s). Minimum: 10000. Maximum: 110000.
`payload.timeoutMS`	`integer`	No	—	Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes). Minimum: 1000. Maximum: 300000.
`payload.tags`	`array`	No	—	Optional tags for tracking usage. Up to 20 tags, each 1 to 50 characters. Maximum items: 20.

Output reference

Name	Type	Required	Default	Details
`result.data`	`object`	Yes	—	—
`result.data.status`	`string`	Yes	—	—
`result.data.url`	`string`	Yes	—	—
`result.data.urls_analyzed`	`array`	Yes	—	—
`result.data.data`	`record`	Yes	—	—
`result.data.metadata`	`object`	Yes	—	—
`result.data.metadata.numUrls`	`integer`	Yes	—	—
`result.data.metadata.maxCrawlDepth`	`integer`	Yes	—	—
`result.data.metadata.numSucceeded`	`integer`	Yes	—	—
`result.data.metadata.numFailed`	`integer`	Yes	—	—
`result.data.metadata.numSkipped`	`integer`	Yes	—	—
`result.data.metadata.numBlocked`	`integer`	Yes	—	—
`result.data.key_metadata`	`object`	No	—	—
`result.meta`	`object`	No	—	—

Deepline cost

Pricing model: provider_usage (provider usage).
Estimated Deepline credits: 0.01 per pricing unit.

​Run this action

​CLI

​Example response

​Input reference

​Output reference

​Deepline cost

​Related documentation

Run this action

CLI

Example response

Input reference

Output reference

Deepline cost

Related documentation