Context.dev: Contextdev Get Web Scrape Sitemap

List every page URL on a website from its sitemap, optionally filtered by a pattern.

Tool ID: contextdev_get_web_scrape_sitemap

Run this action

Use the TypeScript SDK for one call. Use ctx.tools.execute(...) inside a Play for durable calls, scheduled calls, or CSV processing.

import { Deepline } from 'deepline';

const deepline = await Deepline.connect();
const result = await deepline.tools.execute(
  'contextdev_get_web_scrape_sitemap',
  {
    "domain": "example.com"
  },
);

console.log(result.toolResponse.raw);

CLI

deepline tools execute contextdev_get_web_scrape_sitemap --input '{
  "domain": "example.com"
}' --json

Example response

The SDK returns this shape at result.toolResponse.raw. The example values illustrate the response.

{
  "data": {
    "success": true,
    "domain": "example.com",
    "urls": [
      "https://example.com"
    ],
    "meta": {
      "sitemapsDiscovered": 123,
      "sitemapsFetched": 123,
      "sitemapsSkipped": 123,
      "errors": 123
    }
  }
}

Use deepline describe contextdev_get_web_scrape_sitemap --json for the latest machine-readable contract.

Input reference

Crawl an entire website’s sitemap and return all discovered page URLs.

Name	Type	Required	Default	Details
`payload.domain`	`string`	Yes	—	Domain to build a sitemap for Minimum length: 3.
`payload.maxLinks`	`integer`	No	`10000`	Maximum number of links to return from the sitemap crawl. Defaults to 10,000. Minimum is 1, maximum is 100,000. Minimum: 1. Maximum: 100000.
`payload.sitemapUrl`	`string`	No	—	Optional explicit sitemap URL. When provided, exactly this sitemap is crawled instead of discovering the domain’s sitemaps. Format: `uri`.
`payload.urlRegex`	`string`	No	—	Optional RE2-compatible regex pattern. Only URLs matching this pattern are returned and counted against maxLinks. Maximum length: 256.
`payload.search`	`string`	No	—	Optional search phrase. Minimum length: 2. Maximum length: 200.
`payload.headers`	`record`	No	—	Optional outbound HTTP headers forwarded only to the target URL, sent as deep-object query params such as headers[X-Custom]=value. When provided, caching is bypassed: the result is neither read from nor written to cache.
`payload.timeoutMS`	`integer`	No	—	Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes). Minimum: 1. Maximum: 300000.
`payload.zdr`	`"enabled" \| "disabled"`	No	`"disabled"`	Set to enabled to bypass shared caches and omit request and response content from retained usage logs. Requires zero data retention to be enabled for your organization (contact support@context.dev), otherwise the request fails with ZDR_NOT_ENABLED. Successful ZDR responses include X-Context-ZDR: true. Allowed: `enabled`, `disabled`.
`payload.tags`	`array`	No	—	Optional comma-separated caller-defined tags for tracking this request. Tags are recorded on the request’s usage log and can be used to filter usage on the dashboard usage page. Up to 20 tags, each 1-50 characters. Maximum items: 20.

Output reference

Name	Type	Required	Default	Details
`result.data`	`object`	Yes	—	—
`result.data.success`	`true`	Yes	—	Allowed: `true`.
`result.data.domain`	`string`	Yes	—	—
`result.data.urls`	`array`	Yes	—	—
`result.data.meta`	`object`	Yes	—	—
`result.data.meta.sitemapsDiscovered`	`integer`	Yes	—	—
`result.data.meta.sitemapsFetched`	`integer`	Yes	—	—
`result.data.meta.sitemapsSkipped`	`integer`	Yes	—	—
`result.data.meta.errors`	`integer`	Yes	—	—
`result.data.key_metadata`	`object`	No	—	—
`result.meta`	`object`	No	—	—

Deepline cost

Pricing model: provider_usage (provider usage).
Estimated Deepline credits: 0.01 per pricing unit.

​Run this action

​CLI

​Example response

​Input reference

​Output reference

​Deepline cost

​Related documentation

Run this action

CLI

Example response

Input reference

Output reference

Deepline cost

Related documentation