Skip to main content
Sources

Sitemap Source

post/v1/sources/tasks

Authorizations

Authorizationstringheaderrequired

Use an account access token as Bearer <token>.

Get an access token ↗

Read an XML sitemap or follow the sitemaps listed in an index. Retrieve page URLs and last-modified timestamps as flat objects without a collection or input rows.

Public ID: sitemap-source. Get the current definition. You can read it without authentication. Confirm availability in the source catalog.

Settings​

Set sitemapUrl to the full sitemap URL. You can set filterRegex to keep matching page URLs; matching ignores case.

sitemapSourceLimit defaults to 95,000 and cannot exceed that value. It applies across the whole parsing task. When several sitemaps are processed at once, the result count can exceed your requested limit.

For very large sitemap indexes, submit individual sitemap URLs to stay within processing limits.

Missing modification dates return null. Dates without a time use midnight UTC; timestamps without a time zone are treated as UTC.

sitemapUrlstringrequired

Full sitemap URL, such as https://www.datablist.com/sitemap.xml. Sitemap indexes are followed automatically.

format: "uri"
filterRegexstring

Keep only page URLs matching this regular expression.

sitemapSourceLimitnumber

Requested maximum number of page URLs across the task. Defaults to 95,000; concurrent work can exceed a smaller requested limit.

Additional properties are supported.

Outputs​

Each result is a JSON object with the fields below. Sources do not wrap results in inputId, status, or outputs. Fields depend on your settings and the data found; extra fields can appear before the catalog lists them.

sitemapPageUrlstringrequired
format: "uri"
sitemapLastUpdatedstring
format: "date-time"

Additional properties are supported.

Credits and limits​

Cost: 1 credit per 150 URLs parsed.

This source requires a balance of at least 40 credits before execution. Parsing costs 1 credit per 150 returned URLs, proportionally rather than rounding to a complete block. Downloads requiring the protected-site fallback can add 1 credit. Work producing no URLs can still incur a download charge.

Tasks can run for up to 24 hours; completed, failed, or stopped tasks are kept for 90 days. Enrichment and source tasks share your account balance and task limits. Set maxCredits to stop starting new work after a credit threshold; work already underway can still exceed it. See limits and errors.

Accepted task and results​

The API returns 202 Accepted with a task ID. Save it, then check links.self every few seconds. Once status is completed, failed, or stopped, read links.results. Failed and stopped tasks can still have useful results. See the task reference.

Example results (sample data and credit amounts):

json
{
  "taskId": "00000000000040008000000000000001",
  "results": [
    {
      "sitemapPageUrl": "https://example.com/about",
      "sitemapLastUpdated": "2026-10-13T00:00:00+00:00"
    }
  ],
  "nextCursor": null
}

If nextCursor is not null, pass it unchanged in the next request to get more results. Pages default to 50 results; limit accepts 1–100. A page is at most 128 KiB, so it may contain fewer results than requested.