Skip to main content

Source tasks

Source tasks generate data from settings. They require no input rows, workspace, collection, or datastore destination. Fetch the generated objects through the API after the task completes, fails, or stops.

1. Authenticate and discover​

Save a Personal API Key for task operations. Catalog discovery needs no authentication:

bash
curl 'https://api.datablist.com/v1/sources'


curl 'https://api.datablist.com/v1/sources/instagram-source'

Confirm that instagram-source appears in the catalog before using it. If it is absent, choose an available source and adapt your settings to its schema. Read settingsSchema, outputSchema, and limits from the definition.

2. Create a task​

This request uses a sample profile. Replace example_profile with a profile you want to query. The example can consume Datablist credits; lookups can consume credits even when no objects are found.

bash
curl --request POST 'https://api.datablist.com/v1/sources/tasks' \
  --header 'X-API-Key: YOUR_PERSONAL_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "sourceId": "instagram-source",
    "uniqueId": "instagram-followers-20261009",
    "settings": {
      "profiles": "example_profile",
      "scrapingType": "followers",
      "limitPerProfile": 100
    }
  }'

The API returns 202 Accepted. Save @id and links.self. Omit maxCredits for no task-specific limit, or supply a nonnegative decimal string such as "10". The best-effort budget rules apply to the whole task, including all work it performs. A requested result limit is not a guarantee of an exact result count.

3. Check task status​

bash
curl 'https://api.datablist.com/v1/sources/tasks/TASK_ID' \
  --header 'X-API-Key: YOUR_PERSONAL_API_KEY'

Wait for completed, failed, or stopped. Example response while the task is running:

json
{
  "@id": "00000000000040008000000000000001",
  "@type": "SourceTask",
  "kind": "source",
  "sourceId": "instagram-source",
  "uniqueId": "instagram-followers-20261009",
  "status": "running",
  "createdAt": "2026-10-09T10:00:00Z",
  "startedAt": "2026-10-09T10:00:01Z",
  "finishedAt": null,
  "progress": {"results": 20},
  "credits": {"used": "1", "max": null},
  "error": null,
  "links": {
    "self": "https://api.datablist.com/v1/sources/tasks/00000000000040008000000000000001",
    "results": "https://api.datablist.com/v1/sources/tasks/00000000000040008000000000000001/results"
  }
}

progress.results counts generated objects. Zero results is a valid successful outcome. Sources share task states and errors with enrichments, and have no per-input outcomes.

4. Page through results​

bash
curl 'https://api.datablist.com/v1/sources/tasks/TASK_ID/results?limit=50' \
  --header 'X-API-Key: YOUR_PERSONAL_API_KEY'

Each result is a JSON object, as shown in this sample page:

json
{
  "taskId": "00000000000040008000000000000001",
  "results": [
    {
      "instagramUsername": "example_user_1",
      "instagramProfileUrl": "https://www.instagram.com/example_user_1/"
    }
  ],
  "nextCursor": null
}

Sources do not wrap each result in inputId or a per-input status. While the task is active, this endpoint returns 409 ResultsNotReady. Failed and stopped tasks keep results already saved and report credits used.

Once a task finishes, its result order stays the same. That order does not show when each object was found.

If nextCursor is non-null, pass it unchanged and URL-encode it:

bash
curl --get 'https://api.datablist.com/v1/sources/tasks/TASK_ID/results' \
  --header 'X-API-Key: YOUR_PERSONAL_API_KEY' \
  --data-urlencode 'limit=50' \
  --data-urlencode 'cursor=NEXT_CURSOR'

Pages default to 50 objects, accept a limit from 1 to 100, and contain at most 128 KiB. Each execution can return up to 10,000 objects and 5 MiB, with a 16-KiB limit per object. Task data is retained for 90 days after completion, failure, or a stop.

History, cancellation, and retries​

bash
curl 'https://api.datablist.com/v1/sources/tasks?sourceId=instagram-source&limit=50' \
  --header 'X-API-Key: YOUR_PERSONAL_API_KEY'


curl --request POST 'https://api.datablist.com/v1/sources/tasks/TASK_ID/stop' \
  --header 'X-API-Key: YOUR_PERSONAL_API_KEY'

You can also filter history by status. Requesting stop returns 202 and may leave the task in stop_asked while already-started or non-stoppable work finishes. Keep polling, then collect the partial results. See how long a stop can take.

Use the same safe creation retry procedure as for enrichments: inspect history before retrying an ambiguous POST. Sources have no inputs endpoint, and requesting a task through the other family's routes returns 404.

Last updated on