Monitor and cancel bulk jobs

Every bulk mutation returns a job that runs in the background. Use the bulkJob and bulkJobs queries to follow a job, and the cancelBulkJob mutation to stop it.

Reading jobs requires a User Account Token or a Service Account Token with the DATA_READ, BULK_DATA_WRITE, or ADMIN permission. Cancelling a job requires BULK_DATA_WRITE or ADMIN. See Authentication and permissions for details.

Job statuses

A job has one of these statuses.

Status	Meaning
QUEUED	The job is waiting to start.
RUNNING	The job is processing records. progress and results grow over time.
SUCCEEDED	The job processed every selected record.
FAILED	An error stopped the job. The error field describes it.
CANCELLED	The job was stopped on request.

SUCCEEDED, FAILED, and CANCELLED are final statuses. For jobs in a final status, results shows how far the job got.

One active job per environment

Each environment can have only one QUEUED or RUNNING bulk job at a time. The limit applies across all operations, tiers, keys, and submitters, and it includes dry runs.

If you submit a bulk mutation while another job is active, the request fails with BAD_USER_INPUT and the reason JOB_LIMIT. The error includes the activeJobId of the active job when it can still be found. Wait for that job to finish, or cancel it, and then submit again.

{
  "errors": [
    {
      "extensions": {
        "code": "BAD_USER_INPUT",
        "reason": "JOB_LIMIT",
        "activeJobId": "<ID of the active job>"
      }
    }
  ]
}

Poll a job

Use the id returned by the bulk mutation to query the job. The query returns null when no job with that ID exists in the environment.

query GetBulkJob($id: ID!) {
  bulkJob(id: $id) {
    id
    operation
    tier
    dryRun
    status
    cancelRequested
    progress {
      processed
    }
    results {
      perPair {
        matched
        changed
      }
      skipped
      rejected {
        reason
        count
      }
    }
    error {
      code
      message
      attempts
    }
    submittedAt
    startedAt
    finishedAt
    expiresAt
  }
}

{
  "id": "<job ID returned by the bulk mutation>"
}

Poll until status is SUCCEEDED, FAILED, or CANCELLED. Pause between requests, for example a few seconds, because jobs run in batches and a large job can take a while. While a job is QUEUED, startedAt, finishedAt, and expiresAt are null. While it is RUNNING, finishedAt and expiresAt are null. error is null unless the status is FAILED.

To review what a job was asked to do, select the request field. It contains the filter, patches, and match as they were submitted. The submittedBy field shows which User Account or Service Account submitted the job.

List jobs

bulkJobs lists the jobs of the environment, newest first. It lists jobs of every tier, whoever submitted them. Use it to find the job that is blocking a new submission, or to review earlier runs.

query ListBulkJobs($input: BulkJobsInput) {
  bulkJobs(input: $input) {
    nodes {
      id
      operation
      tier
      dryRun
      status
      submittedBy {
        id
        name
        subjectType
      }
      submittedAt
      finishedAt
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}

{
  "input": {
    "filter": {
      "status": "RUNNING"
    },
    "first": 20
  }
}

The filter is optional. Use status to list jobs in one status, and dryRun to list only dry runs (true) or only real runs (false). When you give several filter fields, every field must match.

first accepts values from 1 to 100 and defaults to 20. To read the next page, pass the endCursor as after and repeat the query while hasNextPage is true. Use a cursor only with the same listing that produced it.

Cancel a job

Use cancelBulkJob with the job id.

mutation CancelBulkJob($input: CancelBulkJobInput!) {
  cancelBulkJob(input: $input) {
    job {
      id
      status
      cancelRequested
      progress {
        processed
      }
    }
  }
}

{
  "input": {
    "id": "<job ID returned by the bulk mutation>"
  }
}

The effect depends on the state of the job.

Job state	Result
QUEUED	The job is cancelled at once and never starts. The environment can accept a new job immediately.
RUNNING	The job stays RUNNING with cancelRequested set to true, and stops at its next batch boundary.
Final status	The job is returned unchanged.

A running job keeps the environment's active-job slot until it has stopped. Poll until the job reaches a final status before submitting another job; a job that finished before cancellation took effect can already be SUCCEEDED or FAILED.

You can cancel any job in the environment, whatever its tier and whoever submitted it. An ID that is not a valid job identifier fails with INVALID_JOB_ID, and an ID that matches no job fails with JOB_NOT_FOUND.

Partial completion and cancellation

Bulk jobs have no rollback. A job that is cancelled, or that ends with FAILED, leaves the changes it has already applied in place. Its results report how far it got.

For patch jobs, writes are not transactional, so one batch that is already in progress can still be applied after you cancel. For delete jobs, a batch is either fully committed or leaves data and progress unchanged.

After a partial run, review a new dry run before submitting another real run. Records already carrying the applicable patch are matched but not changed, and deletes only find records that still exist. However, a previous patch can change which conditions match on a later run. Review patch ordering and idempotence before resubmitting conditional patches.

Read the results

results describes what the job matched and did so far. It grows while the job runs and is final once the job reaches a final status.

Matched and changed

results.perPair has one entry for each match and patch pair, in the order you submitted them. A delete has exactly one entry, for its match.

Field	Patch jobs	Delete jobs
matched	Records the pair matched.	Records the match selected.
changed	Matched records where at least one property that the pair sets or removes differed from the stored value.	Matched records planned for deletion.

A record that already carries the patch counts as matched but not changed. A record that was rejected is not counted as changed.

These counts come from the scan, so they can include records that the job later skips. Treat changed as the number of records the job planned to write, and use skipped and rejected to see what was not written. After a hard crash, changed of a patch job can be lower than the number of patches that were applied.

In a dry run, the counts use the same planning and checks as a real run, but a dry run cannot see changes that happen at write time.

Skipped and rejected records

results.skipped counts planned writes that the job dropped because the record changed or disappeared after the job scanned it. A record is skipped even if it would still match. skipped is always 0 for a dry run. The job does not retry skipped records. To include them, submit the job again.

results.rejected counts matched records that the job did not write, grouped by reason. Each record is counted once, not once for each pair. The job continues after a rejection. The job does not store the identifiers of rejected records.

Reason	Meaning and what to do
VALUE_TOO_LARGE	The patched value would exceed the maximum JSON value size. Submit a corrected patch that keeps the value smaller, for example by also removing properties. Preview the corrected job with a dry run before applying it.

Dry runs apply the same size check, so a dry run shows VALUE_TOO_LARGE rejections before you change data. See Errors and limits for the value size limit.

To spot-check records after a job, use listArrayItems and listKeyValues.

Job history and retention

The environment keeps a record of every bulk job. When a job finishes, expiresAt shows when its record becomes eligible for automatic deletion. By default, finished real runs are kept for one year and finished dry runs for one week. These defaults are configurable for your deployment, so rely on expiresAt for the actual date. Queued and running jobs have no expiresAt.

Expiry cleanup is asynchronous. Once a record is removed, bulkJob returns null for its ID and bulkJobs no longer lists it. Removing a job record does not undo its changes to personalization data.