Why the Provisioning API Sometimes Returns “Null” Instead of a Message

[Lohmer]

Summary

The “next message” endpoint (GET /v1/subscriptions/{name}/messages/next) returns either a provisioning message as a JSON object, or the JSON value null (a 200 OK response with the literal body null).

At first glance, a null response would indicate that the message queue is currently empty. But there are actually four situations where the Provisioning API returns null:

1. The message queue is empty.
2. The prefill is in progress or has failed.
3. The previous message has not yet been acknowledged.
4. A suspected race condition between acknowledging the previous message and requesting the next one.

Fortunately, your provisioning consumer should handle all four cases the same way: keep polling until the prefill is finished, the next message is delivered, or the current message is redelivered.

We are investigating case 4 and planning on reducing the redelivery timeout.

1. The Message Queue is Empty

There are no pending messages for your subscription. Your consumer should continue polling. New messages will arrive as changes are made.

2. Prefill is Not Yet Complete

If a subscription was created with request_prefill: true, the endpoint returns null until prefill is complete. You can check the prefill status via the subscription endpoint:

GET /v1/subscriptions/{name}

The response includes a prefill_queue_status field with one of the following values: pending, done, or failed. The standard consumer credentials are sufficient to read their own subscription object.

The prefill will not be retried indefinitely. If it fails, the status is marked as failed and the endpoint will continue returning null. To resolve this, the subscription must be manually deleted and recreated.

Your consumer should continue polling. Messages will be delivered once the prefill reaches status done.

3. The Previous Message Has Not Yet Been Acknowledged

The Provisioning API enforces sequential message processing. Each consumer subscription is configured so that only one message can be “in flight” (delivered but not yet acknowledged) at a time. This guarantees that messages are processed in order.

If you request the next message without acknowledging the previous one, the API returns null. The expected workflow is:

1. Get (seq 42)
2. Acknowledge (seq 42)
3. Get (seq 43)

If step 2 is skipped, step 3 returns null. After a redelivery timeout (currently ~30 seconds), the same unacknowledged message is delivered again.

Your consumer should continue polling. The unacknowledged message will be redelivered after the timeout.

4. Race Condition Between Acknowledgement and Next Message

Even when a consumer correctly acknowledges each message before requesting the next one, null responses can still occur sporadically. This happens roughly every 100 messages in a worst-case load-testing scenario.

We suspect the root cause to be a race condition in our message queue backend. We are investigating this problem.

Your consumer should continue polling. The next message will be delivered after the redelivery timeout.

Affected Versions

Observed on all Provisioning Stack releases on UCS and Nubus for Kubernetes up to at least the time of writing (February 2026).