# Troubleshooting Source: https://docs.pitchpointsolutions.com/developer_guides/webhooks/troubleshooting/ **Beta** ## Overview An event has not arrived. This guide works through it in the order that narrows fastest: ask whether the endpoint is receiving at all, then whether a delivery was created, then what happened to it, and finally how to send it again. ## Step 1: Ask whether the endpoint is receiving Fetch the endpoint and read `receiving`, not `enabled`. `receiving` is true only when all three conditions hold: the endpoint is enabled, its circuit is `CLOSED`, and it has declared at least one product and topic in `acceptedEventTypes`. `receiving` is one answer for the whole endpoint, not one per product or topic. An endpoint that declared `workflow.updated` for `ADV-120` reports `receiving: true` even though it will never receive `finding.reviewed`, or anything for another product. If `receiving` is true and an event still did not arrive, compare the event's product and topic with `acceptedEventTypes` before anything else. This is the most common misdiagnosis. ```bash curl -X GET "https://api.pointservices.com/riskinsight-services-ws/resources/v1/webhooks/endpoints/4711" \ -H "Authorization: Bearer your_access_token_here" ``` ```json { "id": 4711, "organizationalUnitGuid": "OU41502", "displayName": "Production order hook", "url": "https://hooks.example.com/pps/workflows", "enabled": true, "circuitState": "CLOSED", "receiving": false, "notReceivingReason": "NOT_DECLARED", "hasNoContactsRegistered": false, "signatureMode": "SYMMETRIC", "acceptedEventTypes": [] } ``` `enabled: true` with `receiving: false` is the common shape of this problem. `notReceivingReason` names the one condition that failed, and therefore who fixes it. | notReceivingReason | What it means | Your next move | |---|---|---| | NOT_DECLARED | The endpoint declares no accepted event types, so no event can ever match it. | Edit the endpoint and add the products and topics you want. | | DISABLED | The endpoint is switched off (`enabled: false`). PPS does not switch an endpoint off after failures; it opens the circuit instead. | Re-enable it, then bulk replay to recover what was parked. | | CIRCUIT_OPEN | The circuit breaker opened after repeated delivery failures or a `410 Gone`. | Fix your endpoint, then send a test delivery; a successful one closes the circuit. | | NO_GRANT | No live grant covers the endpoint's declared selectors. An endpoint that has declared anything receives its own org unit's events without a grant, so this value is not expected in practice. Handle it anyway. | Contact your PPS account manager; only PPS writes grants. Include your org unit GUID, the `serviceName` and `modelName`, and the topics you need. | Exactly one cause is reported even when several hold at once, in the order of the table: `NOT_DECLARED`, `DISABLED`, `CIRCUIT_OPEN`, then `NO_GRANT`. Work the reported cause, then read the field again. `receiving` covers your own org unit's events. If the events you are waiting for are produced in another org unit, they also need a subscription grant naming your endpoint's org unit as the destination, and `receiving: true` does not tell you whether one exists. [Simulate an event](https://docs.pitchpointsolutions.com/developer_guides/webhooks/troubleshooting/#exercising-the-fan-out-end-to-end) to find out: it reports a delivery for each org unit that routes the product and topic to you. Two traps worth checking at this point. A misspelled `acceptedEventTypes` is dropped silently, because unknown keys are ignored rather than rejected — compare what the response echoes back against what you sent. And a mistyped `modelName` is stored as given and matched exactly at delivery time, so it is accepted and then never matches anything. > **Warning:** An endpoint moved to another org unit goes silently dead if that org unit is not the one producing the events you want, or if no grant names it for another org unit's events. A move returns 200, the endpoint still reports `enabled: true` and `receiving: true`, and deliveries stop arriving. If deliveries stopped after an edit, check `organizationalUnitGuid` against the value you expect. > **API Reference:** [Endpoints API Reference](https://docs.pitchpointsolutions.com/api/webhooks/endpoints) ## Step 2: Ask whether the delivery exists If the endpoint is receiving, look for the delivery row. The listing takes its filter in the request body, so it is a POST that creates nothing. ```bash curl -X POST https://api.pointservices.com/riskinsight-services-ws/resources/v1/webhooks/deliveries/4711/query \ -H "Authorization: Bearer your_access_token_here" \ -H "Content-Type: application/json" \ -d '{ "status": "FAILED", "since": "2026-08-01T00:00:00Z", "until": "2026-08-28T00:00:00Z", "limit": 2 }' ``` ```json { "limit": 2, "status": "FAILED", "count": 2, "nextMarker": "80514", "results": [ { "id": 80512, "eventId": "msg_9f1c2e4a-6b73-3d81-a0c5-4e2f7b9d1350", "topic": "workflow.updated", "url": "https://hooks.example.com/pps/workflows", "status": "FAILED", "simulated": false, "lastResponseStatus": 500, "lastFailureReason": "HTTP_500" }, { "id": 80514, "eventId": "msg_2d7a4c19-8e05-3f62-b431-7c9a0e5d8f26", "topic": "workflow.failed", "url": "https://hooks.example.com/pps/workflows", "status": "FAILED", "simulated": false, "lastResponseStatus": null, "lastFailureReason": "TRANSPORT_SocketTimeoutException" } ] } ``` The window is over each delivery's creation time, which is when the event occurred, not when it was last attempted. `since` is inclusive and `until` is exclusive, so consecutive windows can share a boundary without returning anything twice. `count` equals `limit` here, so the page was full and carries a `nextMarker`. Pass that value back as `marker` for the next page. The small `limit` above keeps the example short; use a larger one in practice. Valid values are 1 to 100, and anything outside that range is replaced by the default of 100 rather than clamped or rejected, so a computed `limit` of `0` returns 100 rows. Compare the response's own `limit` against what you sent. > **Warning:** `nextMarker` on a delivery listing is present only when the page was full, so a page that is not full is the last one and you may stop there. The [endpoint listing](https://docs.pitchpointsolutions.com/api/webhooks/endpoints#list-endpoints) is the opposite: a short or even empty page can still carry a cursor, so there you stop only when `nextMarker` is absent. Do not reuse one paging loop for both. No delivery row at all means the event was never routed to this endpoint. Go back to step 1: the endpoint was not receiving when the event fired, the product and topic did not match what it declared, or the event came from another org unit that no grant routes to yours. ## Step 3: Read what happened to it Fetch the single delivery. This is the one place `payload` and `lastResponseBody` are exposed, so it is where you see exactly what PPS sent and exactly what your server answered. ```bash curl -X GET "https://api.pointservices.com/riskinsight-services-ws/resources/v1/webhooks/deliveries/4711/80512" \ -H "Authorization: Bearer your_access_token_here" ``` ```json { "id": 80512, "eventId": "msg_9f1c2e4a-6b73-3d81-a0c5-4e2f7b9d1350", "topic": "workflow.updated", "url": "https://hooks.example.com/pps/workflows", "status": "FAILED", "simulated": false, "lastResponseStatus": 500, "lastFailureReason": "HTTP_500", "lastResponseBody": "{\"error\":\"internal\"}", "payload": { "type": "workflow.updated", "eventId": "msg_9f1c2e4a-6b73-3d81-a0c5-4e2f7b9d1350", "timestamp": "2026-08-28T14:26:03Z", "data": { "workflowId": "wfpop_7b3f1c9e-2a84-4d16-9f05-c3e8a1b47d92", "runId": "wfrunpop_7b3f1c9e-2a84-4d16-9f05-c3e8a1b47d92", "correlationId": "9c2e14a7-55d0-4b8e-a731-6f0d2b93e185", "serviceName": "ADV", "modelName": "ADV-120", "status": "ready" } } } ``` `lastResponseStatus` and `lastFailureReason` tell you whose problem it is. A 500 from your server is yours. A `null` status with a transport reason means your server never answered at all. `url` is the address snapshotted when the delivery was created, so it tells you where the attempt actually went even if you have edited the endpoint since. `lastResponseBody` is whatever your own server sent back, so treat it as untrusted text: it may contain anything. PPS keeps the first 8,192 characters and discards the rest, with no marker where the cut falls — a body of exactly that length and a body of 50 MB look the same to you. If you are reading it to debug and the JSON stops mid-token, suspect truncation before you suspect your serializer. Put the part you will actually want first. An error code and a message at the start of your response body survive; a stack trace appended after 8,192 characters of context does not. A delivery id that does not belong to the endpoint in the path is a 404, on the same terms as an endpoint you may not see. ## Step 4: Replay one delivery Once your server is fixed, replay the delivery. The request has no body. ```bash curl -X POST https://api.pointservices.com/riskinsight-services-ws/resources/v1/webhooks/deliveries/4711/80512/replay \ -H "Authorization: Bearer your_access_token_here" ``` Check before you replay rather than after. Read `circuitState` on the endpoint and `status` on the delivery: an `OPEN` circuit, or a delivery already `PENDING` or `RETRYING`, means the call will do nothing. Delivery 80512 is `FAILED` and the circuit is `CLOSED`, so the replay above is queued. > **Warning:** A 200 from this operation does not mean the replay happened, and nothing in the response says which you got. A delivery in `PENDING` or `RETRYING`, or an endpoint whose circuit is `OPEN`, causes the call to do nothing and still return 200 with the delivery unchanged. There is no 409 and no error body. The body also reports the delivery's state as it stood before the replay, so a delivery that is now `PENDING` is still reported as `FAILED` — do not drive your state machine from it. Afterwards you can fetch the delivery again: a replay that was actually queued leaves it `PENDING`, sets `nextAttemptDateTime` to the instant of the next attempt, and clears any `deliveredDateTime` it carried. ```bash curl -X GET "https://api.pointservices.com/riskinsight-services-ws/resources/v1/webhooks/deliveries/4711/80512" \ -H "Authorization: Bearer your_access_token_here" ``` ```json { "id": 80512, "eventId": "msg_9f1c2e4a-6b73-3d81-a0c5-4e2f7b9d1350", "topic": "workflow.updated", "url": "https://hooks.example.com/pps/workflows", "status": "PENDING", "simulated": false, "nextAttemptDateTime": "2026-08-28T15:12:40Z", "lastResponseStatus": 500, "lastFailureReason": "HTTP_500", "lastResponseBody": "{\"error\":\"internal\"}", "payload": { "type": "workflow.updated", "eventId": "msg_9f1c2e4a-6b73-3d81-a0c5-4e2f7b9d1350", "timestamp": "2026-08-28T14:26:03Z", "data": { "workflowId": "wfpop_7b3f1c9e-2a84-4d16-9f05-c3e8a1b47d92", "runId": "wfrunpop_7b3f1c9e-2a84-4d16-9f05-c3e8a1b47d92", "correlationId": "9c2e14a7-55d0-4b8e-a731-6f0d2b93e185", "serviceName": "ADV", "modelName": "ADV-120", "status": "ready" } } } ``` > **Warning:** That after-the-fact check cannot tell a queued replay from a refused one on a delivery that was already `PENDING` or `RETRYING`. Being already `PENDING` is itself one of the two refusal conditions, so the delivery reads as `PENDING` either way and the check reports success for a replay that did nothing. This is why the reliable move is to read the status and the circuit before you call. A replay keeps the delivery's original `eventId`, so a consumer deduplicating on that value recognizes it as the same business event rather than as a new one. A replay is a redelivery, not a new event. A delivery is replayable only while its row still exists. Rows are removed once they fall outside the delivery retention window, which is 90 days. ## Step 5: Replay a whole window When an outage cost you more than one delivery, replay the window rather than each row. The call is asynchronous and returns a task handle. ```bash curl -X POST https://api.pointservices.com/riskinsight-services-ws/resources/v1/webhooks/deliveries/4711/replay \ -H "Authorization: Bearer your_access_token_here" \ -H "Content-Type: application/json" \ -d '{ "since": "2026-08-01T00:00:00Z", "until": "2026-08-28T00:00:00Z" }' ``` PPS answers with a 200. The deliveries are queued after the response returns, so poll the task rather than reading this body as a result. ```json { "taskId": "wrt_60184", "endpointId": 4711, "status": "PENDING", "since": "2026-08-01T00:00:00Z", "until": "2026-08-28T00:00:00Z", "replayed": 0 } ``` The window is the bound. `since` is required and `until` is optional, and there is no per-request ceiling on how many deliveries one replay may queue: narrow the window if you want to queue fewer. Only absolute instants are accepted; a relative window such as "the last 30 days" is not. ### Choosing what to replay Omitting `status` selects the two stalled states. `FAILED` is a delivery that exhausted its attempts; `PARKED` is one held aside when its endpoint was disabled or its domain lost authorization, not discarded. That default is what recovers a disabled endpoint: enable it again, then run a bulk replay with no `status`, and the deliveries parked while it was off are queued along with any that exhausted their retries. > **Warning:** A delivery PPS made successfully is `DELIVERED` however your service handled it afterwards, so the default selection does not reach events your own consumer dropped. If your consumer acknowledged deliveries with a 2xx during an outage and then lost them, name `status: "DELIVERED"` and the window your consumer was broken in. This re-sends events your service has already seen, so your handler must tolerate duplicates. ```bash curl -X POST https://api.pointservices.com/riskinsight-services-ws/resources/v1/webhooks/deliveries/4711/replay \ -H "Authorization: Bearer your_access_token_here" \ -H "Content-Type: application/json" \ -d '{ "since": "2026-08-01T00:00:00Z", "until": "2026-08-28T00:00:00Z", "status": "DELIVERED" }' ``` `CANCELLED` is the fourth replayable status. It means the authorizing grant was revoked. PPS does not set it on deliveries, but if you see it and want those deliveries re-sent, name `status: "CANCELLED"` and the window they fall in — the default selection does not reach them. For "deliveries stopped after working for a week", work `receiving` and `notReceivingReason` in step 1 instead. `PARKED`, by contrast, is recoverable rather than terminal: those deliveries are held aside, not given up on, and the default selection does reach them. Naming `PENDING` or `RETRYING` is rejected with a 400, because those rows still have a live delivery task driving them and replaying one would race it. A bulk replay into an endpoint whose circuit is `OPEN` is refused with a 409, never silently accepted. Close the circuit first — fix the endpoint and wait for the next probe, or send a test delivery, which is not subject to the circuit and closes it on success. ### Pacing Replayed deliveries are spread two seconds apart rather than queued all at once, so a replay of 100 deliveries drains over roughly three and a half minutes. Do not read a slow drain as a failed replay. > **Warning:** Concurrent bulk replays against the same endpoint are bounded, not prevented. Nothing serializes two replays whose windows overlap, so a delivery selected by both can be reset and enqueued twice. Deduplicating on `eventId` covers the second delivery; it does not cover the row's delivery state being rewritten underneath you. If a replay seems slow, poll the task rather than issuing a second call, which doubles the traffic. ## Step 6: Poll the replay task ```bash curl -X GET "https://api.pointservices.com/riskinsight-services-ws/resources/v1/webhooks/deliveries/4711/replay/wrt_60184" \ -H "Authorization: Bearer your_access_token_here" ``` ```json { "taskId": "wrt_60184", "endpointId": 4711, "status": "COMPLETED", "since": "2026-08-01T00:00:00Z", "until": "2026-08-28T00:00:00Z", "replayed": 1883 } ``` Poll until the task is no longer `PENDING` or `RUNNING`. `replayed` grows while `RUNNING` and is final at `COMPLETED`. A zero on a `COMPLETED` task means nothing matched the filter, never that the endpoint refused — a refusal is the 409 on the replay call itself. ```python import time import requests BASE = "https://api.pointservices.com/riskinsight-services-ws/resources" TERMINAL = {"COMPLETED", "FAILED"} def wait_for_replay(token, endpoint_id, task_id, poll_seconds=10, timeout_seconds=1800): deadline = time.monotonic() + timeout_seconds while time.monotonic() < deadline: response = requests.get( f"{BASE}/v1/webhooks/deliveries/{endpoint_id}/replay/{task_id}", headers={"Authorization": f"Bearer {token}"}, timeout=30, ) response.raise_for_status() task = response.json() if task["status"] in TERMINAL: return task time.sleep(poll_seconds) raise TimeoutError(f"Replay task {task_id} did not finish within {timeout_seconds}s") ``` Branch on `status`, never on `stoppedReason`, which is text written for a person to read and is present only on a `FAILED` task. Task status is retained for 30 days. After that the task id reads as a 404, so record the outcome you care about rather than relying on the task as a durable record. > **API Reference:** [Delivery statuses](https://docs.pitchpointsolutions.com/api/webhooks/deliveries#delivery-statuses) defines all six, and [Deliveries API Reference](https://docs.pitchpointsolutions.com/api/webhooks/deliveries) covers every delivery operation. ## Your destination domain is not authorized PPS must authorize a destination domain before it will send webhooks to it. Registering or editing an endpoint onto an unauthorized domain fails with a 400 whose `detail` names the domain, and the endpoint is not created. ```json { "type": "about:blank", "title": "Bad Request", "status": 400, "detail": "Webhook URL host is not an approved destination: hooks.example.net. Contact PPS support to request approval.", "instance": "urn:pps:request:5a7d02be-63c1-4f89-b204-9e138c7a06f5" } ``` You cannot fix this in the request. Ask your PPS account manager to authorize the domain, then send the same register call again. HTTPS is required in every case. ## Deliveries are parking and your endpoint looks healthy Deliveries move to `PARKED` when the endpoint's destination domain is no longer authorized. Nothing is discarded and no attempt is spent. > **Warning:** The endpoint itself stays enabled, so `enabled` and `circuitState` look normal and nothing notifies you. A run of `PARKED` deliveries, or a 409 from a test send saying the destination domain is no longer authorized, is how this surfaces. Watch for parked deliveries rather than waiting to be told. To recover, contact your PPS account manager, or edit the endpoint onto a URL on a different domain. Then run a bulk replay to release what parked. ## Exercising the fan out end to end When you want to confirm the whole path rather than one endpoint, simulate an event. It fires through the real fan out to every endpoint entitled to receive it. > **Warning:** Simulating is the real fan out, not a dry run. It writes real delivery rows that appear in your delivery listing, count toward retention, retry on the ordinary schedule, and are replayable. A failing receiver will open the circuit exactly as a real event would, and the endpoint then stops receiving real traffic until the circuit closes. To exercise a single endpoint without that risk, send a test delivery instead. ```bash curl -X POST https://api.pointservices.com/riskinsight-services-ws/resources/v1/webhooks/events/simulate \ -H "Authorization: Bearer your_access_token_here" \ -H "Content-Type: application/json" \ -d '{ "eventType": "workflow.updated", "serviceName": "ADV", "modelName": "ADV-120" }' ``` ```json { "eventId": "msg_1d4b7e02-5a93-4c68-b7f1-0e2a9c6d5384", "simulated": true, "deliveries": [ { "deliveryId": 80531, "endpointId": 4711, "eventId": "msg_1d4b7e02-5a93-4c68-b7f1-0e2a9c6d5384", "sourceOuGuid": "OU41502" } ], "warnings": [] } ``` An empty `deliveries` array is a real answer rather than a failure: nothing was entitled to receive an event of this shape, and `warnings` says which condition was missing. `sourceOuGuid` names the org unit the event was attributed to. PPS always fires one event attributed to your own org unit, which needs no grant and carries no `subscriptionId`, and one more for each other org unit whose grant routes this product and topic to you. A simulation delivers only to endpoints in your own org unit. > **API Reference:** [Events API Reference](https://docs.pitchpointsolutions.com/api/webhooks/events) ## Quoting a request when you raise a ticket Every error body carries `instance`, written as `urn:pps:request:`. The id inside it is the request correlation id, the same value returned in the `x-pps-request-id` and `X-PPS-CorrelationID` response headers. Quote it when you raise a support ticket; it is what lets PPS find that exact request. Do not quote your signing secret. No PPS call needs it, and no response ever returns it.