Troubleshooting & Best Practices
Build a webhook receiver that never loses or double-counts an event: acknowledge fast, verify first, deduplicate by event id, cope with ordering, keep secrets safe, and fix the common failures: signature mismatches, clock skew, redirects, timeouts and refused URLs.
Best practices#
Acknowledge fast, process later#
DevStride waits at most 10 seconds for an answer, and sends deliveries to one endpoint one at a time, so the time your receiver takes to answer is the speed of its queue. Do the minimum in the request:
- verify the signature;
- store the event (a job queue, a database row, a message on SQS, Pub/Sub or Kafka);
- answer
2xxwith an empty body.
Then do the real work (calling other APIs, updating records, posting to chat) in a separate worker. Answering before the event is stored safely risks losing it if your process crashes in between; answering after storing it is the safe order.
Verify before you trust anything#
Check X-DevStride-Signature on every request, against the raw body, before you parse it or act on it. Reject failures with 401. A request that fails verification is either not from DevStride or was altered on the way; nothing in it should reach your systems. The Verifying Signatures page has tested code for six languages.
Deduplicate by event id#
Delivery is at least once, so the same event can arrive twice. Record every event id you have processed and skip repeats:
CREATE TABLE devstride_webhook_events (
event_id uuid PRIMARY KEY,
received_at timestamptz NOT NULL DEFAULT now()
);
-- In the same transaction as the work the event causes:
INSERT INTO devstride_webhook_events (event_id) VALUES ($1)
ON CONFLICT (event_id) DO NOTHING;
-- 0 rows inserted: you have already handled this event; skip it.
Keep ids for at least 30 days, the period over which a delivery can be replayed, then delete old rows. The id is the same on every retry and replay, and the same for every endpoint that receives the event.
Answer 2xx to everything you verified#
Any status other than 2xx is retried for about a day and, while it is, holds up every newer delivery to that endpoint. So:
- answer
2xxto event types you do not handle, includingpingand types added in future; - answer
2xxto an event your code cannot process (a bug, an unexpected value), store it, and alert yourself, rather than letting it block the queue; - reserve error statuses for "try again later" situations you expect to clear, such as your database being briefly unavailable.
Be ready for any order#
Within one endpoint deliveries arrive in the order DevStride queued them, but across endpoints there is no order at all, and two changes made almost at once can be queued the other way round. When order matters:
- compare
occurredAtand ignore an event older than the state you already hold; - for status changes, check that
previousStatusIdmatches the status you have on record; - when in doubt, treat the event as a signal and read the current state from the API by the ids it carries.
Ignore what you do not recognize#
New fields and event types are added without notice. Read the fields you need, ignore unknown keys, and never fail on an unknown type. See Compatibility.
Filter internal notes and drop your own echoes#
comment.created includes internal notes: check visibility is public before showing a comment to anyone outside your team. If your integration also writes to DevStride (for example syncing comments back), run it as a dedicated member and skip events whose origin.actorId is that member's username, so you do not process your own writes as new ones.
Protect the secret#
Keep each endpoint's secret in a secret manager or environment variable, never in source code or logs. Use a different endpoint, and so a different secret, for each system and environment (production and staging should never share one). Rotate on a schedule and whenever someone who had access leaves; if a secret may have leaked, rotate twice to retire it immediately (why).
Keep responses small#
DevStride reads at most 64 KB of your response; a larger one counts as a failure even with a 2xx status. An empty 204 No Content is ideal.
Watch the health signals#
The failing and recovered emails go to everyone with Manage Webhooks: make sure that includes someone who will act on them. The red banner and the delivery log on the Webhooks page show what is failing and why. See Failing and recovered alerts.
Network and firewall notes#
- Allow-list by signature, not by IP address. Deliveries do not come from a fixed, published set of IP addresses, so source-IP filtering is not a reliable way to accept DevStride and refuse everyone else. Authenticate each request by its signature.
- Your endpoint must be reachable from the public internet on HTTPS, with a certificate a standard client trusts (not expired, not self-signed, matching the host name).
- DevStride will not call private addresses. URLs whose host resolves to a private network, loopback, link-local or cloud-metadata address are refused when you save the endpoint and on every attempt. To deliver into a private network, expose a small public relay that verifies the signature and forwards inward.
User-Agent: DevStride-Webhooks/1identifies the requests in your logs and WAF rules. Anyone can send that header, so never treat it as proof.
Common problems#
| Symptom | Likely cause | Fix |
|---|---|---|
Every delivery gets 401 from your receiver |
The body was parsed and re-serialized before verifying, or the wrong secret is configured. | Verify the raw bytes (pitfalls by framework); check you copied the whole secret, whsec_ prefix included, for this endpoint. Run your code against the test vector. |
| Signatures fail only on some events | Characters outside plain ASCII (accented names, emoji) are being re-encoded, or a size limit truncates large comment events. | Hash the bytes as received, not a decoded and re-encoded string; allow request bodies of at least 1 MB. |
| Signatures fail on every event after a while, or intermittently | Your server's clock has drifted more than 5 minutes. | Synchronize the clock with NTP. |
| Signatures start failing about a day after a rotation | The receiver still has the previous secret, which stopped being sent 24 hours after the rotation. | Deploy the current secret, or rotate again and deploy that one. |
Deliveries fail with 413 |
Your server or proxy limits request size (Express defaults to 100 KB). | Raise the limit to at least 1 MB on the webhook route. |
Last response 301, 302 or 303 |
The URL redirects (for example a trailing slash, or www to the apex), and DevStride does not follow redirects that would turn the POST into a GET. |
Register the final URL. |
| No response: a timeout or Fetch deadline exceeded | The receiver took longer than 10 seconds to answer, or the whole request took more than 15 seconds. | Answer first and process in the background. |
| No response: Blocked url scheme 'http:' | A redirect sent the request to plain http://. |
Serve the endpoint, and any redirect target, over HTTPS only. |
| No response: Blocked request to non-public address | The host now resolves to a private or internal address. | Point the name at a public address, or use a public relay. |
| No response: Host could not be resolved | The DNS name does not exist (or no longer does). | Fix the DNS record or the endpoint URL. |
| No response: a certificate error | The certificate is expired, self-signed or for another name. | Install a certificate from a public certificate authority. |
| Creating or saving an endpoint is refused | The URL is not https://, does not resolve, resolves to a refused address, or the organization already has 20 endpoints. |
Read the message under Endpoint URL; fix the URL or delete an unused endpoint. |
A 2xx response still counts as failed |
The response body is over 64 KB. | Answer with an empty or short body. |
| Events stopped arriving and nothing shows as failing | The endpoint is disabled (changes are not queued while it is), or it is no longer subscribed to that event type. | Check Enabled and Events in its row. Changes made while it was disabled are not sent; catch up through the API. |
| New events arrive hours late | An older delivery to the same endpoint is failing and waiting for retries, which holds back the ones behind it. | Find the oldest pending row in the delivery log, fix whatever makes it fail, and the queue drains in order. |
| The same event processed twice | A retry or replay after your receiver had already handled it. | Deduplicate by event id. |
Privacy: erasure and redaction#
Webhook deliveries are copies of your data, so DevStride's privacy tools reach them too:
- Erasing a requester (Anonymize…) deletes every stored
item.createdandcomment.createddelivery that carries that requester's tickets or words, and everycustomer.messageaddressed to them, written by them, or listing them among a reply'sparticipants, including those of requesters merged into them, from the delivery log. Deliveries still waiting to be sent are deleted too, so they are never sent. - Redacting a comment deletes that comment's stored
comment.createddeliveries the same way.
Once deleted, those deliveries can no longer be viewed or replayed. Status and SLA events carry only ids, and are not affected.
Copies already delivered are yours to erase
DevStride cannot reach into your systems. An event your endpoint already received stays wherever you stored it, and no event is sent when a requester is erased or a comment is redacted. If you store comment text or ticket titles from webhooks, include those stores in your own privacy process, and keep only what you need.