Out-of-order AdmissionWebhook events on OBS reconnect #1909
Description
Describe the bug
When a publisher (e.g. OBS) reconnects (such as after a network interruption), OvenMediaEngine sends AdmissionWebhook events out of order. Specifically, a "closing" webhook for the old connection is sent after a new "opening" webhook for the reconnected session. This causes our application to incorrectly mark the stream as offline, as we rely on the received webhooks to update a stream's status in our database, even though OBS and OME show the stream as live.
To Reproduce
Steps to reproduce the behavior:
- Configure OME with admissionwebhooks
- Use OBS (31.1.2) to publish a stream to OME.
- While streaming, disconnect the network (e.g., turn off WiFi). Wait for OBS to show "reconnecting", then restore the network.
- Observe the order of webhook events received by the control server.
Expected behavior
Webhook events should be sent in the correct order: each "opening" event should be followed by a corresponding "closing" event for the same connection. After a successful reconnect, the last webhook sent by OME should be a "status":"opening" webhook, but instead, the final webhook received after successful reconnection, is a "status":"closing" webhook
Logs
These are the logs produced by the webhook server, observe the final webhook is status:closing, at that point, OBS has successfully reconnected
Admission webhook server listening on :8080 (POST /admission)
2025-08-28T17:15:20Z - ⬇️ Starting stream
[AdmissionWebhook] Payload received:
{"client":{"address":"my ip","port":61014,"real_ip":"my ip"},"request":{"direction":"incoming","protocol":"RTMP","status":"opening","time":"2025-08-28T17:15:20.443+00:00","url":"rtmp://server:1935/live/1"}}
[ OBS STARTS ATTEMPTING RECONNECTION]
2025-08-28T17:16:20Z - ⬇️ Starting stream
[AdmissionWebhook] Payload received:
{"client":{"address":"my ip","port":61008,"real_ip":"my ip"},"request":{"direction":"incoming","protocol":"RTMP","status":"opening","time":"2025-08-28T17:16:20.824+00:00","url":"rtmp://server:1935/live/1"}}
2025-08-28T17:16:22Z - ⬆️ Closing stream
[AdmissionWebhook] Payload received:
{"client":{"address":"my ip","port":61008,"real_ip":"my ip"},"request":{"direction":"incoming","new_url":"rtmp://server:1935/live/1","protocol":"RTMP","status":"closing","time":"2025-08-28T17:16:22.642+00:00","url":"rtmp://server:1935/live/1"}}
2025-08-28T17:16:25Z - ⬇️ Starting stream
[AdmissionWebhook] Payload received:
{"client":{"address":"my ip","port":61008,"real_ip":"my ip"},"request":{"direction":"incoming","protocol":"RTMP","status":"opening","time":"2025-08-28T17:16:25.369+00:00","url":"rtmp://server:1935/live/1"}}
2025-08-28T17:16:27Z - ⬆️ Closing stream
[AdmissionWebhook] Payload received:
{"client":{"address":"my ip","port":61008,"real_ip":"my ip"},"request":{"direction":"incoming","new_url":"rtmp://server:1935/live/1","protocol":"RTMP","status":"closing","time":"2025-08-28T17:16:27.210+00:00","url":"rtmp://server:1935/live/1"}}
2025-08-28T17:16:30Z - ⬇️ Starting stream
[AdmissionWebhook] Payload received:
{"client":{"address":"my ip","port":61037,"real_ip":"my ip"},"request":{"direction":"incoming","protocol":"RTMP","status":"opening","time":"2025-08-28T17:16:30.189+00:00","url":"rtmp://server:1935/live/1"}}
[AT THIS POINT OBS SHOWS THE RECONNECT WAS SUCCESFUL]
2025-08-28T17:16:30Z - ⬆️ Closing stream
[AdmissionWebhook] Payload received:
{"client":{"address":"my ip","port":61014,"real_ip":"my ip"},"request":{"direction":"incoming","new_url":"rtmp://server:1935/live/1","protocol":"RTMP","status":"closing","time":"2025-08-28T17:16:30.288+00:00","url":"rtmp://server:1935/live/1"}}
Server (please complete the following information):
- OME: latest docker image
Additional context
This bug causes our website to incorrectly mark streams as offline when they are actually live, due to the out-of-order webhook events. This can be reproduced reliably by interrupting and restoring the network connection during an active RTMP publish session.