Kepware reinitialize results in "System.ObjectDisposedException"

[b-enigma-con]

Describe the bug
OPC Publisher opc-publisher:2.9.12-preview3 is running with no issues for days. The underlying OPC UA server is KepServerEX, which has a reinitialize function. This function is used when the runtime server needs a reset (disconnects all clients). The OPC Publisher does not seem to be able to successfully re-establish a connection afterwards sometimes, resulting in a "System.ObjectDisposedException": "Could not reconnect the Session. System.ObjectDisposedException: Cannot access a disposed object.". This error does not recover anymore and requires a OPC Publisher restart to start working again.

This error seems to be a result of fixing the following issue: #2344

To Reproduce
Steps to reproduce the behavior:

1. Run and configure opc-publisher:2.9.12-preview3 with KepServerEX OPC UA server
2. Reinitialize KepServerEX. Multiple times might be needed to trigger the issue.
3. Inspect OPC Publisher logs

Expected behavior
I would expect the OPC Publisher to re-establish the connection successfully all the times.

Screenshots
N.A.

Additional context
Log snippet:

[25-01-29 12:08:43.6979] fail: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaStack[0]
Publish #99, Reconnecting=False, Error: BadSecureChannelClosed
[25-01-29 12:08:43.6980] fail: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaStack[0]
Publish #100, Reconnecting=False, Error: BadSecureChannelClosed
[25-01-29 12:08:43.6985] fail: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaClient[0]
opc.tcp://xxxx:49320_24F41FFE_da39a3ee5e6b4b0d3255bfef95601890afd80709 [state:Ready|refs:1]: Error BadSecureChannelClosed 'BadSecureChannelClosed' during Publish - triggering reconnect...
[25-01-29 12:08:43.6987] info: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaClient[0]
opc.tcp://xxxx:49320_24F41FFE_da39a3ee5e6b4b0d3255bfef95601890afd80709 [state:Ready|refs:1]: Reconnecting session opc.tcp://xxxx:49320_24F41FFE_da39a3ee5e6b4b0d3255bfef95601890afd80709 due to error BadSecureChannelClosed 'BadSecureChannelClosed'...
[25-01-29 12:08:43.6987] info: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaClient[0]
opc.tcp://xxxx:49320_24F41FFE_da39a3ee5e6b4b0d3255bfef95601890afd80709 [state:Connecting|refs:1]: Session opc.tcp://xxxx:49320_24F41FFE_da39a3ee5e6b4b0d3255bfef95601890afd80709 with opc.tcp://xxxx:49320 changed from Ready to Connecting
[25-01-29 12:08:44.6706] fail: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaStack[0]
Could not reconnect the Session. System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription'.
at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription.OnStateChange(Subscription subscription, SubscriptionStateChangedEventArgs e) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 2481
at Opc.Ua.Client.Subscription.ChangesCompleted()
at Opc.Ua.Client.Subscription.AddItems(IEnumerable1 monitoredItems) at Opc.Ua.Client.Subscription..ctor(Subscription template, Boolean copyEventHandlers) at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription..ctor(OpcUaSubscription subscription, Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 193 at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription.CloneSubscription(Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 273 at Opc.Ua.Client.Session..ctor(ITransportChannel channel, Session template, Boolean copyEventHandlers) at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSession..ctor(OpcUaSession session, ITransportChannel channel, Session template, Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSession.cs:line 118 at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSession.CloneSession(ITransportChannel channel, Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSession.cs:line 174 at Opc.Ua.Client.Session.RecreateAsync(Session sessionTemplate, ITransportChannel transportChannel, CancellationToken ct) at Opc.Ua.Client.DefaultSessionFactory.RecreateAsync(ISession sessionTemplate, ITransportChannel transportChannel, CancellationToken ct) at Opc.Ua.Client.SessionReconnectHandler.DoReconnectAsync() [25-01-29 12:08:46.7499] fail: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaStack[0] Could not reconnect the Session. System.ObjectDisposedException: Cannot access a disposed object. Object name: 'Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription'. at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription.OnStateChange(Subscription subscription, SubscriptionStateChangedEventArgs e) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 2481 at Opc.Ua.Client.Subscription.ChangesCompleted() at Opc.Ua.Client.Subscription.AddItems(IEnumerable1 monitoredItems)
at Opc.Ua.Client.Subscription..ctor(Subscription template, Boolean copyEventHandlers)
at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription..ctor(OpcUaSubscription subscription, Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 193
at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription.CloneSubscription(Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 273
at Opc.Ua.Client.Session..ctor(ITransportChannel channel, Session template, Boolean copyEventHandlers)
at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSession..ctor(OpcUaSession session, ITransportChannel channel, Session template, Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSession.cs:line 118
at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSession.CloneSession(ITransportChannel channel, Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSession.cs:line 174
at Opc.Ua.Client.Session.RecreateAsync(Session sessionTemplate, CancellationToken ct)
at Opc.Ua.Client.Session.RecreateAsync(Session sessionTemplate, ITransportChannel transportChannel, CancellationToken ct)
at Opc.Ua.Client.DefaultSessionFactory.RecreateAsync(ISession sessionTemplate, ITransportChannel transportChannel, CancellationToken ct)
at Opc.Ua.Client.SessionReconnectHandler.DoReconnectAsync()
[25-01-29 12:08:50.4061] fail: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaStack[0]
Could not reconnect the Session. System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription'.
at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription.OnStateChange(Subscription subscription, SubscriptionStateChangedEventArgs e) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 2481
at Opc.Ua.Client.Subscription.ChangesCompleted()
at Opc.Ua.Client.Subscription.AddItems(IEnumerable1 monitoredItems) at Opc.Ua.Client.Subscription..ctor(Subscription template, Boolean copyEventHandlers) at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription..ctor(OpcUaSubscription subscription, Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 193 at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription.CloneSubscription(Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 273 at Opc.Ua.Client.Session..ctor(ITransportChannel channel, Session template, Boolean copyEventHandlers) at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSession..ctor(OpcUaSession session, ITransportChannel channel, Session template, Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSession.cs:line 118 at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSession.CloneSession(ITransportChannel channel, Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSession.cs:line 174 at Opc.Ua.Client.Session.RecreateAsync(Session sessionTemplate, CancellationToken ct) at Opc.Ua.Client.Session.RecreateAsync(Session sessionTemplate, ITransportChannel transportChannel, CancellationToken ct) at Opc.Ua.Client.DefaultSessionFactory.RecreateAsync(ISession sessionTemplate, ITransportChannel transportChannel, CancellationToken ct) at Opc.Ua.Client.SessionReconnectHandler.DoReconnectAsync() [25-01-29 12:08:50.7800] info: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription[0] #1/30: Subscription 1:1:d003442a44b5bfcf4269445ebf2f8b2d942d5426[P0@0] is missing keep alive. [25-01-29 12:08:56.7268] fail: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaStack[0] Error while raising PublishStateChanged event. System.ObjectDisposedException: Cannot access a disposed object. Object name: 'Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription'. at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription.OnPublishStatusChange(Subscription subscription, PublishStateChangedEventArgs e) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 2427 at Opc.Ua.Client.Subscription.HandleOnKeepAliveStopped() [25-01-29 12:08:56.7272] fail: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaStack[0] Unexpected error sending publish request. Opc.Ua.ServiceResultException: Channel has been closed. at Opc.Ua.Client.Session.BeginPublish(Int32 timeout) [25-01-29 12:08:57.7146] fail: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaStack[0] Could not reconnect the Session. System.ObjectDisposedException: Cannot access a disposed object. Object name: 'Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription'. at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription.OnStateChange(Subscription subscription, SubscriptionStateChangedEventArgs e) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 2481 at Opc.Ua.Client.Subscription.ChangesCompleted() at Opc.Ua.Client.Subscription.AddItems(IEnumerable1 monitoredItems)
at Opc.Ua.Client.Subscription..ctor(Subscription template, Boolean copyEventHandlers)
at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription..ctor(OpcUaSubscription subscription, Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 193
at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription.CloneSubscription(Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 273
at Opc.Ua.Client.Session..ctor(ITransportChannel channel, Session template, Boolean copyEventHandlers)
at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSession..ctor(OpcUaSession session, ITransportChannel channel, Session template, Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSession.cs:line 118
at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSession.CloneSession(ITransportChannel channel, Boolean copyEventHandlers) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSession.cs:line 174
at Opc.Ua.Client.Session.RecreateAsync(Session sessionTemplate, CancellationToken ct)
at Opc.Ua.Client.Session.RecreateAsync(Session sessionTemplate, ITransportChannel transportChannel, CancellationToken ct)
at Opc.Ua.Client.DefaultSessionFactory.RecreateAsync(ISession sessionTemplate, ITransportChannel transportChannel, CancellationToken ct)
at Opc.Ua.Client.SessionReconnectHandler.DoReconnectAsync()
[25-01-29 12:09:02.7795] info: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription[0]
#2/30: Subscription 1:1:d003442a44b5bfcf4269445ebf2f8b2d942d5426[P0@0] is missing keep alive.
[25-01-29 12:09:07.7242] fail: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaStack[0]
Error while raising PublishStateChanged event.
System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription'.
at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription.OnPublishStatusChange(Subscription subscription, PublishStateChangedEventArgs e) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 2427
at Opc.Ua.Client.Subscription.HandleOnKeepAliveStopped()
[25-01-29 12:09:07.7243] fail: Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaStack[0]
Unexpected error sending publish request.
Opc.Ua.ServiceResultException: Channel has been closed.
at Opc.Ua.Client.Session.BeginPublish(Int32 timeout)

[alkozlov]

We have faced with the same issue. Guys do you have any progress with fix?

[mregen]

looks like the issue is here access on a disposed object in the callback when a subscription is recreated:

Could not reconnect the Session. System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription'.
at Azure.IIoT.OpcUa.Publisher.Stack.Services.OpcUaSubscription.OnStateChange(Subscription subscription, SubscriptionStateChangedEventArgs e) in /__w/1/s/Industrial-IoT/src/Azure.IIoT.OpcUa.Publisher/src/Stack/Services/OpcUaSubscription.cs:line 2481

[marcschier]

#2375