Feature request: support Observables

[filchristou]

Would be nice if we would have observables that track the remote value and are automatically updated when the value on the remote is changed. Let me know if that sounds like something this repo can accommodate.

[c42f]

A version of this could be made to work and would be very cool.

What would the API be? How about having y = @remote x "just work" when x happens to be an observable on the remote side?

Here's how that can work.

• We add a listener to x on the remote side to send the updates down the RemoteREPL socket
• We create a new Observable y on the local side, and register it with the code which manages the RemoteREPL socket so that updates translate to y[] = new_x_val

The big under-the-hood change needed would be to update the RemoteREPL protocol to give each message an ID and allow out-of-order delivery of messages between client and server. That way we can safely interleave delivery of Observable updates with normal RemoteREPL "REPL traffic", and deliver each message to the correct consumer on the client side.

This is a good change in general because having message IDs and out of order delivery would also greatly help with delivering log messages back to the client in real time. And would probably allow more graceful recovery from disconnected sockets as well.

[filchristou]

How about having y = @remote x "just work" when x happens to be an observable on the remote side?

I guess that could work with some multiple dispatch that expects an Observable and it's also very close to the current design. But there is a corner case. What if the user really just wants to take the Observable x from the remote and not create a new y that tracks it ? How would they do it ?

Also, do you want Observables.jl to be a direct dependency or have as an extension. I think I would prefer not using extensions since Observables.jl has almost no dependencies itself.

update the RemoteREPL protocol to give each message an ID and allow out-of-order delivery of messages between client and server.

How do you imagine that? Like a counter to tell apart which messages are old and which are new? Also why do we need a message ID? We already have a session ID. What would we win to give each message an ID? And if we really need one, maybe we can actually use some sort of a hash of the message content ?

[c42f]

What if the user really just wants to take the Observable x from the remote and not create a new y that tracks it ?

I guess they can't.

But I'm not sure this really works "as expected" anyway? What happens to the listeners on the remote side when the observable is serialized? Are the listeners serialized too? What does it mean to call setindex! on the local copy of that observable? Clearly the listeners on the remote side won't receive any update.

Also, do you want Observables.jl to be a direct dependency

If Observables.jl is low-dependency that seems fine.

How do you imagine that? Like a counter to tell apart which messages are old and which are new? Also why do we need a message ID?

Something like the client generates a sequential ID for a request, and the server sends that ID back to the client in the reply? So not really a message ID I guess, but some kind of transaction ID to mark the request-response cycle.

Why would we need a transaction ID? The point is if you have an Observable on the remote side a setindex! update could be triggered at any time and the server will send that to the client whenever it feels like it. But this can happen concurrently with the user typing something in the REPL.

So the client may be expecting the next message from the server to contain the reply to the REPL command, but the client may get an observable update message instead. The transaction ID allows the client to determine which reply is which.