graphql subscriptions

[predmond]

How can I support subscriptions using graphene?

[globophobe]

That would depend on what kind of subscription, for example long-polling or WebSocket. It also depend on your backend, for example Django, with django channels, could handle subscriptions by broadcasting to connected sockets from a post_save signal.

[jameswyse]

The new databinding features in django-channels look ideal for graphql subscriptions: https://channels.readthedocs.io/en/stable/binding.html

I'd love to see an example of subscriptions in graphene using channels (or any other method) :)

[Jufik]

I'd guess it's pretty straight forward to code a model related subscription.
I'm working with Django, and plan to use Django Channel: my suggestions and answers are opinionated that way.

Getting inspired by mutation from sources, one could push down the data to customer quite easily.
In case of Mutation, mutate is called is the resolver. I guess a general subscribe function could be written to "hookup" the customer channel on an "subscription specific channel", it self hooked up on the "model channel".
Using a post_save, writting down a custom save method or binding could do the Job to propagate the data down to clients.

I don't know how to solve two stuff though:

• how to get the information of which channel is expecting which "query" (eg which fields should be sent to which client).
• parameters handling. Like how to handle a subscription to a model, based on a category filter.

For the first point, I guess I could use caching engine and a 3 layer structure to push data to my channel. Like: Customer Channel hooked up to a Query Channel, that his hooked up to the model channel. The query used in the Query Channel being store in cache. It seems non trivial bu feasible.

For the second point, only a cache based solution seems viable, storing filters in a dict, stored in cache. But that's lot of work.

Any suggestion on those two questions is welcome!

I'll give a basic architecture a try, and create share repo' to give some feedback.

[jeffreybrowning]

@Jufik Any updates?

[hballard]

I implemented a port of the apollo graphql subscriptions modules (graphql-subscriptions and subscriptions-transport-ws) for graphene / python. They work w/ apollo-client.

It is here.

Same basic api. It is still very rough...but works so far, based on my limited testing. Uses redis-py, gevent-websockets, and syrusakbary/promises. It is still missing a few of the server options and keep-alive messages, but those are pretty easy to implement and I'll do that in the next few days. I was going to add a simple example app too. Some of it is below. Only works on python2 for now and it needs tests, setup.py, etc. I figured I'd go ahead and share in case anybody is interested...

Simple example:

Server (using Flask and Flask-Sockets):

from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_sockets import Sockets

from .subscription_manager import SubscriptionManager, RedisPubsub
from .subscription_transport_ws import ApolloSubscriptionServer

app = Flask(__name__)
sockets = Sockets(app)
pubsub = RedisPubsub()
schema = graphene.Schema(
    query=Query,
    mutation=Mutation,
    subscription=Subscription
)
subscription_mgr = SubscriptionManager(schema, pubsub)

@sockets.route('/socket')
def socket_channel(websocket):
    subscription_server = ApolloSubscriptionServer(subscription_mgr, websocket)
    subscription_server.handle()
    return []

if __name__ == "__main__":
    from geventwebsocket import WebSocketServer

    server = WebSocketServer(('', 5000), app)
    print '  Serving at host 0.0.0.0:5000...\n'
    server.serve_forever()

Of course on the server you have to "publish" each time you have a mutation (in this case to a redis channel). That could look something like this (using graphene / sql-alchemy):

class Subscription(graphene.ObjectType):
    users = graphene_sqlalchemy.SQLAlchemyConnectionField(
        User,
        active=graphene.Boolean()
    )

    def resolve_users(self, args, context, info):
        query = User.get_query(context)
        return query.filter_by(id=info.root_value.get('id'))

class AddUser(graphene.ClientIDMutation):

    class Input:
        username = graphene.String(required=True)
        email = graphene.String()

    ok = graphene.Boolean()
    user = graphene.Field(lambda: User)

    @classmethod
    def mutate_and_get_payload(cls, args, context, info):
        _input = args.copy()
        del _input['clientMutationId']
        new_user = UserModel(**_input)
        db.session.add(new_user)
        db.session.commit()
        ok = True
        if pubsub.subscriptions:
            pubsub.publish('users', new_user.as_dict())
        return AddUser(ok=ok, user=new_user)

Client (using react-apollo client):

import React from 'react'
import ReactDOM from 'react-dom'
import { graphql, ApolloProvider } from 'react-apollo'
import gql from 'graphql-tag'
import ApolloClient, { createNetworkInterface } from 'apollo-client'
import { SubscriptionClient, addGraphQLSubscriptions } from 'subscriptions-transport-ws'

import ChatApp from './screens/ChatApp'
import ListBox from '../components/ListBox'

const SUBSCRIPTION_QUERY = gql`
  subscription newUsers {
    users(active: true) {
      edges {
        node {
          id
          username
        }
      }
    }
  }
`

const LIST_BOX_QUERY = gql`
  query AllUsers {
    users(active: true) {
      edges {
        node {
          id
          username
        }
      }
    }
  }
`

class ChatListBox extends React.Component {

  componentWillReceiveProps(newProps) {
    if (!newProps.data.loading) {
      if (this.subscription) {
        return
      }
      this.subscription = newProps.data.subscribeToMore({
        document: SUBSCRIPTION_QUERY,
        updateQuery: (previousResult, {subscriptionData}) => {
          const newUser = subscriptionData.data.users.edges
          const newResult = {
            users: {
              edges: [
                ...previousResult.users.edges,
                ...newUser
              ]
            }
          }
          return newResult
        },
        onError: (err) => console.error(err)
      })
    }
  }

  render() {
    return <ListBox data={this.props.data} />
  }
}

const ChatListBoxWithData = graphql(LIST_BOX_QUERY)(ChatListBox)

export default ChatListBoxWithData

const networkInterface = createNetworkInterface({
  uri: 'http://localhost:5000/graphql'
})

const wsClient = new SubscriptionClient(`ws://localhost:5000/socket`, {
  reconnect: true
})

const networkInterfaceWithSubscriptions = addGraphQLSubscriptions(
  networkInterface,
  wsClient,
)

const client = new ApolloClient({
  dataIdFromObject: o => o.id,
  networkInterface: networkInterfaceWithSubscriptions
})

ReactDOM.render(
  <ApolloProvider client={client}>
    <ChatApp />
  </ApolloProvider>,
  document.getElementById('root')
)

I'm very new to open source, so any critiques or pull requests are welcome.

[dfee]

@hballard checking it out. But wow, this was one of the major things holding me back.

[hballard]

Cool. Let me know your thoughts. I just pushed an update that added the keep_alive messages and the on_connect, on_disconnect, on_subscribe, and on_unsubscribe optional server setup functions. This should make it mostly the same as the Apollo subscriptions API at this point.

[japrogramer]

@globophobe django channels cannot do broadcasting with ws
https://channels.readthedocs.io/en/stable/concepts.html#groups

[globophobe]

@japrogramer You're right. There's an extra step. The client would need to be added to a group. I don't have time (nor ever will) to explore this. Personally waiting for, https://github.com/encode/uvicorn and https://github.com/encode/apistar to bring good things to Django, or obviate my need for it.

[japrogramer]

For anyone interested this is how i implemented subscriptions with channels.
#500 (comment)

[miketout]

I just collaborated with @IlyaRadinsky on a graphene/Tornado/subscriptions integration with websockets that requires one line of code to get your graphene schema + subscriptions up and available as an API. Here's a link to his original repo, which has incorporated my changes as well:
https://github.com/IlyaRadinsky/tornadoql

[japrogramer]

@miketout awesome, lots of interesting techniques and code there, will definitely be taking a look.

[dfee]

I'm about to start digging into subscription support probably tomorrow, but I'm looking at the code here (graphql-python test code)

[dfee]

Here's a very simple gist of how to use subscriptions with graphene.

In a nutshell, it's probably easiest to return an rx.Observable using the pattern in the implementation above, though returning an AsyncIterator is also allowed. rx is a dependency of graphene anyway, meaning that it's not additional requirement you'll need to install, and it works on py2 and py3.

As you're probably using this for a web view, you'll actually want to save that subscriber and store it in a dict so a user can unsubscribe from it in the future. And, you'll probably also want to use websockets, or socket.io, etc. so you can actually push the subscription data out instead of appending it to a list as I've done on that same line.