Skip to content

Commit

Permalink
Remove USubscription CreateTopic() and DeprecateTopic() APIs (#108)
Browse files Browse the repository at this point in the history
* Remove USubscription CreateTopic() and DeprecateTopic() APIs

The topic lifecycle APIs were added as an optimization early on so that subscribers could know if the topic they are subscribing to is valid and the producer is alive however this optimization has given us nothing but heartburn in production as it forces all publishers to call the API when then are initialized and it causes race conditions on initial boot-up. Furthermore, validation of topics could already be done by calling UDiscovery so there was no added benefit of adding the extra states to the subscription service.

#96

* Apply suggestions from code review

Co-authored-by: Kai Hudalla <[email protected]>

---------

Co-authored-by: Kai Hudalla <[email protected]>
  • Loading branch information
Steven Hartley and sophokles73 authored Apr 9, 2024
1 parent ca3e8a3 commit 2ff303c
Show file tree
Hide file tree
Showing 8 changed files with 48 additions and 431 deletions.
26 changes: 0 additions & 26 deletions up-core-api/uprotocol/core/usubscription/v3/usubscription.proto
Original file line number Diff line number Diff line change
Expand Up @@ -75,19 +75,6 @@ service uSubscription {
}


// API called by producers to register a topic. This API
// informs the Subscription Service that to create the topic and it is ready to publish.
rpc CreateTopic(CreateTopicRequest) returns (uprotocol.v1.UStatus) {
option (method_id) = 4;
}

// Request deprecation of a topic. Producers call this to inform the uSubscription
// that it will no longer produce to said topic. The topic is flagged as deprcated
// which
rpc DeprecateTopic(DeprecateTopicRequest) returns (uprotocol.v1.UStatus) {
option (method_id) = 5;
}

// Register to receive subscription change notifications that are published on the
// 'up:/core.usubscription/3/subscriptions#Update'
rpc RegisterForNotifications(NotificationsRequest) returns (uprotocol.v1.UStatus) {
Expand Down Expand Up @@ -315,19 +302,6 @@ message NotificationsRequest {
}


// Message passed to CreateTopic()
message CreateTopicRequest {
// uProtocol Topic
v1.UUri topic = 1;
}


// Message passed to DeprecateTopic()
message DeprecateTopicRequest {
// uProtocol Topic
v1.UUri topic = 1;
}


// Subscription Update Message.
// This Update message is sent from uSusbcription on the topic:
Expand Down
110 changes: 15 additions & 95 deletions up-l3/usubscription/v3/README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -30,26 +30,26 @@ SPDX-License-Identifier: Apache-2.0
----


Publisher & Subscribers Architecture Pattern allows subscribers to subscribe to topics published by publisher that can be running on the same device and/or different devices (that might not even maintain a persistent connection to each other).
Publisher & Subscribers Architecture pattern allows subscribers to subscribe to topics published by publisher that can be running on the same device and/or different devices who's connection to each other could even be temporal.

Off the shelf broker and broker-less solutions, rely on specific Internet protocols that do not scale for automotive use cases nor do they support producers and consumers that talk different pub/sub protocols (ex. MQTT, DDS, SOME/IP, proprietary, etc...).

uSubscriptions purpose is to explain _how_ the distributed publisher-subscriber architecture pattern can be implemented through a ubiquitous language and communication protocol across the distributed devices. We will define the interfaces, sequences, and state machines to be able to support the use cases mentioned above.


NOTE: uSubscription interface is defined link:../../../up-core-api/uprotocol/core/usubscription/v3/usubscription.proto[usubscription.proto] and follows link:../../../basics/error_model.adoc[uProtocol Error Model] for return codes.
NOTE: uSubscription interface is declared in link:../../../up-core-api/uprotocol/core/usubscription/v3/usubscription.proto[usubscription.proto] and follows link:../../../basics/error_model.adoc[uProtocol Error Model] for return codes.


== Topics

A topic describes what the calling applications (subscribers) wants to subscribe too. Topic are expressed in link:../../../basics/README.adoc#_uprotocol_uri[uProtocol URIs] format. Topics used in uSubscription APIs have the additional requirements of:

* *MUST* contain `UE_NAME`, `UE_VERSION`, `RESOURCE`, and `MESSAGE`
* *MUST* contain `UEntity` and `UResource` information
* `UE_VERSION` *MUST* contain the `MAJOR`, *MUST NOT* contain the `MINOR` or `PATCH`

NOTE: Only the `MAJOR` is used in topics to support backwards compatibility between minor version updates of uEs to avoid the need to re-subscribe to topics when the uE `MAJOR` was not updated. Subscribers *MUST* re-subscribe to topics when the uE `MAJOR` version has changed

Example: `up:/body.access/1/door.front_left#Door`
Example: `/body.access/1/door.front_left#Door`


== Subscription States
Expand All @@ -63,17 +63,9 @@ NOTE: `SUBSCRIBE_PENDING` and `UNSUBSCRIBE_PENDING` states only apply to remote
|===
|State |Description |Entry |Do |Exit

|`*INVALID*`
|Topic does not exist because the producer has not called CreateTopic() yet.
|Initialization
a|* Active subscriptions are removed
* Notify any observers when subscription has changed
|CreateTopic() called by producer

| `*UNSUBSCRIBED*`
|Subscriber uE is not subscribed but the topic exists
a|* Topic created
* Subscriber unsubscribed
a|* Subscriber unsubscribed
|
|Subscribe() is called by a consumer

Expand All @@ -93,45 +85,17 @@ a|* Topic created
|Unsubscribe is pending processing by the producers remote uBus
|Last subscriber called Unsubscribe()
a|* Send an unsubscribe request to the remote uBus
* return https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto[google.rpc.Status] with:
** https://github.com/googleapis/googleapis/blob/master/google/rpc/code.proto[google.rpc.Code]: OK
** SubscriptionStatus
* return link:../../../../up-core-api/uprotocol/ustatus.proto[UStatus] with `UCode.OK` and `SubscriptionStatus`
|
|===

.Subscription State Machine
image::subscription_sm.drawio.svg[State Machine]


=== Topic Lifecycle

.Topic State Machine
[width="100%",cols="24%,76%",options="header",]
|===
|State |Description

|`*INVALID*` |Topic does not exist because the producer has not called CreateTopic() yet
|`*ACTIVE*` |Producer has called CreateTopic() and uSubscription can accept subscriptions to said topic
|`*DEPRECATED*` a|
Producer has tagged the topic as deprecated so uSubscription

* Subscription requests to deprecated topics *MUST* return `FAILED_PRECONDITION`

|===

NOTE: uSubscription will persist the known topic state (`ACTIVE`, or `DEPRECATED`)

.Topic State Machine
image::topic_sm.drawio.svg[Topic State Machine]

Changing the topic state happens through the APIs CreateTopic() and DeprecateTopic(). Below are the requirements for the different topic related APIs:

* *MUST* be a valid topic URI (containing UE_NAME, UE_VERSION, RESOURCE, and MESSAGE)
* *MUST* verify that only the producer of a topic can call `CreateTopic()` or `DeprecateTopic()`. Verification is done by ensuring the uE name and version inside the `SubscriberInfo` and topic URIs match

== Subscription Change Notifications

The uSubscription service notifies observers when there is a change in a subscription states by publishing notification events to the topic `up:/core.usubscription/3/subscriptions#Update`. Below are the specific nuances about the subscription change notification based on the observer type.
The uSubscription service notifies observers when there is a change in a subscription states by publishing notification events to the topic `/core.usubscription/3/subscriptions#Update`. Below are the specific nuances about the subscription change notification based on the observer type.

=== Subscribers

Expand All @@ -142,53 +106,26 @@ Subscribers are automatically registered to receive subscription change notifica

=== Publishers

* *MUST* *NOT* be permitted to call `RegisterForNotifications()` for topics they do not create through the `CreateTopic()`, only the topics they produce
* *MUST* *NOT* be permitted to call `RegisterForNotifications()` for topics they do not produce, i.e. the uEntity sending the request is not equal to the `UEntity` of the request message's `UUri` parameter.
* Subscription change notifications *MUST* be sent for changes to `SubscriptionState` for any subscriber that is subscribed to the topic

=== Dispatchers

Dispatchers are also permitted to register for subscription change notifications to facilitate the multicasting of events. The mechanics and requirements of the dispatcher and uSubscription communication are platform deployment specific.
Dispatchers are also permitted to register for subscription change notifications to facilitate the multicasting of messages. The mechanics and requirements of the dispatcher and uSubscription communication are platform deployment specific.

== Event Delivery

uSubscription service, in addition to managing subscriptions for subscribers to topics, also plays a key role in providing event delivery configuration details as not every platform delivers events using the same mechanism.

the `EventDeliveryConfig` message is one of the fields returned to the subscriber to provide details of how to consume the data (how events are delivered). For example if the subscriber has to consume from a different topic or from different event infrastructure, this message will store those delivery semantics.
The uSubscription service also allows to indicate to subscribers how they shall consume published events by means of the `EventDeliveryConfig` returned in the `SubscriptionResponse` message. For example, if the subscriber has to consume from a different topic or from different messaging infrastructure, this message will store the corresponding delivery semantics.

NOTE: Delivery semantics (if any) are deployment specific and not covered in this specification

== uSubscription Sequences

In the following section, we will elaborate on the various subscription flows for local and remote topics. When a consumer subscribes to a remote topic, it is the responsibility of the Subscription Service to relay the subscription request to the remote Subscription Service as can be illustrated in the sequence diagrams below.

For the flows below we will use a fictitious device called Device1 hosting a service called uExample that has a resource that emits a Message.
In the following section, we will elaborate on the various subscription flows for local and remote topics. When a consumer subscribes to a remote topic, it is the responsibility of the (local) uSubscription service to relay the subscription request to the remote uSubscription service as can be seen in the sequence diagrams below.

NOTE: Throughout this section we will use the sample topic `up://Device1/uexample/1/resource#Event` to illustrate the various sequences. The above-mentioned topic will be replaced with `_topic_` in the diagrams
NOTE: Throughout this section we will use the sample topic `//Device1/uexample/1/resource#Event` to illustrate the various sequences. The above-mentioned topic will be replaced with `_topic_` in the diagrams

uSubscription exposes APIs to broker communication between subscribers and producers. The service shall implement two design patterns that shall be described in the table below.

.uSubscription Design Pattern Use Cases
[width="100%",cols="76%,24%",options="header",]
|===
|Purpose |Pattern

|* uEs (subscribers and producers) to invoke uSubscription APIs service to perform operations such as `Subscribe()`, `Unsubscribe()`, `CreateTopic()` etc...
* For uSubscription to communicate with other uSubscription services running on other devices when subscribing to remote topics
|*RPC*

|* Informing subscribers of changes to their subscription. This allows subscribers to react if/when a subscription is not successful at a remote device or when a topic is no longer being served by the producer
* Informing registered observers of added or removed subscribers

|*Notification*
|===


=== Topic Creation

Topic creations, also known as the setup phase, is the first step in the subscription process and performed by the producer to notify uSubscription that it will produce to a given topic.

.Setup Flow
image::create_topic.svg[Create Topic Flow]

=== Subscription

Expand All @@ -204,7 +141,7 @@ image::local_subscribe.svg[Local Subscription Flow]
.Remote Subscription Flow
image::remote_subscription.svg[Remote Subscription Flow]

* uSubscription *MUST* change the subscriber to itself (core.usubscription) when subscribing to remote topics, this allows the reverse flow (publication) to be properly multicasted to local subscribers by the uBus when it queries the local uSubscription for a list of local subscribers
* uSubscription *MUST* change the subscriber to itself (core.usubscription) when subscribing to remote topics, this allows the reverse flow (publication) to be properly multicasted to local subscribers by the local disaptcher (ex. uBus) when it queries the local uSubscription for a list of local subscribers

=== Unsubscribe

Expand All @@ -220,28 +157,11 @@ image::unsub_remote.svg[Unsubscribe Remote Flow]

* uSubscription *MUST* change the subscriber to itself (core.usubscription) when unsubscribing to remote topics

=== Topic Deprecation

This API is used to tag a topic as deprecated. Deprecated topics can no longer be subscribed to by subscribers. The following are the uSubscription API requirements:

* *MUST* be changed the topic state to DEPRECATED
* *MUST* no longer allow subscribers to Subscribe() to said topic
* *MUST* not change the SubscriptionState for existing subscribers (i.e. no subscription change notification is sent)

==== Within a uDevice

.Local Topic Deprecation Flow
image::topic_deprecate.svg[Topic Deprecation]

==== Between uDevices

NOTE: Dissemination of the state of deprecated topics to remote uSubscription services that has subscribers to the topic is not covered in this version of the specification.

== Timeout & Retry Logic

Subscribe (and unsubscribe) to remote topics are handled by RPC calls between uSubscription services running on the different devices. Given that devices are not always connected to each other, the onus is on uSubscription service to ensure that a command is received in time. to ensure consistency between implementations of uSubscription, we will explain the retry and timeout policies for remote command invocations in the sequence flows below. command is on the uSubscription service.
Subscribe (and unsubscribe) to remote topics are handled by RPC calls between uSubscription services running on the different devices. Given that devices are not always connected to each other, the onus is on uSubscription service to ensure that a command is received in time. Below are the common retry and timeout policies for USubscription service implementations to follow:

* Remote requests *MUST* have a maximum timeout of 5 minutes
* All timed-out remote commands *MUST* be retied indefinitely until the business logic behind it no longer requires the command to be sent. +
Example: Remote Subscribe() request will be retied until the subscriber no longer wishes to be subscribed (they call the Unsubscribe() API)
* All timed-out remote commands *MUST* be retied indefinitely until the business logic behind it no longer requires the command to be sent. (ex. the subscriber cals `Unsubscribe()` )
* Remote commands *MUST* be retried upon device to device connectivity (link up) and *MUST NOT* be tried when there is no device connectivity (link down)
44 changes: 0 additions & 44 deletions up-l3/usubscription/v3/create_topic.puml

This file was deleted.

Loading

0 comments on commit 2ff303c

Please sign in to comment.