Skip to content

rm: Remove "pull queues" from manual #930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
doanac wants to merge 1 commit into from
Closed

rm: Remove "pull queues" from manual #930

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 9 additions & 38 deletions source/reference-manual/factory/event-queues.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,47 +16,21 @@ These include:
* When an OTA update starts and completes.

Event queues are implemented using Google PubSub_ to provide a well understood and tested framework.
There are two types of event queues:

* Push: Works as a webhook_ service.
Events are sent to a managed URL where they can be processed.

* Pull: Works like a typical message queue system where one can write their own client to receive and process events.

The PubSub documentation includes a very useful guide_ for deciding which approach will work best for you.
They also include a wide range of `client libraries`_ for consuming the Pull API.
FoundriesFactory leverages "push subscriptions" as the delivery mechanism for events.
PubSub subscriptions are created with default retention, expiration, and acknowledgement values_.

Implementation Details
----------------------

Each FoundriesFactory is given a single PubSub Topic.
Each Push and Pull queue created by a customer results in the creation of a PubSub subscription.
Each Topic results in the creation of a PubSub "push" subscription.
The FoundriesFactory API provides a thin, multi-tenant friendly wrapper to manage everything.

.. note::
For performance reasons, new push queues can take up to five minutes before they start receiving events.

Creating a Pull Queue
---------------------

A pull queue can be created using Fioctl:

.. code-block:: console

$ fioctl event-queues mk-pull <name> <where to save creds-file>
$ fioctl event-queues mk-pull docs-example $HOME/.fio-pull-queue.creds

Fioctl can also monitor this queue:
For performance reasons, new queues can take up to five minutes before they start receiving events.

.. code-block:: console

$ fioctl event-queues listen docs-example $HOME/.fio-pull-queue.creds

This command also serves as a reference example_ on implementing a pull queue listener.

Creating a Push Queue
---------------------
Creating a Queue
----------------

A push queue requires a little up front work:

Expand Down Expand Up @@ -91,8 +65,8 @@ Once the server is running, you can create a push queue with:

At this point events will start showing up in the example server.

Push Queue Payloads
~~~~~~~~~~~~~~~~~~~
Event Queue Payloads
~~~~~~~~~~~~~~~~~~~~

Incoming HTTP requests will look similar to:

Expand All @@ -112,8 +86,8 @@ Incoming HTTP requests will look similar to:
"subscription":"projects/osf-prod/subscriptions/xxxxxxxx"
}

Push Queue Security
~~~~~~~~~~~~~~~~~~~
Event Queue Security
~~~~~~~~~~~~~~~~~~~~

Incoming requests will include a header, ``Authorization: Bearer <jwt>``.
This JWT is signed with one of Google's own private keys.
Expand Down Expand Up @@ -237,9 +211,6 @@ DEVICE_PUBKEY_CHANGE
.. _PubSub:
https://cloud.google.com/pubsub/docs/overview

.. _webhook:
https://en.wikipedia.org/wiki/Webhook

.. _guide:
https://cloud.google.com/pubsub/docs/subscriber

Expand Down
Loading