Skip to content

Commit 6faf68c

Browse files
ndevenishndevenish
committed
Add example RabbitMQ configuration and user files
This is (at time of commit) the current rabbitmq configuration being applied at Diamond, and an example of the folder to use for user generation. Addresses request in #181.
1 parent 22fcd05 commit 6faf68c

File tree

4 files changed

+178
-1
lines changed

4 files changed

+178
-1
lines changed

‎HISTORY.rst

+2-1
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@ History
44

55
Unreleased
66
----------
7-
* ``zocalo.configure_rabbitmq``: Show explicit error when VHost not created.
7+
* Add example rabbitmq configuration and user files to ``contrib/`` (`#245 <https://github.com/DiamondLightSource/python-zocalo/pull/245>`_)
8+
* ``zocalo.configure_rabbitmq``: Show explicit error when VHost not created. (`#241 <https://github.com/DiamondLightSource/python-zocalo/pull/241>`_)
89

910
0.30.2 (2023-09-06)
1011
-------------------

‎contrib/rabbitmq-configuration.yaml

+164
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
---
2+
# RabbitMQ Configuration
3+
#
4+
# This is a snapshot of the active rabbitmq configuration used for the
5+
# zocalo deployment at Diamond.
6+
#
7+
# This file is used to declare RabbitMQ queues and their configuration.
8+
# Jenkins parses this file using the Python script in this directory to
9+
# generate and deploy the actual RabbitMQ configuration.
10+
#
11+
# Queues are grouped together in groups, within a group every queue has
12+
# the same settings. It is possible to optionally influence the RabbitMQ
13+
# queue, exchange, and bindings settings per group.
14+
15+
# Note that no VHost is declared here - at time of writing, this is not
16+
# handled very gracefully and changes are applied by deleting the VHost;
17+
# which causes all queues, outstanding messages to be deleted.
18+
#
19+
# If you understand these risks and want to specify the vhost in this
20+
# file, you can specify vhost as:
21+
# # vhosts:
22+
# # - zocalo
23+
24+
25+
exchanges:
26+
- name: delayed
27+
vhost: zocalo
28+
type: x-delayed-message
29+
arguments:
30+
x-delayed-type: direct
31+
- name: results
32+
vhost: zocalo
33+
type: topic
34+
35+
policies:
36+
- name: redelivery
37+
definition:
38+
delivery-limit: 5
39+
vhost: zocalo
40+
pattern: .*
41+
42+
- name: ttl
43+
definition:
44+
delivery-limit: 5
45+
message-ttl: 60000
46+
vhost: zocalo
47+
priority: 1
48+
pattern: ^xrc.*
49+
50+
- name: immediate
51+
definition:
52+
# delivery-limit: 5
53+
message-ttl: 0
54+
vhost: zocalo
55+
priority: 1
56+
pattern: per_image_analysis.ssx
57+
58+
- name: dlq
59+
definition:
60+
dead-letter-exchange: ''
61+
vhost: zocalo
62+
63+
groups:
64+
65+
# No-prefetch queues
66+
# These are the workhorse queues of Zocalo. Any listener will only get a
67+
# single message from these queues, and only get the next message after
68+
# acknowledging/rejecting the previous one.
69+
- names:
70+
- archive.filelist
71+
- archive.pattern
72+
- cluster.submission
73+
- darc.dropzone
74+
- dispatcher
75+
- filewatcher
76+
- htcondorwatcher
77+
- images
78+
- index
79+
- ispyb_connector
80+
- ispyb_pia
81+
- mailnotification
82+
- mimas
83+
- mimas.held
84+
- nexusparser.find_related_files
85+
- notify_gda
86+
- per_image_analysis
87+
- per_image_analysis.hdf5_select
88+
- processing_recipe
89+
- pymca.fitter
90+
- relion.dev.stop
91+
- ssx.plot
92+
- transient.destination
93+
- transient.transaction
94+
# - transient.system_test.{guid}
95+
- trigger
96+
- validation
97+
- bridge.test
98+
99+
settings:
100+
queues:
101+
type: quorum
102+
dead-letter-routing-key-pattern: dlq.{name}
103+
bindings:
104+
- ''
105+
- delayed
106+
vhost: zocalo
107+
108+
- names:
109+
- per_image_analysis.ssx
110+
111+
settings:
112+
queues:
113+
type: quorum
114+
# dead-letter-routing-key-pattern: per_image_analysis
115+
# dead-letter-queue-create: false
116+
bindings:
117+
- ''
118+
vhost: zocalo
119+
120+
# Single active consumer queues
121+
# Every queue in this group only allows a single listener to read from the
122+
# queue at any one time. That listener gets all the messages from the queue
123+
# at once.
124+
# https://www.rabbitmq.com/consumers.html#single-active-consumer
125+
- names:
126+
- reduce.dc_sim
127+
- reduce.xray_centering
128+
- statistics.cluster
129+
- jsonlines
130+
settings:
131+
queues:
132+
type: quorum
133+
dead-letter-routing-key-pattern: dlq.{name}
134+
single_active_consumer: true
135+
bindings:
136+
- ''
137+
- delayed
138+
vhost: zocalo
139+
140+
# Broadcast topics
141+
# This is TBC. I suspect we can do something cleverer here, eg. have
142+
# temporary queues tied to each connection or something like that.
143+
- names:
144+
- command
145+
- transient.status
146+
- transient.statistics.cluster
147+
settings:
148+
broadcast: true
149+
queues:
150+
type: quorum
151+
vhost: zocalo
152+
153+
- names:
154+
- xrc.i03
155+
settings:
156+
queues:
157+
type: quorum
158+
dead-letter-queue-create: false
159+
vhost: zocalo
160+
161+
bindings:
162+
- source: results
163+
destination: xrc.i03
164+
vhost: zocalo

‎contrib/rabbitmq_users/README.md

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# RabbitMQ Users Specification
2+
3+
The ini-files in this folder are examples of the folder used by the
4+
`zocalo.configure_rabbitmq` tool. Use as:
5+
6+
```
7+
zocalo.configure_rabbitmq rabbitmq-configuration.yaml --user-config <THIS FOLDER>
8+
```

‎contrib/rabbitmq_users/custom-user.ini

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
[rabbitmq]
2+
username = custom
3+
password = custompass
4+
tags = administrator

0 commit comments

Comments
 (0)