Skip to content

Commit ea8fd44

Browse files
ndevenishndevenish
authored
Add example RabbitMQ configuration and user files (#245)
This is (at time of commit) the current rabbitmq configuration being applied at Diamond, and an example of the folder to use for user generation. Addresses request in #181.
1 parent 82912eb commit ea8fd44

File tree

4 files changed

+178
-1
lines changed

4 files changed

+178
-1
lines changed

‎HISTORY.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@ History
44

55
Unreleased
66
----------
7-
- Switch python packaging backend to poetry.
7+
* Add example rabbitmq configuration and user files to ``contrib/`` (`#245 <https://github.com/DiamondLightSource/python-zocalo/pull/245>`_)
8+
* Switch python packaging backend to poetry.
89

910
0.32.0 (2024-04-05)
1011
-------------------

‎contrib/rabbitmq-configuration.yaml

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
---
2+
# RabbitMQ Configuration
3+
#
4+
# This is a snapshot of the active rabbitmq configuration used for the
5+
# zocalo deployment at Diamond.
6+
#
7+
# This file is used to declare RabbitMQ queues and their configuration.
8+
# Jenkins parses this file using the Python script in this directory to
9+
# generate and deploy the actual RabbitMQ configuration.
10+
#
11+
# Queues are grouped together in groups, within a group every queue has
12+
# the same settings. It is possible to optionally influence the RabbitMQ
13+
# queue, exchange, and bindings settings per group.
14+
15+
# Note that no VHost is declared here - at time of writing, this is not
16+
# handled very gracefully and changes are applied by deleting the VHost;
17+
# which causes all queues, outstanding messages to be deleted.
18+
#
19+
# If you understand these risks and want to specify the vhost in this
20+
# file, you can specify vhost as:
21+
# # vhosts:
22+
# # - zocalo
23+
24+
25+
exchanges:
26+
- name: delayed
27+
vhost: zocalo
28+
type: x-delayed-message
29+
arguments:
30+
x-delayed-type: direct
31+
- name: results
32+
vhost: zocalo
33+
type: topic
34+
35+
policies:
36+
- name: redelivery
37+
definition:
38+
delivery-limit: 5
39+
vhost: zocalo
40+
pattern: .*
41+
42+
- name: ttl
43+
definition:
44+
delivery-limit: 5
45+
message-ttl: 60000
46+
vhost: zocalo
47+
priority: 1
48+
pattern: ^xrc.*
49+
50+
- name: immediate
51+
definition:
52+
# delivery-limit: 5
53+
message-ttl: 0
54+
vhost: zocalo
55+
priority: 1
56+
pattern: per_image_analysis.ssx
57+
58+
- name: dlq
59+
definition:
60+
dead-letter-exchange: ''
61+
vhost: zocalo
62+
63+
groups:
64+
65+
# No-prefetch queues
66+
# These are the workhorse queues of Zocalo. Any listener will only get a
67+
# single message from these queues, and only get the next message after
68+
# acknowledging/rejecting the previous one.
69+
- names:
70+
- archive.filelist
71+
- archive.pattern
72+
- cluster.submission
73+
- darc.dropzone
74+
- dispatcher
75+
- filewatcher
76+
- htcondorwatcher
77+
- images
78+
- index
79+
- ispyb_connector
80+
- ispyb_pia
81+
- mailnotification
82+
- mimas
83+
- mimas.held
84+
- nexusparser.find_related_files
85+
- notify_gda
86+
- per_image_analysis
87+
- per_image_analysis.hdf5_select
88+
- processing_recipe
89+
- pymca.fitter
90+
- relion.dev.stop
91+
- ssx.plot
92+
- transient.destination
93+
- transient.transaction
94+
# - transient.system_test.{guid}
95+
- trigger
96+
- validation
97+
- bridge.test
98+
99+
settings:
100+
queues:
101+
type: quorum
102+
dead-letter-routing-key-pattern: dlq.{name}
103+
bindings:
104+
- ''
105+
- delayed
106+
vhost: zocalo
107+
108+
- names:
109+
- per_image_analysis.ssx
110+
111+
settings:
112+
queues:
113+
type: quorum
114+
# dead-letter-routing-key-pattern: per_image_analysis
115+
# dead-letter-queue-create: false
116+
bindings:
117+
- ''
118+
vhost: zocalo
119+
120+
# Single active consumer queues
121+
# Every queue in this group only allows a single listener to read from the
122+
# queue at any one time. That listener gets all the messages from the queue
123+
# at once.
124+
# https://www.rabbitmq.com/consumers.html#single-active-consumer
125+
- names:
126+
- reduce.dc_sim
127+
- reduce.xray_centering
128+
- statistics.cluster
129+
- jsonlines
130+
settings:
131+
queues:
132+
type: quorum
133+
dead-letter-routing-key-pattern: dlq.{name}
134+
single_active_consumer: true
135+
bindings:
136+
- ''
137+
- delayed
138+
vhost: zocalo
139+
140+
# Broadcast topics
141+
# This is TBC. I suspect we can do something cleverer here, eg. have
142+
# temporary queues tied to each connection or something like that.
143+
- names:
144+
- command
145+
- transient.status
146+
- transient.statistics.cluster
147+
settings:
148+
broadcast: true
149+
queues:
150+
type: quorum
151+
vhost: zocalo
152+
153+
- names:
154+
- xrc.i03
155+
settings:
156+
queues:
157+
type: quorum
158+
dead-letter-queue-create: false
159+
vhost: zocalo
160+
161+
bindings:
162+
- source: results
163+
destination: xrc.i03
164+
vhost: zocalo

‎contrib/rabbitmq_users/README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# RabbitMQ Users Specification
2+
3+
The ini-files in this folder are examples of the folder used by the
4+
`zocalo.configure_rabbitmq` tool. Use as:
5+
6+
```
7+
zocalo.configure_rabbitmq rabbitmq-configuration.yaml --user-config <THIS FOLDER>
8+
```

‎contrib/rabbitmq_users/custom-user.ini

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
[rabbitmq]
2+
username = custom
3+
password = custompass
4+
tags = administrator

0 commit comments

Comments
 (0)