From bb5918b1fbcf40f7afdf1dafcb7daebe735b8d69 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Haris=20Osmanagi=C4=87?= Date: Thu, 10 Mar 2022 16:03:08 +0100 Subject: [PATCH 1/3] Update documentation about mode --- README.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index f3b2b90..5cfa779 100644 --- a/README.md +++ b/README.md @@ -76,13 +76,12 @@ column names for the configured table and set them in memory. ## Configuration Options | name | description | required | default | -| ---------------- | ------------------------------------------------------------------------------------------------------------------------------- | -------------------- | -------------------- | +|------------------|---------------------------------------------------------------------------------------------------------------------------------|----------------------|----------------------| | table | the name of the table in Postgres that the connector should read | yes | n/a | | url | formatted connection string to the database. | yes | n/a | | columns | comma separated string list of column names that should be built in to each Record's payload. | no | `*` (all columns) | | key | column name that records should use for their `Key` fields. defaults to the column's primary key if nothing is specified | no | primary key of table | -| snapshot | whether or not the plugin will take a snapshot of the entire table acquiring a read level lock before starting cdc mode | n/a | enabled | -| cdc | enables CDC features | req. for CDC mode | off | +| mode | the connector's operation mode. Available modes: `cdc`, `snapshot` | no | `cdc` | | publication_name | name of the publication to listen for WAL events | req. for CDC mode | `pglogrepl` | | slot_name | name of the slot opened for replication events | req. for CDC mode | `pglogrepl_demo` | | replication_url | URL for the CDC connection to use. If no replication_url is provided, then the CDC connection attempts the use the `url` value. | optional in CDC mode | n/a | From db83402a449119434275f7d03efc31cc2e632f72 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Haris=20Osmanagi=C4=87?= Date: Thu, 10 Mar 2022 16:07:23 +0100 Subject: [PATCH 2/3] remove references to replication_url --- README.md | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 5cfa779..0285d74 100644 --- a/README.md +++ b/README.md @@ -41,19 +41,17 @@ Publication and slot name are user configurable, and must be correctly set. The plugin will do what it can to be smart about publication and slot management, but it can't handle everything. -If a `replication_url` is provided, it will be used for CDC features instead of -the url. If no `replication_url` is provided, but cdc is enabled, then it will -attempt to use that url value for CDC features and logical replication setup. Example configuration for CDC features: -``` -"cdc": "true", -"publication_name": "meroxademo", -"slot_name": "meroxademo", -"url": url, -"replication_url": url, -"key": "key", -"table": "records", -"columns": "key,column1,column2,column3", +```json +{ + "mode": "cdc", + "publication_name": "meroxademo", + "slot_name": "meroxademo", + "url": "url", + "key": "key", + "table": "records", + "columns": "key,column1,column2,column3" +} ``` ### CDC Event Buffer @@ -84,7 +82,6 @@ column names for the configured table and set them in memory. | mode | the connector's operation mode. Available modes: `cdc`, `snapshot` | no | `cdc` | | publication_name | name of the publication to listen for WAL events | req. for CDC mode | `pglogrepl` | | slot_name | name of the slot opened for replication events | req. for CDC mode | `pglogrepl_demo` | -| replication_url | URL for the CDC connection to use. If no replication_url is provided, then the CDC connection attempts the use the `url` value. | optional in CDC mode | n/a | # Destination The Postgres Destination takes a `record.Record` and parses it into a valid From 01d791831645c7fd8662d4d0ceb8354bfa572590 Mon Sep 17 00:00:00 2001 From: dylan_lott Date: Mon, 21 Mar 2022 16:02:02 -0600 Subject: [PATCH 3/3] updates default values in README for conduit slot and publication --- README.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 0285d74..95299c2 100644 --- a/README.md +++ b/README.md @@ -73,15 +73,15 @@ column names for the configured table and set them in memory. ## Configuration Options -| name | description | required | default | -|------------------|---------------------------------------------------------------------------------------------------------------------------------|----------------------|----------------------| -| table | the name of the table in Postgres that the connector should read | yes | n/a | -| url | formatted connection string to the database. | yes | n/a | -| columns | comma separated string list of column names that should be built in to each Record's payload. | no | `*` (all columns) | -| key | column name that records should use for their `Key` fields. defaults to the column's primary key if nothing is specified | no | primary key of table | -| mode | the connector's operation mode. Available modes: `cdc`, `snapshot` | no | `cdc` | -| publication_name | name of the publication to listen for WAL events | req. for CDC mode | `pglogrepl` | -| slot_name | name of the slot opened for replication events | req. for CDC mode | `pglogrepl_demo` | +| name | description | required | default | +| ---------------- | ------------------------------------------------------------------------------------------------------------------------ | ----------------- | -------------------- | +| table | the name of the table in Postgres that the connector should read | yes | n/a | +| url | formatted connection string to the database. | yes | n/a | +| columns | comma separated string list of column names that should be built in to each Record's payload. | no | `*` (all columns) | +| key | column name that records should use for their `Key` fields. defaults to the column's primary key if nothing is specified | no | primary key of table | +| mode | the connector's operation mode. Available modes: `cdc`, `snapshot` | no | `cdc` | +| publication_name | name of the publication to listen for WAL events | req. for CDC mode | `conduitpub` | +| slot_name | name of the slot opened for replication events | req. for CDC mode | `conduitslot` | # Destination The Postgres Destination takes a `record.Record` and parses it into a valid