Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update documentation about mode, remove unused param #6

Merged
merged 4 commits into from
Mar 22, 2022
Merged

Update documentation about mode, remove unused param #6

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bb5918b
Update documentation about mode
hariso Mar 10, 2022
db83402
remove references to replication_url
hariso Mar 10, 2022
899b394
Merge branch 'main' of github.com:ConduitIO/conduit-connector-postgre…
dylanlott Mar 21, 2022
01d7918
updates default values in README for conduit slot and publication
dylanlott Mar 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 19 additions & 23 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,19 +41,17 @@ Publication and slot name are user configurable, and must be correctly set.
The plugin will do what it can to be smart about publication and slot
management, but it can't handle everything.

If a `replication_url` is provided, it will be used for CDC features instead of
the url. If no `replication_url` is provided, but cdc is enabled, then it will
attempt to use that url value for CDC features and logical replication setup.
Example configuration for CDC features:
```
"cdc": "true",
"publication_name": "meroxademo",
"slot_name": "meroxademo",
"url": url,
"replication_url": url,
"key": "key",
"table": "records",
"columns": "key,column1,column2,column3",
```json
{
"mode": "cdc",
"publication_name": "meroxademo",
"slot_name": "meroxademo",
"url": "url",
"key": "key",
"table": "records",
"columns": "key,column1,column2,column3"
}
```

### CDC Event Buffer
Expand All @@ -75,17 +73,15 @@ column names for the configured table and set them in memory.

## Configuration Options

| name | description | required | default |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------- | -------------------- | -------------------- |
| table | the name of the table in Postgres that the connector should read | yes | n/a |
| url | formatted connection string to the database. | yes | n/a |
| columns | comma separated string list of column names that should be built in to each Record's payload. | no | `*` (all columns) |
| key | column name that records should use for their `Key` fields. defaults to the column's primary key if nothing is specified | no | primary key of table |
| snapshot | whether or not the plugin will take a snapshot of the entire table acquiring a read level lock before starting cdc mode | n/a | enabled |
| cdc | enables CDC features | req. for CDC mode | off |
| publication_name | name of the publication to listen for WAL events | req. for CDC mode | `pglogrepl` |
| slot_name | name of the slot opened for replication events | req. for CDC mode | `pglogrepl_demo` |
| replication_url | URL for the CDC connection to use. If no replication_url is provided, then the CDC connection attempts the use the `url` value. | optional in CDC mode | n/a |
| name | description | required | default |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------ | ----------------- | -------------------- |
| table | the name of the table in Postgres that the connector should read | yes | n/a |
| url | formatted connection string to the database. | yes | n/a |
| columns | comma separated string list of column names that should be built in to each Record's payload. | no | `*` (all columns) |
| key | column name that records should use for their `Key` fields. defaults to the column's primary key if nothing is specified | no | primary key of table |
| mode | the connector's operation mode. Available modes: `cdc`, `snapshot` | no | `cdc` |
| publication_name | name of the publication to listen for WAL events | req. for CDC mode | `conduitpub` |
| slot_name | name of the slot opened for replication events | req. for CDC mode | `conduitslot` |

# Destination
The Postgres Destination takes a `record.Record` and parses it into a valid
Expand Down