Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

discovery doesn't generate schema for merge_fields and interests fields/objects #40

Open
RuslanBergenov opened this issue Jun 8, 2021 · 0 comments

Comments

@RuslanBergenov
Copy link

RuslanBergenov commented Jun 8, 2021

In our mailchimp, we have merge_fields object/field with some sub-fields. The following objects/fields are nested inside merge_fields:

  • first name
  • last name
  • address
  • phone
  • birthday

When we run discovery, subfields names and data type are not being included in schema/tap-catalog file.

I run this command to do schema discovery:
"{my_directory}\venv_mailchimp\Scripts\tap-mailchimp" --config rsc/dt-tap-config.json --discover > dt-tap-catalog-discovery.json

Schema discovery ouputs tap-catalog.json file, which contains the following:

{...
 "merge_fields": {},
...
         "interests": {}
...
 }      

This is a valid json, but it doesn't specify which fields/objects are nested inside merge_fields and interests and their data types. It returns an undefined object with no properties. As a result, we cannot pipe this output into target-bigquery (either data load fails, or it skips merge_fields and interests fields/objects).

We worked around this issue by modifying merge_fields and interests in tap-catalog and specifying that they are a string.

Is it an issue? Is it intentional and by design?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant