You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/unify/data-graph/index.md
+29-24
Original file line number
Diff line number
Diff line change
@@ -13,12 +13,16 @@ The Data Graph acts as a semantic layer that allows businesses to define relatio
13
13
14
14
## Prerequisites
15
15
16
+
> info "Why you need both materialized and unmaterialized tables"
17
+
> Segment recommends using materialized views for Profiles Sync to optimize performance and reduce query costs with Linked Audiences. However, due to schema inference requirements, you still need to select the matching **unmaterialized tables** as well. Segment relies on the unmaterialized tables during setup, even if they’re not used when queries run.
18
+
16
19
To use the Data Graph, you'll need the following:
17
20
18
21
- A supported data warehouse with the appropriate Data Graph permissions
19
22
- Workspace Owner or Unify Read-only/Admin and Entities Admin permissions
20
23
- For Linked Audiences, set up [Profiles Sync](/docs/unify/profiles-sync/) in a Unify space with ready-to-use [data models and tables](/docs/unify/profiles-sync/tables/) in your warehouse. When setting up selective sync, Segment recommends the following settings:
21
24
- Under **Profile materialized tables**, select all the tables (`user_identifier`, `user_traits`, `profile_merges`) for faster and more cost-efficient Linked Audiences computations in your data warehouse.
25
+
-**Make sure to include the unmaterialized tables as well**. Segment needs them during setup to understand your schema.
22
26
- Under **Track event tables**, select **Sync all Track Call Tables** to enable filtering on event history for Linked Audiences conditions.
23
27
24
28
> info ""
@@ -29,7 +33,7 @@ To use the Data Graph, you'll need the following:
29
33
> Data Graph, Reverse ETL, and Profiles Sync require different warehouse permissions.
30
34
31
35
To get started with the Data Graph, set up the required permissions in your warehouse. Segment supports the following:
32
-
- Linked Audiences: [BigQuery](/docs/unify/data-graph/setup-guides/BigQuery-setup/), [Databricks](/docs/unify/data-graph/setup-guides/databricks-setup/), and [Snowflake](/docs/unify/data-graph/setup-guides/snowflake-setup/)
36
+
- Linked Audiences: [BigQuery](/docs/unify/data-graph/setup-guides/BigQuery-setup/), [Databricks](/docs/unify/data-graph/setup-guides/databricks-setup/), [Redshift](/docs/unify/data-graph/setup-guides/redshift-setup/), and [Snowflake](/docs/unify/data-graph/setup-guides/snowflake-setup/)
33
37
- Linked Events: [BigQuery](/docs/unify/data-graph/setup-guides/BigQuery-setup/), [Databricks](/docs/unify/data-graph/setup-guides/databricks-setup/), [Redshift](/docs/unify/data-graph/setup-guides/redshift-setup/), and [Snowflake](/docs/unify/data-graph/setup-guides/snowflake-setup/)
34
38
35
39
To track the data sent to Segment on previous syncs, Segment uses [Reverse ETL](/docs/connections/reverse-etl/) infrastructure to store diffs in tables within a dedicated schema called `_segment_reverse_etl` in your data warehouse. You can choose which database or project in your warehouse this data lives in.
@@ -107,8 +111,8 @@ data_graph {
107
111
primary_key = "SUB_ID"
108
112
}
109
113
110
-
# Define the profile entity, which corresponds to Segment Profiles tables synced via Profiles Sync
111
-
#Recommend setting up Profiles Sync materialized views to optimize warehouse compute costs
114
+
# Define the profile entity, which corresponds to Segment Profiles tables synced with Profiles Sync
115
+
#Use materialized views in Profiles Sync to reduce query costs and speed things up
112
116
profile {
113
117
profile_folder = "PRODUCTION.SEGMENT"
114
118
type = "segment:materialized"
@@ -118,22 +122,22 @@ data_graph {
118
122
relationship "user-accounts" {
119
123
name = "Premium Accounts"
120
124
related_entity = "account-entity"
121
-
# Join the profile entity with an identifier (e.g. email) on the related entity table
125
+
# Join the profile entity with an identifier (like email) on the related entity table
122
126
# Option to replace with the trait block below to join with a profile trait on the entity table instead
123
127
external_id {
124
128
type = "email"
125
129
join_key = "EMAIL_ID"
126
130
}
127
131
128
132
# Define 1:many relationship between accounts and carts
129
-
#e.g. an account can be associated with many carts
133
+
#for example, an account can be associated with many carts
# Define many:many relationship between carts and products
136
-
#e.g. there can be multiple carts, and each cart can be associated with multiple products
140
+
#for example, there can be multiple carts, and each cart can be associated with multiple products
137
141
relationship "products" {
138
142
name = "Purchased Products"
139
143
related_entity = "product-entity"
@@ -157,7 +161,7 @@ data_graph {
157
161
}
158
162
159
163
# Define 1:many relationship between households and subscriptions
160
-
#e.g. a household can be associated with multiple subscriptions
164
+
#for example, a household can be associated with multiple subscriptions
161
165
relationship "user-subscriptions" {
162
166
name = "Subscriptions"
163
167
related_entity = "subscription-entity"
@@ -203,10 +207,10 @@ data_graph {
203
207
204
208
Next, define the profile. This is a special class of entity that represents Segment Profiles, which corresponds to the Profiles Sync tables and models. For Linked Audiences, this allows marketers to filter on profile traits, event history, etc. There can only be one profile for a Data Graph.
|`profile_folder`| Define the fully qualified path of the folder or schema location for the profile tables. |
209
-
|`type`| Identify the materialization method of the profile tables defined in your Profiles Syncconfiguration under [Selective Sync settings](/docs/unify/profiles-sync/profiles-sync-setup/#step-3-set-up-selective-sync): `segment:unmaterialized` or `segment:materialized`.|
|`profile_folder`| Define the fully qualified path of the folder or schema location for the profile tables.|
213
+
|`type`| Use `segment:materialized` to sync materialized views withProfiles Sync. Segment recommends this configuration forall Linked Audiences and Data Graph setups. If you can't sync materialized views, [reach out to Segment support](https://segment.com/help/contact/){:target="_blank"} for help. |
210
214
211
215
**Example:**
212
216
@@ -238,23 +242,24 @@ This is the first level of relationships and a unique type of relationship betwe
|`relationship`| An immutable slug for the relationship, and will be treated as a delete if you make changes. The slug must be inall lowercase, and supports dashes or underscores (e.g.`user-account`or`user_account`) |
245
+
|`relationship`| An immutable slug for the relationship, and will be treated as a delete if you make changes. The slug must be inall lowercase, and supports dashes or underscores (like`user-account`or`user_account`) |
242
246
|`name`| A label displayed throughout your Segment space for Linked Events, Linked Audiences, etc. This name can be modified at any time |
243
247
|`related_entity`| References your already defined entity |
244
248
245
249
To define a profile-to-entity relationship, reference your entity table and depending on your table columns, choose to join on one of the following:
246
250
247
-
**Option 1 (Most common) - Join on an external ID:** Use the `external_id` block to join the profile entity with an entity table using external IDs from your [Unify ID resolution](/docs/unify/identity-resolution/externalids/) settings. Typically these identifiers are `user_id`, `email`, or`phone` depending on the column in the entity table that you want to join with.
248
-
-`type`: Represents the [external IDtype](/docs/unify/identity-resolution/externalids/#default-externalids) (`email`, `phone`, `user_id`) in your id-res settings. Depending on if you are using materialized or unmaterialized profiles, these correspond to different columns in your Profiles Sync warehouse tables:
249
-
- [Materialized](/docs/unify/profiles-sync/tables/#the-user_identifiers-table) (Recommended): This corresponds to the `type` column in your Profiles Sync `user_identifiers` table.
250
-
- [Unmaterialized](/docs/unify/profiles-sync/tables/#the-external_id_mapping_updates-table): This corresponds to the `external_id_type` column in your Profiles Sync `external_id_mapping_updates` table.
251
-
-`join_key`: This is the column on the entity table that you are matching to the external identifier.
251
+
**Option 1 (Most common) - Join on an external ID:** Use the `external_id` block to join the profile entity with an entity table using external IDs from your [Unify ID resolution](/docs/unify/identity-resolution/externalids/) settings. Typically these identifiers are `user_id`, `email`, or`phone` depending on the structure of your entity table.
252
+
-`type`: Represents the [external IDtype](/docs/unify/identity-resolution/externalids/#default-externalids) (`email`, `phone`, `user_id`) in your ID resolution settings.
253
+
- This maps to the `type` column in the `user_identifiers` table when using materialized views.
254
+
-`join_key`: The column on the entity table that matches the external ID.
255
+
256
+
> note ""
257
+
> Segment recommends using materialized views with Profiles Sync. However, Segment may still reference unmaterialized tables during setup for schema detection.
252
258
253
259
**Option 2- Join on a profile trait:** Use the `trait` block to join the profile entity with an entity table using [Profile Traits](/docs/unify/#enrich-profiles-with-traits).
254
-
-`name`: Represents a trait name in your Unify profiles. Depending on if you are using materialized or unmaterialized profiles, these correspond to different columns in your Profiles Sync warehouse tables:
255
-
- [Materialized](/docs/unify/profiles-sync/tables/#the-profile_traits-table) (Recommended): The trait name corresponds to a unique value of the `name` column in your Profiles Sync `user_traits` table.
256
-
- [Unmaterialized](/docs/unify/profiles-sync/tables/#the-profile_traits_updates-table): This corresponds to a column in the Profile Sync `profile_trait_updates` table.
257
-
-`join_key`: This is the column on the entity table that you are matching to the trait.
260
+
-`name`: Represents a trait name in your Unify profiles.
261
+
- This maps to the `name` column in the `user_traits` table when using materialized views.
262
+
-`join_key`: The column on the entity table that you're matching to the trait.
258
263
259
264
**Example:**
260
265
```python
@@ -277,7 +282,7 @@ data_graph {
277
282
name = "Premium Accounts"
278
283
related_entity = "account-entity"
279
284
280
-
# Option 1: Join the profile entity with an identifier (e.g. email) on the related entity table
285
+
# Option 1: Join the profile entity with an identifier (like email) on the related entity table
281
286
external_id {
282
287
type = "email"
283
288
join_key = "EMAIL_ID"
@@ -298,7 +303,7 @@ For 1:many relationships, define the join on between the two entity tables using
|`relationship`| An immutable slug for the relationship, and will be treated as a delete if you make changes. The slug must be inall lowercase, and supports dashes or underscores (e.g.`user-account`or`user_account`) |
306
+
|`relationship`| An immutable slug for the relationship, and will be treated as a delete if you make changes. The slug must be inall lowercase, and supports dashes or underscores (like`user-account`or`user_account`) |
302
307
|`name`| A label displayed throughout your Segment space for Linked Events, Linked Audiences, and so on. This name can be modified at any time |
303
308
|`related_entity`| References your already defined entity |
304
309
|`join_on`| Defines relationship between the two entity tables `[lefty entity slug].[column name] = [right entity slug].[column name]`. Note that since you’re referencing the entity slug for the join on, you do not need to define the full table reference |
@@ -343,7 +348,7 @@ For many:many relationships, define the join on between the two entity tables wi
|`relationship`| An immutable slug for the relationship, and will be treated as a delete if you make changes. The slug must be inall lowercase, and supports dashes or underscores (e.g.`user-account`or`user_account`) |
351
+
|`relationship`| An immutable slug for the relationship, and will be treated as a delete if you make changes. The slug must be inall lowercase, and supports dashes or underscores (like`user-account`or`user_account`) |
347
352
|`name`| A label displayed throughout your Segment space for Linked Events, Linked Audiences, and so on. This name can be modified at any time |
348
353
|`related_entity`| References your already defined entity |
> BigQuery for Data Graph is in beta and Segment is actively working on this feature. Some functionality may change before it becomes generally available. This feature is governed by Segment’s [First Access and Beta Preview Terms](https://www.twilio.com/en-us/legal/tos){:target="_blank"}.
9
+
> warning ""
10
+
> Data Graph, Reverse ETL, and Profiles Sync require different warehouse permissions.
11
11
12
12
Set up your BigQuery data warehouse to Segment for the [Data Graph](/docs/unify/data-graph/data-graph/).
13
13
14
-
15
14
## Step 1: Roles and permissions
16
15
> warning ""
17
16
> You need to be an account admin to set up the Segment BigQuery connector as well as write permissions for the `__segment_reverse_etl` dataset.
> Redshift for Data Graph is in beta and Segment is actively working on this feature. Some functionality may change before it becomes generally available. This feature is governed by Twilio Segment’s [First Access and Beta Preview Terms](https://www.twilio.com/en-us/legal/tos){:target="_blank"}.
9
+
> warning ""
10
+
> Data Graph, Reverse ETL, and Profiles Sync require different warehouse permissions.
11
11
12
12
Set up your Redshift data warehouse to Segment for the [Data Graph](/docs/unify/data-graph/).
13
13
14
14
## Prerequisite
15
15
16
-
To use Linked Audiences with Redshift, the Data Graph only supports [materialized views](/docs/unify/profiles-sync/tables/#tables-segment-materializes).
17
-
18
16
If you're setting up Profiles Sync for the first time in the Unify space, go through the setup flow for Selective sync. If Profiles Sync is already set up for your Unify space, follow these steps to configure Profiles Sync for your Unify space:
0 commit comments