Address-Docs-Feedback-2025-05-14 (#1872)

* add guide to create cohort via CSV

* remove reference to record_collect_fonts

* add time_event to Javascript SDK

* add clarification about not set values for UTM tracking and marketing attribution

* update pie chart support for comparisons

* add requirement to send events to merge IDs for Simplified API

* add deduplication mechanism to dev docs

* add future timestamp correction warning

* spelling

Author: myronkaifung

openapi/src/docs/ingestion/track-event-deduplication.md

@@ -1,7 +1,28 @@
-Event deduplication allows a project to send the same exact event while only recording that event once.
-Deduplication only occurs when a subset of the event data is exactly identical.
+Mixpanel provides an event deduplication mechanism to ensure that duplicate events do not skew your analytics. Deduplication is essential when events may be sent multiple times due to network retries, client-side batching, or integration with multiple data sources.
+
+<br />
+
+## How Deduplication Works
+
+Mixpanel deduplicates events using a combination of four key event properties:
+
+- Event Name (`event`)
+- Distinct ID (`distinct_id`)
+- Timestamp (`time`)
+- Insert ID (`$insert_id`)
+
+If all four of these properties are identical across two or more events, Mixpanel considers them duplicates and will only show the most recent version of that event in your reports. This applies regardless of whether the events are sent via SDKs, APIs, or other integrations. 
+
+The `$insert_id` should be a randomly generated, unique value for each event to ensure proper deduplication. If `$insert_id` are reused, events may be unintentionally deduplicated.
+
+Only the four key event properties listed above are used for deduplication. Additional event properties are not considered for the deduplication mechanism. For example, if two events share the same Event Name, Distinct ID, Timestamp, and Insert ID, but have different $city value, they are still considered duplicate events.
+
+### Deduplication Example
+
+Deduplication occurs when a subset of the event data (event name, distinct_id, timestamp, $insert_id) is identical. Other event properties are not considered.
 
 **Required [Event Object](doc:data-model#anatomy-of-an-event) attributes**
+
 [block:parameters]
 {
   "data": {
@@ -13,6 +34,7 @@ Deduplication only occurs when a subset of the event data is exactly identical.
     "0-2": "A name for the event. For example, \"Signed up\", or \"Uploaded Photo\".",
     "1-0": "**properties**",
     "1-1": "<span style=\"font-family: courier\">Object</span></br><span style=\"color: red\">required</span>",
+    "1-2": "",
     "2-0": "**properties.distinct_id**",
     "2-1": "<span style=\"font-family: courier\">String</span></br><span style=\"color: red\">required</span>",
     "2-2": "The value of `distinct_id` will be treated as a string, and used to uniquely identify a user associated with your event. If you provide a distinct_id property with your events, you can track a given user through funnels and distinguish unique users for retention analyses. You should always send the same distinct_id when an event is triggered by the same user.",
@@ -27,32 +49,56 @@ Deduplication only occurs when a subset of the event data is exactly identical.
     "5-2": "A unique UUID tied to exactly one occurrence of an event."
   },
   "cols": 3,
-  "rows": 6
+  "rows": 6,
+  "align": [
+    "left",
+    "left",
+    "left"
+  ]
 }
 [/block]
 
-In other words, each event containing an $insert_id is checked for duplication after being minimized to the following shape:
+
+In other words, each event containing an `$insert_id` is checked for duplication after being minimized to the following shape:
 
 ```json
 {
-  "event": "Back to Back",
+  "event": "Item Purchased",
   "properties": {
-    "token": "project_token",
-    "distinct_id": "[email protected]",
+    "token": "my_project_token",
+    "distinct_id": "user123xyz",
     "time": 1601412131000,
     "$insert_id": "88B7hahbaschhhB66cbsg"
   },
 }
 ```
 
-If this simplified object is an exact match to any other simplified event it is marked as a duplicate. Ingested events that have been marked as a duplicate will be deleted within 24 hours.
+If this minimized event object is an exact match to any other minimized event object, it is marked as a duplicate. Ingested events that have been marked as a duplicates will be deduplicated.
 
-If an event is sent to the Ingestion API without an `$insert_id` one will be generated for it. However, it will not qualify for the deduplication process.
+If an event is sent to the Ingestion API without an `$insert_id`, one will be generated for it. However, it will not qualify for the deduplication process.
 
-[block:callout]
-{
-  "type": "warning",
-  "title": "Deduplication does not rewrite data",
-  "body": "Using $insert_id is only used to prevent duplicate event data. It cannot be used to update, replace, or delete existing events."
-}
-[/block]
+## Deduplication Mechanisms
+
+Mixpanel uses two main deduplication processes:
+
+### Query-Time Deduplication
+
+- When: Happens immediately when you query data in the Mixpanel UI.
+- How: If multiple events share the same event_name, distinct_id, timestamp, and $insert_id, only the most recent version of the event is shown in reports (based on the API ingestion time). This ensures that duplicate events do not affect your analytics in real time.
+- Scope: This deduplication is visible in the Mixpanel UI and reports, but not in raw data exports. Raw event export will contain all data as they were ingested, without any deduplication.
+
+### Compaction-Time Deduplication
+
+- When: Runs periodically in the backend, typically after a few hours and again after about 20 days, once data ingestion for a day is complete.
+- How: During compaction, Mixpanel scans for events with the same event name, distinct_id, and $insert_id (timestamp does not need to match exactly, just the same calendar day). The older event is deleted, and only the latest remains in storage.
+- Scope: This process helps reduce storage of duplicate events and may affect event counts if duplicates were present with different timestamps
+
+<br />
+
+## Important Notes
+
+**Raw Event Export** - Deduplication is not applied to raw data exports. If you export events via the API, you may see duplicates. It is recommended to apply the same deduplication logic (event name, distinct_id, timestamp, $insert_id) to your exported data
+
+**Insert ID Best Practice** - Always generate a unique $insert_id for each event. Reusing $insert_id (e.g., setting it to the user’s distinct_id) can cause unintended deduplication and data loss
+
+**Deduplication Timing** - Query-time deduplication is immediate. Compaction-time deduplication timing is not guaranteed and may take hours to days to complete.

openapi/src/ingestion.openapi.yaml

@@ -101,7 +101,7 @@ paths:
                       time:
                         type: integer
                         title: time
-                        description: The time at which the event occurred, in seconds or milliseconds since UTC epoch.
+                        description: The time at which the event occurred, in seconds or milliseconds since UTC epoch. If the time value is set in the future, it will be overwritten with the current present time at ingestion.
                       distinct_id:
                         type: string
                         title: distinct_id
@@ -163,7 +163,7 @@ paths:
                       time:
                         type: integer
                         title: time
-                        description: The time at which the event occurred, in seconds or milliseconds since UTC epoch.
+                        description: The time at which the event occurred, in seconds or milliseconds since UTC epoch. If the time value is set in the future, it will be overwritten with the current present time at ingestion.
                       distinct_id:
                         type: string
                         title: distinct_id

pages/docs/data-structure/user-profiles.mdx

@@ -159,9 +159,11 @@ See here for more on how to [import](https://docs.mixpanel.com/docs/tracking-met
 Historical properties can be used anywhere that regular profile properties can be used.
 
 For eg, when you apply breakdown by historical plan-type property, the property value will be picked based on the time of the event, instead of the current property value.
+
 ![image](/historical_property_value.webp)
 
 When you hover over a historical property, the context menu that pops up will show that the property was sourced from a history table, as well as the name of the source. This means that the value of the property used in charts can vary over time.
+
 ![image](/dropdown_historical_property.webp)
 
 ## Deleting Profiles

pages/docs/features/advanced.md

@@ -439,7 +439,7 @@ Comparisons are supported across all insights chart types. Depending on the exac
 | Insights Stacked Line | No | No | Yes |
 | Insights Bar | Yes | Yes | Yes |
 | Insights Stacked Bar | Yes | No | No |
-| Insights Pie | No | No | Yes |
+| Insights Pie | No | No | No |
 | Insights Metric | Yes | Yes | Yes |
 | Funnels Steps | Yes | Yes | No |
 | Funnel Trends | Yes | Yes | No |

pages/docs/features/attribution.mdx

@@ -10,7 +10,7 @@ import { Callout } from 'nextra/components'
 
 Attribution helps teams attribute conversion credit to the touchpoints in a user journey, whether it's to the first or last touch (single-touch attribution models) or to multiple touchpoints using a multi-touch attribution model like U-shape or Linear. 
 
-Let’s consider an example user journey:
+Consider the following example user journey:
 1. A user sees an ad for a product on Facebook
 2. The user clicks on the ad and is taken to the product page on the company's website
 3. The user adds the product to their cart and begins the checkout process
@@ -64,7 +64,7 @@ If you use a Mixpanel js-sdk, we’ve updated our sdk to track utm parameters mo
 - **Attributed by property:** This is the property on a touchpoint event that we use for the attribution model. The canonical example is utm_source
 - **Lookback window:** The time window where a user's events with this attribution property are counted towards the calculation. The window ends when the conversion metric happens.
 
-## Frequently Asked Questions
+## FAQ
 
 ### How does Mixpanel compute attribution under the hood?
 
@@ -146,3 +146,7 @@ NOTE: You can apply a filter on an attribution property only after an attributio
 - Step 1: Turn on Attribution analysis by going to the breakdown section and choosing `Attributed by..` and property `XYZ`
 - Step 2 (a): Once attribution model has been applied, go to the filter section and choose the computed property `Attributed by XXX`. You can apply an attribution filter only on the property used in the attribution breakdown
 - Step 2 (b): Once attribution model has been applied, click on the chart bar and filter/exclude the segments as needed
+
+### What does the "(not set)" attribution segment mean?
+
+You may see a "(not set)" segment in your report when using the Attribution feature. This occurs when the attribution property is missing from all events being evaluated for the user.

pages/docs/session-replay/implement-session-replay/session-replay-web.mdx

@@ -100,7 +100,6 @@ mixpanel.init(
 | --- | --- | --- |
 | `record_block_class` | CSS class name or regular expression for elements which will be replaced with an empty element of the same dimensions, blocking all contents.  | `new RegExp('^(mp-block\|fs-exclude\|amp-block\|rr-block\|ph-no-capture)$')` <br/> (common industry block classes) |
 | `record_block_selector` | CSS selector for elements which will be replaced with an empty element of the same dimensions, blocking all contents.  | `"img, video"` |
-| `record_collect_fonts` | When true, Mixpanel will collect and store the fonts on your site to use in playback. | `false` |
 | `record_idle_timeout_ms` | Duration of inactivity in milliseconds before ending a contiguous replay. A new replay collection will start when active again. | `1800000`<br/>(30 minutes) |
 | `record_mask_text_class` | CSS class name or regular expression for elements that will have their text contents masked. | `new RegExp('^(mp-mask\|fs-mask\|amp-mask\|rr-mask\|ph-mask)$')` <br/> (common industry mask classes) |
 | `record_mask_text_selector` | CSS selector for elements that will have their text contents masked. | `"*"` |

pages/docs/tracking-best-practices/traffic-attribution.mdx

@@ -23,6 +23,12 @@ Mixpanel's Javascript library will also track initial_utm_parameters as a profil
 
 UTM parameters are by default persisted across events as [Super Properties](/docs/tracking-methods/sdks/javascript#setting-super-properties). To opt in to the recommended modern behavior most compatible with our [attribution models](/docs/features/attribution), use the SDK initialization option `{stop_utm_persistence: true}` to disable UTM param persistence (refer to our [Release Notes](https://github.com/mixpanel/mixpanel-js/releases/tag/v2.52.0) in GitHub).
 
+#### Organic Traffic
+
+If a user arrives at your landing page organically, no UTM tags will be parsed because the URL does not contain them. As a result, the UTM property will be absent from the events and will appear as "(not set)" when used as a breakdown in a report. You can interpret a "(not set)" value for any UTM property as indicating organic or direct traffic.
+
+Learn more about falsy values [here](/docs/data-structure/property-reference/data-type#undefined-and-null).
+
 ### Initial Referrer and Initial Referring Domain Properties
 
 Mixpanel's Javascript library will track Initial Referrer and Initial Referring Domain and append them as a property to any event that a user completes. These properties are stored in the Mixpanel cookie the first time a user comes to your site and will not change on future site visits as long as the cookie is not cleared.
@@ -33,7 +39,7 @@ Having this information allows you to build reports to see how users from differ
 
 #### $direct
 
-An initial referrer is equal to $direct when a user first lands on a site without being referred by another website. The user may have typed the website address directly, clicked a bookmark, clicked a link from an email, or might have security settings in their browser that prevent referrer data from being passed.
+An initial referrer is equal to `$direct` when a user first lands on a site without being referred by another website. The user may have typed the website address directly, clicked a bookmark, clicked a link from an email, or might have security settings in their browser that prevent referrer data from being passed.
 
 ## Mobile Attribution
 

pages/docs/tracking-methods/id-management/identifying-users-simplified.mdx

@@ -54,11 +54,12 @@ If an event contains a `$user_id`, the value of the `$user_id` will be set as th
 
 ## Client-side Identity Management
 
-If using our Web/Mobile SDKs or a CDP like Segment or Rudderstack, there are only 2 steps:
-1. Call `.identify(<user_id>)` when a user signs up or logs in. Pass in the user's known identifier (eg: their ID from your database).
-2. Call `.reset()` when a user logs out.
+If using our Web/Mobile SDKs or a CDP like Segment or Rudderstack, there are only 2 steps to identity management:
+1. Call `.identify(<user_id>)` when a user signs up or logs in, passing in the user's known identifier (eg: their ID from your database).
+2. Send at least one event after the `.identify()` call. This is necessary to get the `$user_id` and `$device_id` to merge. Learn more about [the merge mechanism above](/docs/tracking-methods/id-management/identifying-users-simplified#mechanism).
+3. Call `.reset()` when a user logs out.
 
-- Any events prior to calling `.identify` are considered anonymous events. Mixpanel's SDKs will generate a `$device_id` to associate these events to the same anonymous user. By calling `.identify(<user_id>)` when a user signs up or logs in, you're telling Mixpanel that `$device_id` belongs to a known user with ID `user_id`.
+- Any events prior to calling `.identify()` are considered anonymous events. Mixpanel's SDKs will generate a `$device_id` to associate these events to the same anonymous user. By calling `.identify(<user_id>)` when a user signs up or logs in, you're telling Mixpanel that `$device_id` belongs to a known user with ID `user_id`.
 
 - Under the hood, Mixpanel will stitch the event streams of those users together. This works even if a user has multiple anonymous sessions (eg: on desktop and mobile). As long as you always call `.identify` when the user logs in, all of that activity will be stitched together.