Overall doc review (mozilla#366)

This includes quite a number of things, but the main goals were:
* update, fix, or remove references to deprecated tools (like ATMO)
* Add GCP data platform reference material, and shift the focus from the AWS pipeline to the GCP one
* General cleanup, such as splitting long lines and using like references instead of inline links.

Other things:
* document `sample_id`
* Fix a bunch of broken links
* Update projects page

Author: mreid-moz

.spelling

@@ -56,6 +56,7 @@ Colaboratory
 config
 CommonMark
 CPOW
+CRC
 CSV
 cutover
 dashboarding
@@ -187,6 +188,7 @@ Prefs
 prefs
 prepended
 protobuf
+PubSub
 pushlog
 PyPI
 PySpark
@@ -246,6 +248,7 @@ toolbars
 toolchain
 trailhead
 trailheads
+transpiler
 triaged
 triaging
 TTL

README.md

@@ -2,17 +2,18 @@
 
 [![Linux Build Status](https://travis-ci.org/mozilla/firefox-data-docs.svg?branch=master)](https://travis-ci.org/mozilla/firefox-data-docs)
 
-This document is intended to help Mozilla's developers and data scientists analyze and interpret the data gathered
-by the Firefox Telemetry system.
+This document is intended to help Mozilla's developers and data scientists
+analyze and interpret the data gathered by the Firefox Telemetry system.
 
-At [Mozilla](https://www.mozilla.org), our data-gathering and data-handling practices are anchored in our
-[Data Privacy Principles](https://www.mozilla.org/en-US/privacy/principles/) and elaborated in the
-[Mozilla Privacy Policy](https://www.mozilla.org/en-US/privacy/).
+At [Mozilla](https://www.mozilla.org), our data-gathering and data-handling
+practices are anchored in our
+[Data Privacy Principles](https://www.mozilla.org/en-US/privacy/principles/)
+and elaborated in the [Mozilla Privacy Policy](https://www.mozilla.org/en-US/privacy/).
 
-To learn more about what data Firefox collects and the choices you can make as a user, please see the
-[Firefox Privacy Notice](https://www.mozilla.org/en-US/privacy/firefox/).
+To learn more about what data Firefox collects and the choices you can make
+as a user, please see the [Firefox Privacy Notice](https://www.mozilla.org/en-US/privacy/firefox/).
 
-You can find the [rendered documentation here](https://mozilla.github.io/firefox-data-docs/).
+The rendered documentation is hosted at [https://docs.telemetry.mozilla.org/](https://docs.telemetry.mozilla.org/).
 
 This documentation is rendered with [mdBook](https://github.com/rust-lang/mdBook).
 

src/SUMMARY.md

@@ -11,10 +11,11 @@
   * [Getting Help](concepts/getting_help.md)
 * [Tools](tools/index.md)
   * [Project Glossary](tools/projects.md)
-  * [Overview of Mozilla's Data Pipeline](concepts/pipeline/data_pipeline.md)
-    * [In-depth Data Pipeline Detail](concepts/pipeline/data_pipeline_detail.md)
+  * [Overview of Mozilla's Data Pipeline](concepts/pipeline/gcp_data_pipeline.md)
     * [HTTP Edge Server Specification](concepts/pipeline/http_edge_spec.md)
     * [Event Pipeline Detail](concepts/pipeline/event_pipeline.md)
+    * [Previous AWS Pipeline Overview](concepts/pipeline/data_pipeline.md)
+    * [In-depth AWS Data Pipeline Detail](concepts/pipeline/data_pipeline_detail.md)
   * [Analysis Interfaces](tools/interfaces.md)
   * [Custom analysis with Spark](tools/spark.md)
   * [SQL Style Guide](concepts/sql_style.md)
@@ -25,7 +26,6 @@
   * [Accessing and working with BigQuery](cookbooks/bigquery.md)
     * [Scheduling BigQuery Queries in Airflow](cookbooks/bigquery-airflow.md)
   * [Dataset Specific](cookbooks/dataset_specific.md)
-    * [Working with Crash Pings](cookbooks/crash_pings.md)
     * [Working with Normandy events](cookbooks/normandy_events.md)
   * [Real-time](cookbooks/realtime.md)
     * [Creating a Real-time Analysis Plugin](cookbooks/realtime_analysis_plugin.md)
@@ -50,11 +50,9 @@
     * [Addons Daily](datasets/other/addons_daily/reference.md)
     * [Clients Daily](datasets/batch_view/clients_daily/reference.md)
     * [Clients Last Seen](datasets/bigquery/clients_last_seen/reference.md)
-    * [Cross Sectional](datasets/batch_view/cross_sectional/reference.md)
     * [Events](datasets/batch_view/events/reference.md)
     * [Exact MAU](datasets/bigquery/exact_mau/reference.md)
     * [First Shutdown Summary](datasets/batch_view/first_shutdown_summary/reference.md)
-    * [Longitudinal](datasets/batch_view/longitudinal/reference.md)
     * [Main Summary](datasets/batch_view/main_summary/reference.md)
     * [New Profile](datasets/batch_view/new_profile/reference.md)
     * [Socorro Crash Reports](datasets/other/socorro_crash/reference.md)
@@ -72,15 +70,16 @@
     * [Stub installer ping](datasets/other/stub_installer/reference.md)
     * [Activity Stream](datasets/other/activity-stream/reference.md)
   * [Obsolete Datasets](datasets/obsolete.md)
-    * [Heavy Users](datasets/obsolete/heavy_users/reference.md)
-    * [Crash Aggregates](datasets/obsolete/crash_aggregates/reference.md)
-    * [Crash Summary](datasets/obsolete/crash_summary/reference.md)
+    * [Churn](datasets/obsolete/churn/reference.md)
     * [Client Count](datasets/obsolete/client_count/reference.md)
     * [Client Count Daily](datasets/obsolete/client_count_daily/reference.md)
+    * [Crash Aggregates](datasets/obsolete/crash_aggregates/reference.md)
+    * [Crash Summary](datasets/obsolete/crash_summary/reference.md)
+    * [Error Aggregates](datasets/obsolete/error_aggregates/reference.md)
+    * [Heavy Users](datasets/obsolete/heavy_users/reference.md)
+    * [Longitudinal](datasets/obsolete/longitudinal/reference.md)
     * [Retention](datasets/obsolete/retention/reference.md)
-    * [Churn](datasets/obsolete/churn/reference.md)
     * [Sync Summary](datasets/obsolete/sync_summary/reference.md)
-    * [Error Aggregates](datasets/obsolete/error_aggregates/reference.md)
   * [Firefox Accounts Datasets](datasets/fxa.md)
     * [Firefox Account Attribution](datasets/fxa_metrics/attribution.md)
     * [Firefox Account Funnel Metrics](datasets/fxa_metrics/funnels.md)
@@ -99,6 +98,7 @@
         * [Channel Normalization and Querying](concepts/channels/channel_normalization.md)
     * [Census metrics](concepts/censuses.md)
     * [Engagement metrics](concepts/engagement.md)
+    * [Sampling](concepts/sample_id.md)
 
 ---