Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feat/usage reporting/add active users #6627

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

Feat/usage reporting/add active users #6627

wants to merge 4 commits into from
+970 −8

Conversation

gkatre
Copy link
Contributor

@gkatre gkatre commented Dec 6, 2024 •
edited by data-sync-user
Loading

Description

Adding templates for active_users and active_users_aggregates ETL for usage_reporting ping.

This has dependencies on the feat/add_usage_reporting-to-glean_usage branch (PR: #6581)

Tables:

[app_name]_derived.usage_reporting_active_users_aggregates_v1 - A daily aggregate table of the usage_reporting ping.

Views:

[app_name].usage_reporting_active_users - A view that merges the computations for client first seen
and last seen metrics with the daily client aggregations from clients_daily for the usage_reporting ping.
[app_name].usage_reporting_active_users_aggregates

Related Tickets & Documents

┆Issue is synchronized with this Jira Task

@gkatre gkatre requested a review from kik-kik December 6, 2024 23:15
@dataops-ci-bot
Copy link

Integration report for "usage reporting / adding active users"

sql.diff

Click to expand! 
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn: extensionsession
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozillavpn: extensionsession
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_vpn: extensionsession
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxvpn: extensionsession
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxvpn_network_extension: extensionsession
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:03:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:06:02.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-28"
+    DATE(submission_timestamp) >= "2024-12-06"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:03:10.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:06:02.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-28"
+    DATE(submission_timestamp) >= "2024-12-06"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:03:10.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:06:02.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-28"
+    DATE(submission_timestamp) >= "2024-12-06"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:03:10.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:06:02.000000000 +0000
@@ -87,7 +87,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-28"
+    DATE(submission_timestamp) >= "2024-12-06"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:03:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:06:02.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-28"
+    DATE(submission_timestamp) >= "2024-12-06"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2024-12-06 23:02:22.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2024-12-06 23:12:00.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: source
+  mode: NULLABLE
+- name: source
   type: STRING
-- mode: NULLABLE
-  name: event_type
+  mode: NULLABLE
+- name: event_type
   type: STRING
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: subdivision1
+  mode: NULLABLE
+- name: subdivision1
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: event_count
+  mode: NULLABLE
+- name: event_count
   type: INTEGER
-- mode: NULLABLE
-  name: user_count
+  mode: NULLABLE
+- name: user_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2024-12-06 23:02:22.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2024-12-06 23:12:11.000000000 +0000
@@ -1,40 +1,40 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: impression_count
+  mode: NULLABLE
+- name: impression_count
   type: INTEGER
-- mode: NULLABLE
-  name: click_count
+  mode: NULLABLE
+- name: click_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:03:10.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:06:02.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-28"
+    DATE(submission_timestamp) >= "2024-12-06"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/schema.yaml	2024-12-06 23:03:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/schema.yaml	2024-12-06 23:12:06.000000000 +0000
@@ -246,6 +246,9 @@
     - name: broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled_in_pbm
       type: BOOLEAN
       mode: NULLABLE
+    - name: broken_site_report_tab_info_antitracking_btp_has_purged_site
+      type: BOOLEAN
+      mode: NULLABLE
   - name: labeled_counter
     type: RECORD
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/view.sql	2024-12-06 23:03:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/view.sql	2024-12-06 23:06:53.000000000 +0000
@@ -32,7 +32,8 @@
       metrics.boolean.broken_site_report_tab_info_frameworks_marfeel,
       metrics.boolean.broken_site_report_tab_info_frameworks_mobify,
       metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled,
-      metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled_in_pbm
+      metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled_in_pbm,
+      metrics.boolean.broken_site_report_tab_info_antitracking_btp_has_purged_site
     ) AS `boolean`,
     STRUCT(
       metrics.labeled_counter.glean_error_invalid_label,
@@ -121,7 +122,8 @@
       metrics.boolean.broken_site_report_tab_info_frameworks_marfeel,
       metrics.boolean.broken_site_report_tab_info_frameworks_mobify,
       metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled,
-      metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled_in_pbm
+      metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled_in_pbm,
+      metrics.boolean.broken_site_report_tab_info_antitracking_btp_has_purged_site
     ) AS `boolean`,
     STRUCT(
       metrics.labeled_counter.glean_error_invalid_label,
@@ -210,7 +212,8 @@
       metrics.boolean.broken_site_report_tab_info_frameworks_marfeel,
       metrics.boolean.broken_site_report_tab_info_frameworks_mobify,
       metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled,
-      metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled_in_pbm
+      metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled_in_pbm,
+      metrics.boolean.broken_site_report_tab_info_antitracking_btp_has_purged_site
     ) AS `boolean`,
     STRUCT(
       metrics.labeled_counter.glean_error_invalid_label,
@@ -299,7 +302,8 @@
       metrics.boolean.broken_site_report_tab_info_frameworks_marfeel,
       metrics.boolean.broken_site_report_tab_info_frameworks_mobify,
       metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled,
-      metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled_in_pbm
+      metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled_in_pbm,
+      metrics.boolean.broken_site_report_tab_info_antitracking_btp_has_purged_site
     ) AS `boolean`,
     STRUCT(
       metrics.labeled_counter.glean_error_invalid_label,
@@ -388,7 +392,8 @@
       metrics.boolean.broken_site_report_tab_info_frameworks_marfeel,
       metrics.boolean.broken_site_report_tab_info_frameworks_mobify,
       metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled,
-      metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled_in_pbm
+      metrics.boolean.broken_site_report_browser_info_prefs_third_party_cookie_blocking_enabled_in_pbm,
+      metrics.boolean.broken_site_report_tab_info_antitracking_btp_has_purged_site
     ) AS `boolean`,
     STRUCT(
       metrics.labeled_counter.glean_error_invalid_label,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2024-12-06 23:02:22.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2024-12-06 23:12:26.000000000 +0000
@@ -26,6 +26,9 @@
 - name: adjust_network
   type: STRING
   mode: NULLABLE
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: retained_week_2
   type: BOOLEAN
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2024-12-06 23:02:22.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2024-12-06 23:12:10.000000000 +0000
@@ -48,6 +48,10 @@
   description: 'The type of source of a client installation.
 
     '
+- name: install_source
+  type: STRING
+  mode: NULLABLE
+  description: null
 - name: new_profiles
   type: INTEGER
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml	2024-12-06 23:03:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml	2024-12-06 23:15:21.000000000 +0000
@@ -1,11 +1,11 @@
 fields:
 - name: normalized_app_id
-  mode: NULLABLE
   type: STRING
+  mode: NULLABLE
   description: App ID of the channel data was received from
 - name: normalized_channel
-  mode: NULLABLE
   type: STRING
+  mode: NULLABLE
   description: Normalized channel name
 - name: additional_properties
   type: STRING
@@ -2183,6 +2183,126 @@
         - name: value
           type: INTEGER
           mode: NULLABLE
+    - name: networking_http_1_download_throughput_100
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+    - name: networking_http_1_download_throughput_10_50
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+    - name: networking_http_1_download_throughput_50_100
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+    - name: networking_http_2_download_throughput_100
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+    - name: networking_http_2_download_throughput_10_50
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+    - name: networking_http_2_download_throughput_50_100
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
   - name: labeled_counter
     type: RECORD
     mode: NULLABLE
@@ -4277,6 +4397,16 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
+    - name: networking_https_http_or_local
+      type: RECORD
+      mode: REPEATED
+      fields:
+      - name: key
+        type: STRING
+        mode: NULLABLE
+      - name: value
+        type: INTEGER
+        mode: NULLABLE
   - name: quantity
     type: RECORD
     mode: NULLABLE
@@ -13081,6 +13211,82 @@
       type: RECORD
       mode: NULLABLE
       fields:
+      - name: bucket_count
+        type: INTEGER
+        mode: NULLABLE
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+      - name: histogram_type
+        type: STRING
+        mode: NULLABLE
+      - name: overflow
+        type: INTEGER
+        mode: NULLABLE
+      - name: range
+        type: FLOAT
+        mode: REPEATED
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: time_unit
+        type: STRING
+        mode: NULLABLE
+      - name: underflow
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+    - name: fingerprinting_protection_canvas_noise_calculate_time_ns
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: bucket_count
+        type: INTEGER
+        mode: NULLABLE
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+      - name: histogram_type
+        type: STRING
+        mode: NULLABLE
+      - name: overflow
+        type: INTEGER
+        mode: NULLABLE
+      - name: range
+        type: FLOAT
+        mode: REPEATED
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: time_unit
+        type: STRING
+        mode: NULLABLE
+      - name: underflow
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+    - name: fingerprinting_protection_canvas_noise_calculate_time_2
+      type: RECORD
+      mode: NULLABLE
+      fields:
       - name: bucket_count
         type: INTEGER
         mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/view.sql	2024-12-06 23:03:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/view.sql	2024-12-06 23:09:01.000000000 +0000
@@ -322,7 +322,13 @@
       metrics.custom_distribution.networking_http_3_udp_datagram_segments_received,
       metrics.custom_distribution.networking_http_3_loss_ratio,
       metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_received,
-      metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_sent
+      metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_sent,
+      metrics.custom_distribution.networking_http_1_download_throughput_100,
+      metrics.custom_distribution.networking_http_1_download_throughput_10_50,
+      metrics.custom_distribution.networking_http_1_download_throughput_50_100,
+      metrics.custom_distribution.networking_http_2_download_throughput_100,
+      metrics.custom_distribution.networking_http_2_download_throughput_10_50,
+      metrics.custom_distribution.networking_http_2_download_throughput_50_100
     ) AS `custom_distribution`,
     STRUCT(
       metrics.labeled_counter.crash_metrics_crash_count,
@@ -533,7 +539,8 @@
       metrics.labeled_counter.networking_http_3_quic_frame_count,
       metrics.labeled_counter.network_sso_entra_success,
       metrics.labeled_counter.networking_captive_portal_banner_display_time,
-      metrics.labeled_counter.networking_https_record_state
+      metrics.labeled_counter.networking_https_record_state,
+      metrics.labeled_counter.networking_https_http_or_local
     ) AS `labeled_counter`,
     STRUCT(
       metrics.quantity.gfx_adapter_primary_ram,
@@ -880,7 +887,9 @@
       metrics.timing_distribution.javascript_gc_max_pause,
       metrics.timing_distribution.javascript_gc_task_start_delay,
       metrics.timing_distribution.javascript_gc_time_between,
-      metrics.timing_distribution.javascript_gc_time_between_slices
+      metrics.timing_distribution.javascript_gc_time_between_slices,
+      metrics.timing_distribution.fingerprinting_protection_canvas_noise_calculate_time_ns,
+      metrics.timing_distribution.fingerprinting_protection_canvas_noise_calculate_time_2
     ) AS `timing_distribution`,
     STRUCT(
       metrics.memory_distribution.glean_upload_discarded_exceeding_pings_size,
@@ -1342,7 +1351,13 @@
       metrics.custom_distribution.networking_http_3_udp_datagram_segments_received,
       metrics.custom_distribution.networking_http_3_loss_ratio,
       metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_received,
-      metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_sent
+      metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_sent,
+      metrics.custom_distribution.networking_http_1_download_throughput_100,
+      metrics.custom_distribution.networking_http_1_download_throughput_10_50,
+      metrics.custom_distribution.networking_http_1_download_throughput_50_100,
+      metrics.custom_distribution.networking_http_2_download_throughput_100,
+      metrics.custom_distribution.networking_http_2_download_throughput_10_50,
+      metrics.custom_distribution.networking_http_2_download_throughput_50_100
     ) AS `custom_distribution`,
     STRUCT(
       metrics.labeled_counter.crash_metrics_crash_count,
@@ -1553,7 +1568,8 @@
       metrics.labeled_counter.networking_http_3_quic_frame_count,
       metrics.labeled_counter.network_sso_entra_success,
       metrics.labeled_counter.networking_captive_portal_banner_display_time,
-      metrics.labeled_counter.networking_https_record_state
+      metrics.labeled_counter.networking_https_record_state,
+      metrics.labeled_counter.networking_https_http_or_local
     ) AS `labeled_counter`,
     STRUCT(
       metrics.quantity.gfx_adapter_primary_ram,
@@ -1900,7 +1916,9 @@
       metrics.timing_distribution.javascript_gc_max_pause,
       metrics.timing_distribution.javascript_gc_task_start_delay,
       metrics.timing_distribution.javascript_gc_time_between,
-      metrics.timing_distribution.javascript_gc_time_between_slices
+      metrics.timing_distribution.javascript_gc_time_between_slices,
+      metrics.timing_distribution.fingerprinting_protection_canvas_noise_calculate_time_ns,
+      metrics.timing_distribution.fingerprinting_protection_canvas_noise_calculate_time_2
     ) AS `timing_distribution`,
     STRUCT(
       metrics.memory_distribution.glean_upload_discarded_exceeding_pings_size,
@@ -2380,7 +2398,13 @@
       metrics.custom_distribution.networking_http_3_udp_datagram_segments_received,
       metrics.custom_distribution.networking_http_3_loss_ratio,
       metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_received,
-      metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_sent
+      metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_sent,
+      metrics.custom_distribution.networking_http_1_download_throughput_100,
+      metrics.custom_distribution.networking_http_1_download_throughput_10_50,
+      metrics.custom_distribution.networking_http_1_download_throughput_50_100,
+      metrics.custom_distribution.networking_http_2_download_throughput_100,
+      metrics.custom_distribution.networking_http_2_download_throughput_10_50,
+      metrics.custom_distribution.networking_http_2_download_throughput_50_100
     ) AS `custom_distribution`,
     STRUCT(
       metrics.labeled_counter.crash_metrics_crash_count,
@@ -2591,7 +2615,8 @@
       metrics.labeled_counter.networking_http_3_quic_frame_count,
       metrics.labeled_counter.network_sso_entra_success,
       metrics.labeled_counter.networking_captive_portal_banner_display_time,
-      metrics.labeled_counter.networking_https_record_state
+      metrics.labeled_counter.networking_https_record_state,
+      metrics.labeled_counter.networking_https_http_or_local
     ) AS `labeled_counter`,
     STRUCT(
       metrics.quantity.gfx_adapter_primary_ram,
@@ -2938,7 +2963,9 @@
       metrics.timing_distribution.javascript_gc_max_pause,
       metrics.timing_distribution.javascript_gc_task_start_delay,
       metrics.timing_distribution.javascript_gc_time_between,
-      metrics.timing_distribution.javascript_gc_time_between_slices
+      metrics.timing_distribution.javascript_gc_time_between_slices,
+      metrics.timing_distribution.fingerprinting_protection_canvas_noise_calculate_time_ns,
+      metrics.timing_distribution.fingerprinting_protection_canvas_noise_calculate_time_2
     ) AS `timing_distribution`,
     STRUCT(
       metrics.memory_distribution.glean_upload_discarded_exceeding_pings_size,
@@ -3427,7 +3454,13 @@
       metrics.custom_distribution.networking_http_3_udp_datagram_segments_received,
       metrics.custom_distribution.networking_http_3_loss_ratio,
       metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_received,
-      metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_sent
+      metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_sent,
+      metrics.custom_distribution.networking_http_1_download_throughput_100,
+      metrics.custom_distribution.networking_http_1_download_throughput_10_50,
+      metrics.custom_distribution.networking_http_1_download_throughput_50_100,
+      metrics.custom_distribution.networking_http_2_download_throughput_100,
+      metrics.custom_distribution.networking_http_2_download_throughput_10_50,
+      metrics.custom_distribution.networking_http_2_download_throughput_50_100
     ) AS `custom_distribution`,
     STRUCT(
       metrics.labeled_counter.crash_metrics_crash_count,
@@ -3638,7 +3671,8 @@
       metrics.labeled_counter.networking_http_3_quic_frame_count,
       metrics.labeled_counter.network_sso_entra_success,
       metrics.labeled_counter.networking_captive_portal_banner_display_time,
-      metrics.labeled_counter.networking_https_record_state
+      metrics.labeled_counter.networking_https_record_state,
+      metrics.labeled_counter.networking_https_http_or_local
     ) AS `labeled_counter`,
     STRUCT(
       metrics.quantity.gfx_adapter_primary_ram,
@@ -3985,7 +4019,9 @@
       metrics.timing_distribution.javascript_gc_max_pause,
       metrics.timing_distribution.javascript_gc_task_start_delay,
       metrics.timing_distribution.javascript_gc_time_between,
-      metrics.timing_distribution.javascript_gc_time_between_slices
+      metrics.timing_distribution.javascript_gc_time_between_slices,
+      metrics.timing_distribution.fingerprinting_protection_canvas_noise_calculate_time_ns,
+      metrics.timing_distribution.fingerprinting_protection_canvas_noise_calculate_time_2
     ) AS `timing_distribution`,
     STRUCT(
       metrics.memory_distribution.glean_upload_discarded_exceeding_pings_size,
@@ -4456,7 +4492,13 @@
       metrics.custom_distribution.networking_http_3_udp_datagram_segments_received,
       metrics.custom_distribution.networking_http_3_loss_ratio,
       metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_received,
-      metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_sent
+      metrics.custom_distribution.networking_http_3_ecn_ce_ect0_ratio_sent,
+      metrics.custom_distribution.networking_http_1_download_throughput_100,
+      metrics.custom_distribution.networking_http_1_download_throughput_10_50,
+      metrics.custom_distribution.networking_http_1_download_throughput_50_100,
+      metrics.custom_distribution.networking_http_2_download_throughput_100,
+      metrics.custom_distribution.networking_http_2_download_throughput_10_50,
+      metrics.custom_distribution.networking_http_2_download_throughput_50_100
     ) AS `custom_distribution`,
     STRUCT(
       metrics.labeled_counter.crash_metrics_crash_count,
@@ -4667,7 +4709,8 @@
       metrics.labeled_counter.networking_http_3_quic_frame_count,
       metrics.labeled_counter.network_sso_entra_success,
       metrics.labeled_counter.networking_captive_portal_banner_display_time,
-      metrics.labeled_counter.networking_https_record_state
+      metrics.labeled_counter.networking_https_record_state,
+      metrics.labeled_counter.networking_https_http_or_local
     ) AS `labeled_counter`,
     STRUCT(
       metrics.quantity.gfx_adapter_primary_ram,
@@ -5014,7 +5057,9 @@
       metrics.timing_distribution.javascript_gc_max_pause,
       metrics.timing_distribution.javascript_gc_task_start_delay,
       metrics.timing_distribution.javascript_gc_time_between,
-      metrics.timing_distribution.javascript_gc_time_between_slices
+      metrics.timing_distribution.javascript_gc_time_between_slices,
+      metrics.timing_distribution.fingerprinting_protection_canvas_noise_calculate_time_ns,
+      metrics.timing_distribution.fingerprinting_protection_canvas_noise_calculate_time_2
     ) AS `timing_distribution`,
     STRUCT(
       metrics.memory_distribution.glean_upload_discarded_exceeding_pings_size,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/user_characteristics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/user_characteristics/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/user_characteristics/schema.yaml	2024-12-06 23:03:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/user_characteristics/schema.yaml	2024-12-06 23:13:19.000000000 +0000
@@ -732,6 +732,30 @@
     - name: characteristics_gl_vertex_shader_software
       type: STRING
       mode: NULLABLE
+    - name: characteristics_canvasdata3avif
+      type: STRING
+      mode: NULLABLE
+    - name: characteristics_canvasdata3avifsoftware
+      type: STRING
+      mode: NULLABLE
+    - name: characteristics_canvasdata3cspsafe
+      type: STRING
+      mode: NULLABLE
+    - name: characteristics_canvasdata3cspsafesoftware
+      type: STRING
+      mode: NULLABLE
+    - name: characteristics_canvasdata3png
+      type: STRING
+      mode: NULLABLE
+    - name: characteristics_canvasdata3pngsoftware
+      type: STRING
+      mode: NULLABLE
+    - name: characteristics_canvasdata3webp
+      type: STRING
+      mode: NULLABLE
+    - name: characteristics_canvasdata3webpsoftware
+      type: STRING
+      mode: NULLABLE
   - name: string_list
     type: RECORD
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/user_characteristics/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/user_characteristics/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/user_characteristics/view.sql	2024-12-06 23:03:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/user_characteristics/view.sql	2024-12-06 23:08:47.000000000 +0000
@@ -203,7 +203,15 @@
       metrics.string.characteristics_gl_vendor_raw_software,
       metrics.string.characteristics_gl_vendor_software,
       metrics.string.characteristics_gl_version_raw_software,
-      metrics.string.characteristics_gl_vertex_shader_software
+      metrics.string.characteristics_gl_vertex_shader_software,
+      metrics.string.characteristics_canvasdata3avif,
+      metrics.string.characteristics_canvasdata3avifsoftware,
+      metrics.string.characteristics_canvasdata3cspsafe,
+      metrics.string.characteristics_canvasdata3cspsafesoftware,
+      metrics.string.characteristics_canvasdata3png,
+      metrics.string.characteristics_canvasdata3pngsoftware,
+      metrics.string.characteristics_canvasdata3webp,
+      metrics.string.characteristics_canvasdata3webpsoftware
     ) AS `string`,
     STRUCT(metrics.string_list.characteristics_gamepads) AS `string_list`,
     STRUCT(
@@ -465,7 +473,15 @@
       metrics.string.characteristics_gl_vendor_raw_software,
       metrics.string.characteristics_gl_vendor_software,
       metrics.string.characteristics_gl_version_raw_software,
-      metrics.string.characteristics_gl_vertex_shader_software
+      metrics.string.characteristics_gl_vertex_shader_software,
+      metrics.string.characteristics_canvasdata3avif,
+      metrics.string.characteristics_canvasdata3avifsoftware,
+      metrics.string.characteristics_canvasdata3cspsafe,
+      metrics.string.characteristics_canvasdata3cspsafesoftware,
+      metrics.string.characteristics_canvasdata3png,
+      metrics.string.characteristics_canvasdata3pngsoftware,
+      metrics.string.characteristics_canvasdata3webp,
+      metrics.string.characteristics_canvasdata3webpsoftware
     ) AS `string`,
     STRUCT(metrics.string_list.characteristics_gamepads) AS `string_list`,
     STRUCT(
@@ -727,7 +743,15 @@
       metrics.string.characteristics_gl_vendor_raw_software,
       metrics.string.characteristics_gl_vendor_software,
       metrics.string.characteristics_gl_version_raw_software,
-      metrics.string.characteristics_gl_vertex_shader_software
+      metrics.string.characteristics_gl_vertex_shader_software,
+      metrics.string.characteristics_canvasdata3avif,
+      metrics.string.characteristics_canvasdata3avifsoftware,
+      metrics.string.characteristics_canvasdata3cspsafe,
+      metrics.string.characteristics_canvasdata3cspsafesoftware,
+      metrics.string.characteristics_canvasdata3png,
+      metrics.string.characteristics_canvasdata3pngsoftware,
+      metrics.string.characteristics_canvasdata3webp,
+      metrics.string.characteristics_canvasdata3webpsoftware
     ) AS `string`,
     STRUCT(metrics.string_list.characteristics_gamepads) AS `string_list`,
     STRUCT(
@@ -989,7 +1013,15 @@
       metrics.string.characteristics_gl_vendor_raw_software,
       metrics.string.characteristics_gl_vendor_software,
       metrics.string.characteristics_gl_version_raw_software,
-      metrics.string.characteristics_gl_vertex_shader_software
+      metrics.string.characteristics_gl_vertex_shader_software,
+      metrics.string.characteristics_canvasdata3avif,
+      metrics.string.characteristics_canvasdata3avifsoftware,
+      metrics.string.characteristics_canvasdata3cspsafe,
+      metrics.string.characteristics_canvasdata3cspsafesoftware,
+      metrics.string.characteristics_canvasdata3png,
+      metrics.string.characteristics_canvasdata3pngsoftware,
+      metrics.string.characteristics_canvasdata3webp,
+      metrics.string.characteristics_canvasdata3webpsoftware
     ) AS `string`,
     STRUCT(metrics.string_list.characteristics_gamepads) AS `string_list`,
     STRUCT(
@@ -1251,7 +1283,15 @@
       metrics.string.characteristics_gl_vendor_raw_software,
       metrics.string.characteristics_gl_vendor_software,
       metrics.string.characteristics_gl_version_raw_software,
-      metrics.string.characteristics_gl_vertex_shader_software
+      metrics.string.characteristics_gl_vertex_shader_software,
+      metrics.string.characteristics_canvasdata3avif,
+      metrics.string.characteristics_canvasdata3avifsoftware,
+      metrics.string.characteristics_canvasdata3cspsafe,
+      metrics.string.characteristics_canvasdata3cspsafesoftware,
+      metrics.string.characteristics_canvasdata3png,
+      metrics.string.characteristics_canvasdata3pngsoftware,
+      metrics.string.characteristics_canvasdata3webp,
+      metrics.string.characteristics_canvasdata3webpsoftware
     ) AS `string`,
     STRUCT(metrics.string_list.characteristics_gamepads) AS `string_list`,
     STRUCT(
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:03:10.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:06:02.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-28"
+    DATE(submission_timestamp) >= "2024-12-06"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/broken_site_report/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/broken_site_report/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/broken_site_report/schema.yaml	2024-12-06 23:03:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/broken_site_report/schema.yaml	2024-12-06 23:04:30.000000000 +0000
@@ -352,6 +352,13 @@
       description: 'Value of `network.cookie.cookieBehavior.optInPartitioning.pbmode`
 
         '
+    - name: broken_site_report_tab_info_antitracking_btp_has_purged_site
+      type: BOOLEAN
+      mode: NULLABLE
+      description: 'Whether the site in the current tab has recently been purged by
+        Bounce Tracking Protection.
+
+        '
   - name: labeled_counter
     type: RECORD
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml	2024-12-06 23:02:22.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml	2024-12-06 23:12:03.000000000 +0000
@@ -1,64 +1,66 @@
 fields:
-- description: Unique ID for the client installation.
-  mode: NULLABLE
-  name: client_id
+- name: client_id
   type: STRING
-- description: Sample ID - A number ranging from 0 - 99 based on client ID; used to pull a small sample of data related to a subset of clients over time
   mode: NULLABLE
-  name: sample_id
-  type: INT64
-- description: Submission Date
+  description: Unique ID for the client installation.
+- name: sample_id
+  type: INTEGER
   mode: NULLABLE
-  name: submission_date
+  description: Sample ID - A number ranging from 0 - 99 based on client ID; used to
+    pull a small sample of data related to a subset of clients over time
+- name: submission_date
   type: DATE
-- description: First Seen Date - The date this client was first seen
   mode: NULLABLE
-  name: first_seen_date
+  description: Submission Date
+- name: first_seen_date
   type: DATE
-- description: Days Since First Seen - The number of days since the client was first seen
   mode: NULLABLE
-  name: days_since_first_seen
-  type: INT64
-- description: Days Since Active
+  description: First Seen Date - The date this client was first seen
+- name: days_since_first_seen
+  type: INTEGER
   mode: NULLABLE
-  name: days_since_active
-  type: INT64
-- description: First Reported Country - The country this client ID was first reported from
+  description: Days Since First Seen - The number of days since the client was first
+    seen
+- name: days_since_active
+  type: INTEGER
   mode: NULLABLE
-  name: first_reported_country
+  description: Days Since Active
+- name: first_reported_country
   type: STRING
-- description: Attribution
   mode: NULLABLE
-  name: attribution
+  description: First Reported Country - The country this client ID was first reported
+    from
+- name: attribution
   type: RECORD
+  mode: NULLABLE
   fields:
-  - mode: NULLABLE
-    name: source
+  - name: source
     type: STRING
+    mode: NULLABLE
     description: Attribution Source
-  - mode: NULLABLE
-    name: medium
+  - name: medium
     type: STRING
+    mode: NULLABLE
     description: Attribution Medium
-  - mode: NULLABLE
-    name: campaign
+  - name: campaign
     type: STRING
+    mode: NULLABLE
     description: Attribution Campaign
-  - mode: NULLABLE
-    name: content
+  - name: content
     type: STRING
+    mode: NULLABLE
     description: Attribution Content
-  - mode: NULLABLE
-    name: experiment
+  - name: experiment
     type: STRING
+    mode: NULLABLE
     description: Attribution Experiment
-  - mode: NULLABLE
-    name: variation
+  - name: variation
     type: STRING
+    mode: NULLABLE
     description: Attribution Variation
-  - mode: NULLABLE
-    name: dltoken
+  - name: dltoken
     type: STRING
+    mode: NULLABLE
     description: Attribution Download Token
   - name: dlsource
     type: STRING
@@ -68,40 +70,43 @@
     type: STRING
     mode: NULLABLE
     description: Attribution UA
-- description: Active
-  mode: NULLABLE
-  name: active
-  type: INT64
-- description: Ad Clicks - The number of ad clicks from this client on the submission date
+  description: Attribution
+- name: active
+  type: INTEGER
   mode: NULLABLE
-  name: ad_clicks
-  type: INT64
-- description: Total Historic Ad Clicks - The number of ad clicks from this client on or before the submission date
+  description: Active
+- name: ad_clicks
+  type: INTEGER
   mode: NULLABLE
-  name: total_historic_ad_clicks
-  type: INT64
-- description: Days Seen Bytes
+  description: Ad Clicks - The number of ad clicks from this client on the submission
+    date
+- name: total_historic_ad_clicks
+  type: INTEGER
   mode: NULLABLE
-  name: days_seen_bytes
+  description: Total Historic Ad Clicks - The number of ad clicks from this client
+    on or before the submission date
+- name: days_seen_bytes
   type: BYTES
-- description: Pattern
   mode: NULLABLE
-  name: pattern
+  description: Days Seen Bytes
+- name: pattern
   type: INTEGER
-- description: Death Time
   mode: NULLABLE
-  name: death_time
+  description: Pattern
+- name: death_time
   type: INTEGER
-- description: Max Days
   mode: NULLABLE
-  name: max_days
+  description: Death Time
+- name: max_days
   type: INTEGER
-- description: Markov States
   mode: NULLABLE
-  name: markov_states
+  description: Max Days
+- name: markov_states
   type: RECORD
-  fields:
-  - description: Desktop States V1
     mode: NULLABLE
-    name: desktop_states_v1
+  fields:
+  - name: desktop_states_v1
     type: STRING
+    mode: NULLABLE
+    description: Desktop States V1
+  description: Markov States
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml	2024-12-06 23:03:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml	2024-12-06 23:04:30.000000000 +0000
@@ -7145,6 +7145,20 @@
         - "others"
 
         '
+    - name: networking_https_http_or_local
+      type: RECORD
+      mode: REPEATED
+      fields:
+      - name: key
+        type: STRING
+        mode: NULLABLE
+      - name: value
+        type: INTEGER
+        mode: NULLABLE
+      description: 'Whether a top-level request is HTTPS, HTTP, or HTTP for local
+        domain
+
+        '
   - name: memory_distribution
     type: RECORD
     mode: NULLABLE
@@ -15959,6 +15973,92 @@
       description: 'Time spent in between garbage collections for the main runtime.
 
         '
+    - name: fingerprinting_protection_canvas_noise_calculate_time_ns
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: bucket_count
+        type: INTEGER
+        mode: NULLABLE
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+        description: This was accidentally sent in the past and is now deprecated.
+          See https://bugzilla.mozilla.org/show_bug.cgi?id=1799509#c5
+      - name: histogram_type
+        type: STRING
+        mode: NULLABLE
+      - name: overflow
+        type: INTEGER
+        mode: NULLABLE
+      - name: range
+        type: FLOAT
+        mode: REPEATED
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: time_unit
+        type: STRING
+        mode: NULLABLE
+      - name: underflow
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+      description: 'Counts how long to generate canvas random noises.
+
+        '
+    - name: fingerprinting_protection_canvas_noise_calculate_time_2
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: bucket_count
+        type: INTEGER
+        mode: NULLABLE
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+        description: This was accidentally sent in the past and is now deprecated.
+          See https://bugzilla.mozilla.org/show_bug.cgi?id=1799509#c5
+      - name: histogram_type
+        type: STRING
+        mode: NULLABLE
+      - name: overflow
+        type: INTEGER
+        mode: NULLABLE
+      - name: range
+        type: FLOAT
+        mode: REPEATED
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: time_unit
+        type: STRING
+        mode: NULLABLE
+      - name: underflow
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+      description: 'Counts how long to generate canvas random noises.
+
+        '
   - name: quantity
     type: RECORD
     mode: NULLABLE
@@ -18409,6 +18509,162 @@
       description: 'HTTP3: ECN CE to ECT0 ratio (multiply by 10000) sent.
 
         '
+    - name: networking_http_1_download_throughput_100
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+        description: This was accidentally sent in the past and is now deprecated.
+          See https://bugzilla.mozilla.org/show_bug.cgi?id=1799509#c5
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+      description: 'The download throughput for http/1.0 and http/1.1 requests size
+        larger than 100MB. Measured in megabits per second, Mbps.
+
+        '
+    - name: networking_http_1_download_throughput_10_50
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+        description: This was accidentally sent in the past and is now deprecated.
+          See https://bugzilla.mozilla.org/show_bug.cgi?id=1799509#c5
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+      description: 'The download throughput for http/1.0 and http/1.1 requests between
+        10MB and 50MB. Measured in megabits per second, Mbps.
+
+        '
+    - name: networking_http_1_download_throughput_50_100
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+        description: This was accidentally sent in the past and is now deprecated.
+          See https://bugzilla.mozilla.org/show_bug.cgi?id=1799509#c5
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+      description: 'The download throughput for http/1.0 and http/1.1 requests between
+        50MB and 100MB. Measured in megabits per second, Mbps.
+
+        '
+    - name: networking_http_2_download_throughput_100
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+        description: This was accidentally sent in the past and is now deprecated.
+          See https://bugzilla.mozilla.org/show_bug.cgi?id=1799509#c5
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+      description: 'The download throughput for http/2 requests size larger than 100MB.
+        Measured in megabits per second, Mbps.
+
+        '
+    - name: networking_http_2_download_throughput_10_50
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+        description: This was accidentally sent in the past and is now deprecated.
+          See https://bugzilla.mozilla.org/show_bug.cgi?id=1799509#c5
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+      description: 'The download throughput for http/2 requests between 10MB and 50MB.
+        Measured in megabits per second, Mbps.
+
+        '
+    - name: networking_http_2_download_throughput_50_100
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+        description: This was accidentally sent in the past and is now deprecated.
+          See https://bugzilla.mozilla.org/show_bug.cgi?id=1799509#c5
+      - name: sum
+        type: INTEGER
+        mode: NULLABLE
+      - name: values
+        type: RECORD
+        mode: REPEATED
+        fields:
+        - name: key
+          type: STRING
+          mode: NULLABLE
+        - name: value
+          type: INTEGER
+          mode: NULLABLE
+      description: 'The download throughput for http/2 requests between 50MB and 100MB.
+        Measured in megabits per second, Mbps.
+
+        '
   - name: url2
     type: RECORD
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml	2024-12-06 23:02:22.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml	2024-12-06 23:12:12.000000000 +0000
@@ -1,48 +1,47 @@
 fields:
-- description: Submission Timestamp
-  mode: NULLABLE
-  name: submission_timestamp
+- name: submission_timestamp
   type: TIMESTAMP
-- description: Normalized Country Code, Examples - US, AR, BR, etc.
   mode: NULLABLE
-  name: normalized_country_code
+  description: Submission Timestamp
+- name: normalized_country_code
   type: STRING
-- description: Normalized Channel, Examples - release, nightly, aurora, esr, beta
   mode: NULLABLE
-  name: normalized_channel
+  description: Normalized Country Code, Examples - US, AR, BR, etc.
+- name: normalized_channel
   type: STRING
-- description: Document ID
   mode: NULLABLE
-  name: document_id
+  description: Normalized Channel, Examples - release, nightly, aurora, esr, beta
+- name: document_id
   type: STRING
-- description: Pocket Enabled
   mode: NULLABLE
-  name: pocket_enabled
+  description: Document ID
+- name: pocket_enabled
   type: BOOLEAN
-- description: Pocket Sponsored Stories Enabled
   mode: NULLABLE
-  name: pocket_sponsored_stories_enabled
+  description: Pocket Enabled
+- name: pocket_sponsored_stories_enabled
   type: BOOLEAN
-- description: Newtab Locale
   mode: NULLABLE
-  name: newtab_locale
+  description: Pocket Sponsored Stories Enabled
+- name: newtab_locale
   type: STRING
-- description: App Build
   mode: NULLABLE
-  name: app_build
+  description: Newtab Locale
+- name: app_build
   type: STRING
-- description: App Display Version
   mode: NULLABLE
-  name: app_display_version
+  description: App Build
+- name: app_display_version
   type: STRING
-- description: Client ID
   mode: NULLABLE
-  name: client_id
+  description: App Display Version
+- name: client_id
   type: STRING
+  mode: NULLABLE
+  description: Client ID
 - name: events
   type: RECORD
   mode: REPEATED
-  description: Events
   fields:
   - name: category
     type: STRING
@@ -51,7 +50,6 @@
   - name: extra
     type: RECORD
     mode: REPEATED
-    description: Extras
     fields:
     - name: key
       type: STRING
@@ -61,6 +59,7 @@
       type: STRING
       mode: NULLABLE
       description: Value
+    description: Extras
   - name: name
     type: STRING
     mode: NULLABLE
@@ -69,3 +68,4 @@
     type: INTEGER
     mode: NULLABLE
     description: Event Timestamp
+  description: Events
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/user_characteristics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/user_characteristics/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/user_characteristics/schema.yaml	2024-12-06 23:03:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/user_characteristics/schema.yaml	2024-12-06 23:04:18.000000000 +0000
@@ -1359,6 +1359,62 @@
       description: 'Hash of the transformed source of the vertex shader.
 
         '
+    - name: characteristics_canvasdata3avif
+      type: STRING
+      mode: NULLABLE
+      description: 'The sha1 hash of an image rendered on a canvas. The image is a
+        stretched photographic image, used to test image operations.
+
+        '
+    - name: characteristics_canvasdata3avifsoftware
+      type: STRING
+      mode: NULLABLE
+      description: 'The sha1 hash of an image rendered on a canvas. The image is a
+        stretched photographic image, used to test image operations.
+
+        '
+    - name: characteristics_canvasdata3cspsafe
+      type: STRING
+      mode: NULLABLE
+      description: 'The sha1 hash of an image rendered on a canvas. The image is a
+        stretched photographic image, used to test image operations.
+
+        '
+    - name: characteristics_canvasdata3cspsafesoftware
+      type: STRING
+      mode: NULLABLE
+      description: 'The sha1 hash of an image rendered on a canvas. The image is a
+        stretched photographic image, used to test image operations.
+
+        '
+    - name: characteristics_canvasdata3png
+      type: STRING
+      mode: NULLABLE
+      description: 'The sha1 hash of an image rendered on a canvas. The image is a
+        stretched photographic image, used to test image operations.
+
+        '
+    - name: characteristics_canvasdata3pngsoftware
+      type: STRING
+      mode: NULLABLE
+      description: 'The sha1 hash of an image rendered on a canvas. The image is a
+        stretched photographic image, used to test image operations.
+
+        '
+    - name: characteristics_canvasdata3webp
+      type: STRING
+      mode: NULLABLE
+      description: 'The sha1 hash of an image rendered on a canvas. The image is a
+        stretched photographic image, used to test image operations.
+
+        '
+    - name: characteristics_canvasdata3webpsoftware
+      type: STRING
+      mode: NULLABLE
+      description: 'The sha1 hash of an image rendered on a canvas. The image is a
+        stretched photographic image, used to test image operations.
+
+        '
   - name: string_list
     type: RECORD
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:03:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:06:03.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-28"
+    DATE(submission_timestamp) >= "2024-12-06"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:03:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql	2024-12-06 23:06:03.000000000 +0000
@@ -77,7 +77,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-28"
+    DATE(submission_timestamp) >= "2024-12-06"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/broken_site_report/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/broken_site_report/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/broken_site_report/schema.yaml	2024-12-06 23:03:10.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/broken_site_report/schema.yaml	2024-12-06 23:04:15.000000000 +0000
@@ -352,6 +352,13 @@
       description: 'Value of `network.cookie.cookieBehavior.optInPartitioning.pbmode`
 
         '
+    - name: broken_site_report_tab_info_antitracking_btp_has_purged_site
+      type: BOOLEAN
+      mode: NULLABLE
+      description: 'Whether the site in the current tab has recently been purged by
+        Bounce Tracking Protection.
+
+        '
   - name: labeled_counter
     type: RECORD
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/metrics/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/metrics/schema.yaml	2024-12-06 23:03:10.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/metrics/schema.yaml	2024-12-06 23:04:19.000000000 +0000
@@ -4067,6 +4067,20 @@
         - "others"
 
         '
+    - name: networking_https_http_or_local
+      type: RECORD
+      mode: REPEATED
+      fields:
+      - name: key
+        type: STRING
+        mode: NULLABLE
+      - name: value
+        type: INTEGER
+        mode: NULLABLE
+      description: 'Whether a top-level request is HTTPS, HTTP, or HTTP for local
+        domain
+
+        '
   - name: memory_distribution
     type: RECORD
     mode: NULLABLE
@@ -6157,6 +6171,162 @@
       description: 'HTTP3: ECN CE to ECT0 ratio (multiply by 10000) sent.
 
         '
+    - name: networking_http_1_download_throughput_100
+      type: RECORD
+      mode: NULLABLE
+      fields:
+      - name: count
+        type: INTEGER
+        mode: NULLABLE
+        description: This was accidentally sent in the past and is now deprecated.
+          See https://bugzilla.m

⚠️ Only part of the diff is displayed.

Link to full diff

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kik-kik kik-kik Awaiting requested review from kik-kik

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@gkatre @dataops-ci-bot @kik-kik