Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RDV: Respect overriding preference for Untangling Calypso Settings #99388

Merged
merged 13 commits into from
Feb 10, 2025

Conversation

okmttdhr
Copy link
Member

@okmttdhr okmttdhr commented Feb 6, 2025

Related to #99361

Proposed Changes

This PR is a follow-up to #99361 and introduces the following changes:

  • Ensure RDV override (remove_duplicate_views_experiment_assignment_160125) is respected across all places where RDV experiment is used by creating the selector getIsRemoveDuplicateViewsExperimentOverride.
  • Ensures that the A/A test (calypso_post_onboarding_aa_150125) is called exactly the same number of times as calypso_post_onboarding_holdout_160125

Business logic mainly resides in client/lib/remove-duplicate-views-experiment/index.ts with key functions structured as follows:

  • useRemoveDuplicateViewsExperimentEnabled
    • Hook for use in function components.
  • isRemoveDuplicateViewsExperimentEnabled
    • Utility function used in controllers.
  • loadRemoveDuplicateViewsExperimentAssignment
    • Loads the assignment from ExPlat.
    • Used in isRemoveDuplicateViewsExperimentEnabled
    • Used in the redux action (getRemoveDuplicateViewsExperimentAssignment). This redux action is used in class components and useRemoveDuplicateViewsExperimentEnabled.

Why are these changes being made?

p1738769811530729/1738766421.687729-slack-C029GN3KD

Testing Instructions

Escape hatched users regression test:

  • Prepare a non-a8c-user and add them to "escape hatch user" by following p7DVsv-m73-p2
  • Prepare a Default interface site
  • Go to /sites
    • Select the site
    • On Simple, observe no Settings tab
    • On Atomic, observe the Server Settings tab instead of the Settings tab
      • Click the Server Settings tab
      • Observe no breaks
  • Go to /home and click Settings > General
    • Observe the Calypso version of the settings page (/settings/general)
  • You revert to treatment by running the same query that was used to assign it to control.

Non-escape hatched users regression test:

  • Reset your overridden assignment by using the Console:
    • Set calypso_preferences {"remove_duplicate_views_experiment_assignment_160125":null}
    • Screenshot 2025-02-07 at 11 49 55
  • Assign yourself to treatment via 22167-explat-experiment
  • Go to /sites
    • Select the site
    • Observe the Settings tab
  • Go to /home and click Settings > General
    • Observe that you're redirected to wp-admin
  • Assign yourself to control via 22167-explat-experiment
  • Observe the same result as escape hatched users

Pre-merge Checklist

  • Has the general commit checklist been followed? (PCYsg-hS-p2)
  • Have you written new tests for your changes?
  • Have you tested the feature in Simple (P9HQHe-k8-p2), Atomic (P9HQHe-jW-p2), and self-hosted Jetpack sites (PCYsg-g6b-p2)?
  • Have you checked for TypeScript, React or other console errors?
  • Have you used memoizing on expensive computations? More info in Memoizing with create-selector and Using memoizing selectors and Our Approach to Data
  • Have we added the "[Status] String Freeze" label as soon as any new strings were ready for translation (p4TIVU-5Jq-p2)?
    • For UI changes, have we tested the change in various languages (for example, ES, PT, FR, or DE)? The length of text and words vary significantly between languages.
  • For changes affecting Jetpack: Have we added the "[Status] Needs Privacy Updates" label if this pull request changes what data or activity we track or use (p4TIVU-aUh-p2)?

@matticbot
Copy link
Contributor

matticbot commented Feb 6, 2025

This PR modifies the release build for the following Calypso Apps:

For info about this notification, see here: PCYsg-OT6-p2

  • notifications
  • wpcom-block-editor

To test WordPress.com changes, run install-plugin.sh $pluginSlug update/rdv-experiment-override-with-state on your sandbox.

@okmttdhr okmttdhr changed the title RDV: Respect overriding preference RDV: Respect overriding preference for Untangling Calypso Settings Feb 6, 2025
@matticbot
Copy link
Contributor

matticbot commented Feb 6, 2025

Here is how your PR affects size of JS and CSS bundles shipped to the user's browser:

App Entrypoints (~229 bytes added 📈 [gzipped])

name                 parsed_size           gzip_size
entry-main               +1215 B  (+0.1%)     +229 B  (+0.0%)
entry-login              +1215 B  (+0.1%)     +229 B  (+0.0%)
entry-subscriptions        +46 B  (+0.0%)      +25 B  (+0.0%)
entry-stepper              +46 B  (+0.0%)      +25 B  (+0.0%)

Common code that is always downloaded and parsed every time the app is loaded, no matter which route is used.

Sections (~14 bytes removed 📉 [gzipped])

name                  parsed_size           gzip_size
hosting                    -317 B  (-0.0%)      -68 B  (-0.0%)
site-settings              -308 B  (-0.0%)      -64 B  (-0.0%)
settings                   -209 B  (-0.0%)      -61 B  (-0.0%)
settings-writing            +57 B  (+0.0%)       +4 B  (+0.0%)
settings-security           +57 B  (+0.0%)       +7 B  (+0.0%)
settings-reading            +57 B  (+0.0%)       +5 B  (+0.0%)
settings-podcast            +57 B  (+0.0%)       +4 B  (+0.0%)
settings-performance        +57 B  (+0.0%)       +6 B  (+0.0%)
settings-newsletter         +57 B  (+0.0%)       +4 B  (+0.0%)
settings-jetpack            +57 B  (+0.0%)       +3 B  (+0.0%)
settings-discussion         +57 B  (+0.0%)       +9 B  (+0.0%)
marketing                   -45 B  (-0.0%)       +3 B  (+0.0%)
site-marketing              -44 B  (-0.0%)       +3 B  (+0.0%)

Sections contain code specific for a given set of routes. Is downloaded and parsed only when a particular route is navigated to.

Legend

What is parsed and gzip size?

Parsed Size: Uncompressed size of the JS and CSS files. This much code needs to be parsed and stored in memory.
Gzip Size: Compressed size of the JS and CSS files. This much data needs to be downloaded over network.

Generated by performance advisor bot at iscalypsofastyet.com.

@okmttdhr okmttdhr self-assigned this Feb 6, 2025
@okmttdhr okmttdhr force-pushed the update/rdv-experiment-override-with-state branch from 567fb65 to 5a79c7f Compare February 6, 2025 08:26
@okmttdhr okmttdhr marked this pull request as ready for review February 7, 2025 03:19
@okmttdhr okmttdhr requested a review from a team February 7, 2025 03:20
@matticbot matticbot added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Feb 7, 2025
@okmttdhr okmttdhr requested review from dsas and mmtr February 7, 2025 03:21
Copy link
Contributor

@fushar fushar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read the changes (it makes sense) and tested everything except this case:

Non-escape hatched users regression test, in control group

@okmttdhr okmttdhr merged commit 3fc3ec8 into trunk Feb 10, 2025
19 of 24 checks passed
@okmttdhr okmttdhr deleted the update/rdv-experiment-override-with-state branch February 10, 2025 05:28
@github-actions github-actions bot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Feb 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants