feat: removed verbatim ff #4350 (#4363)

Author: MillenniumFalconMechanic

app/components/Export/components/AnVILExplorer/ExportMethod/exportMethod.tsx

@@ -11,7 +11,7 @@ We are excited to introduce users of the [AnVIL Portal]({portalURL}) to the [AnV
 
 ## What is the AnVIL Data Explorer?
 
-Until now, the way to browse datasets through the AnVIL portal has been to use the [AnVIL Dataset Catalog]({portalURL}/data/consortia), which allows you to organize your search results based on the workspaces that contain a subset of a dataset by its consent code. The AnVIL Dataset Catalog provides summary level information of the dataset and study.
+Until now, the way to browse datasets through the AnVIL Portal has been to use the [AnVIL Dataset Catalog]({portalURL}/data/consortia), which allows you to organize your search results based on the workspaces that contain a subset of a dataset by its consent code. The AnVIL Dataset Catalog provides summary level information of the dataset and study.
 
 With the addition of the [AnVIL Data Explorer]({browserURL}), you’ll be able to sort the datasets you’re browsing based on the following managed access categories:
 
@@ -31,7 +31,7 @@ When working with NIH data, particularly managed access data, users must follow
 
 ## How do I use the AnVIL Data Explorer?
 
-Below you’ll find step-by-step instructions for navigating the AnVIL Data Explorer and exporting data to Terra. In order to successfully import data from the AnVIL Data Explorer to Terra, you’ll need to make your selection in the AnVIL Data Explorer, export this selection to a Terra workspace, and (as an interim stopgap) run a final step in the form of a Jupyter Notebook in order to retrieve all of the data relevant to your selection.
+Below you’ll find step-by-step instructions for navigating the AnVIL Data Explorer and exporting data to Terra.
 
 ### Step 1: Finding the AnVIL Data Explorer
 
@@ -61,37 +61,20 @@ The facets are visible in the screen above in the column on the left. If you cli
 
 When you select multiple facets, only data matching all selected facets is displayed (e.g. filter by Anatomical Site AND BioSample Type). When you select multiple values within a facet, data matching any of the facet values is displayed (e.g. selecting both Blood and Tissue above will list studies that include Blood OR Tissue samples).
 
-#### Exploring Studies
+#### Exploring Datasets
 
-Note: Currently the summary pages of studies in the AnVIL Data Explorer are not populated with information, yet the information described below will be populated for all of the studies as development progresses.
+When you click on a dataset, you’ll be taken a summary page where you can find a variety of information and helpful links, including but not limited to:
 
-When you click on a study, you’ll be taken a summary page where you can find a variety of information and helpful links, including but not limited to:
-
-- What consortium the data is associated with
+- What consortium the dataset is associated with
 - The quantity and types of data
 - Links to APIs for accessing the data programmatically
 - Links to request access
-- A button for exporting the data to a Terra workspace
+- A button for exporting the dataset to a Terra workspace
 
-<Figure alt="Exploring Studies" src="/guides/exploring-studies.png" />
+<Figure alt="Exploring Datasets" src="/guides/exploring-studies.png" />
 
 ### Step 4: Exporting Data
 
-<Alert severity="warning">
-  <AlertTitle>Note</AlertTitle>
-
-The format of exported study data is currently being updated. Exporting study
-data is temporarily disabled by default while this work is in progress. Export
-will be reenabled when the format is stable and working well.
-
-</Alert>
-
-To use data you’ve found through the AnVIL Data Explorer within your Terra workspace, you’ll need to follow two steps:
-
-First, you export the data from the AnVIL Data Explorer to Terra.
-
-Second, currently, you will need to use a publicly available notebook to fill in some of the metadata that doesn’t populate automatically.
-
 #### Exporting from The AnVIL Data Explorer
 
 Once you’re ready to export the data, you can click Export to Terra from within a particular study, or you can also click the Export button at the top right of your screen when you are on the AnVIL Data Explorer’s main page.
@@ -113,15 +96,13 @@ After you click this button, you will be prompted to wait while the system gener
   src="/guides/export-to-terra2.png"
 />
 
-#### Working with the Data in Terra
+### Working with the Data in Terra
 
 Until recently, AnVIL data has been hosted and shared from multiple Terra workspaces making it hard to generate cohorts across differing studies. To resolve this, we created the AnVIL Data Explorer enabling you to create custom cohorts and then hand them off to your own Terra workspaces.
 
 Depending on the AnVIL dataset/study, the data in question have varying schemas (different columns and structure to the data). In an effort to ingest all of the AnVIL datasets, the Broad's Data Sciences Platform created a common subset schema across all AnVIL datasets. When you use the AnVIL Data Explorer, it actually searches through a specialized subset - called the Findability Subset (FSS) - that only contains the attributes which are most commonly used by researchers across a broad range of study data types and for diverse analyses.
 
-At this point in development, the data that you hand off from the AnVIL Data Explorer to your Terra workspace is incomplete - it only contains the columns that were deemed relevant for the FSS used in the AnVIL Data Explorer. Currently, if you want the complete data, you will need to perform one additional step after exporting to your workspace. This step is performed by running a publicly available Jupyter Notebook, as instructed below.
-
-##### Working with NIH Data in Terra
+#### Working with NIH Data in Terra
 
 When working with NIH data in Terra, we require users to import data to workspaces with the checkbox for protected data marked. Optionally, an Authorization Domain may be applied and is highly recommended if working with controlled access data.
 
@@ -132,7 +113,7 @@ In the data handoff from the AnVIL Data Explorer, you will transition to the Ter
   src="/guides/working-with-data.png"
 />
 
-##### Selecting Workspace
+#### Selecting Workspace
 
 Next, you’ll see a workspace selection screen where you can either choose an existing workspace or create a new workspace to receive the data.
 
@@ -150,21 +131,3 @@ If you choose to “start with a new workspace”, you will see that the import
 <Figure alt="Create New Workspace" src="/guides/create-new-workspace.png" />
 
 Once you’ve completed this step, your workspace will spin up and you can go to the Data tab of your workspace to see a set of tables have been successfully imported into the workspace.
-
-##### Using the Notebook
-
-The data you now see in your workspace is incomplete - to retrieve the complete data, you’ll need to run a Jupyter Notebook created for this purpose called \*get_non_findability_subset_data_v7.ipynb\*. We’ve published this notebook in a [public workspace](https://app.terra.bio/#workspaces/anvil-datastorage/AnVIL_Explorer_FSS_Tool) for your convenience. To complete the import process, all you need to do is copy the notebook from the public workspace into the workspace to which you’ve imported your data, and run the notebook from within that workspace.
-
-##### Copy the Notebook
-
-Go to the public workspace containing the AnVIL FSS tool notebook, navigate to the Analyses section, and use the three-dot menu to the right of the notebook name to find the option to copy the notebook to another workspace:
-
-<Figure alt="Copy the Notebook" src="/guides/copy-notebook.png" />
-
-##### Run the Notebook
-
-Once the notebook is in the same workspace as the data, set up a Clou open the notebook in edit mode (the default environment is sufficient), and select “Run All” from the Cell menu:
-
-<Figure alt="Run the Notebook" src="/guides/run-notebook.png" />
-
-Once you’ve completed this step, you should be able to see a new set of tables in the Data tab of your workspace.

app/content/anvil-cmg/guides.mdx

@@ -104,10 +104,9 @@ import {
 import * as C from "../../../../components";
 import * as MDX from "../../../../components/common/MDXContent/anvil-cmg";
 import { Description } from "../../../../components/Detail/components/MDX/components/Description/description";
-import { ExportMethod } from "../../../../components/Export/components/AnVILExplorer/ExportMethod/exportMethod";
+import { ExportMethod } from "@databiosphere/findable-ui/lib/components/Export/components/ExportMethod/exportMethod";
 import { METADATA_KEY } from "../../../../components/Index/common/entities";
 import { getPluralizedMetadataLabel } from "../../../../components/Index/common/indexTransformer";
-import { FEATURE_FLAGS } from "../../../common/contants";
 import { Unused, Void } from "../../../common/entities";
 import { SUMMARY_DISPLAY_TEXT } from "./summaryMapper/constants";
 import { mapExportSummary } from "./summaryMapper/summaryMapper";
@@ -167,31 +166,6 @@ export const buildAlertDatasetManifestDownloadWarning = (
   };
 };
 
-/**
- * Build props for dataset-related export warning Alert component.
- * @param _ - Unused.
- * @param viewContext - View context.
- * @returns model to be used as props for the Alert component.
- */
-export const buildAlertDatasetTerraExportWarning = (
-  _: Unused,
-  viewContext: ViewContext<Unused>
-): React.ComponentProps<typeof MDX.Alert> => {
-  const {
-    exploreState: { featureFlagState },
-  } = viewContext;
-  const content = featureFlagState?.includes(FEATURE_FLAGS.VERBATIM)
-    ? isUserAuthenticated(viewContext)
-      ? "To export this dataset, please request access."
-      : "To export this dataset, please sign in and, if necessary, request access."
-    : "Export functionality is currently under development. Check back soon for updates.";
-  return {
-    ...ALERT_PROPS.STANDARD_WARNING,
-    component: C.FluidPaper,
-    content,
-  };
-};
-
 /**
  * Build props for list view access warning Alert component.
  * @param _ - Unused.
@@ -1473,50 +1447,6 @@ export const renderWhenUnAuthenticated = (
   };
 };
 
-/**
- * Renders dataset export to Terra component when the given datasests response is accessible. Note,
- * this can be removed once the verbatim feature flag is removed (use renderDatasetExport instead).
- * @param datasetsResponse - Response model return from datasets API.
- * @param viewContext - View context.
- * @returns model to be used as props for the ConditionalComponent component.
- * @deprecated
- */
-export const renderDatasetTerraExport = (
-  datasetsResponse: DatasetsResponse,
-  viewContext: ViewContext<DatasetsResponse>
-): React.ComponentProps<typeof C.ConditionalComponent> => {
-  const {
-    exploreState: { featureFlagState },
-  } = viewContext;
-  return {
-    isIn:
-      isDatasetAccessible(datasetsResponse) &&
-      Boolean(featureFlagState?.includes(FEATURE_FLAGS.VERBATIM)),
-  };
-};
-
-/**
- * Renders dataset export to Terra warning component when the given datasests response is accessible. Note,
- * this can be removed once the verbatim feature flag is removed (use renderDatasetExportWarning instead).
- * @param datasetsResponse - Response model return from datasets API.
- * @param viewContext - View context.
- * @returns model to be used as props for the ConditionalComponent component.
- * @deprecated
- */
-export const renderDatasetTerraExportWarning = (
-  datasetsResponse: DatasetsResponse,
-  viewContext: ViewContext<DatasetsResponse>
-): React.ComponentProps<typeof C.ConditionalComponent> => {
-  const {
-    exploreState: { featureFlagState },
-  } = viewContext;
-  return {
-    isIn:
-      !isDatasetAccessible(datasetsResponse) ||
-      Boolean(!featureFlagState?.includes(FEATURE_FLAGS.VERBATIM)),
-  };
-};
-
 /**
  * Renders dataset export-related components (either the choose export method component,
  * or specific export components) when the given dataset is accessble.

app/viewModelBuilders/azul/anvil-cmg/common/viewModelBuilders.ts

@@ -6,5 +6,5 @@ export const DATE_TIME_FORMAT_OPTIONS: Intl.DateTimeFormatOptions = {
 };
 export const DATE_TIME_LOCALES = "en-US";
 export const FEATURE_FLAGS = {
-  VERBATIM: "verbatim",
+  VERBATIM: "verbatim", // Deprecated
 };

app/viewModelBuilders/common/contants.ts

@@ -11,6 +11,7 @@ import {
 import {
   MUI_ALERT_ROOT,
   MUI_BUTTON_GROUP_ROOT,
+  MUI_FORM_CONTROL_ROOT,
   MUI_TABLE_CELL_ROOT,
   MUI_TABLE_ROOT,
   MUI_TABLE_ROW_ROOT,
@@ -151,11 +152,26 @@ describe("Dataset", () => {
     // Confirm Terra export method is visible and click it.
     await clickLink(page, BUTTON_TEXT_ANALYZE_IN_TERRA);
 
-    // Confirm the analyze in Terra page is loaded: check the "coming soon"
-    // message is displayed.
-    await expect(
-      page.locator(`${MUI_ALERT_ROOT}:has-text("under development")`)
-    ).toBeVisible();
+    // Confirm the analyze in Terra page is loaded: check for form elements.
+    await expect(page.locator(MUI_FORM_CONTROL_ROOT).first()).toBeVisible();
+  });
+
+  test("displays analyze in Terra selected data", async ({ page }) => {
+    await goToDataset(page, CHIP_TEXT_ACCESS_GRANTED);
+
+    // Confirm export button is visible and click it.
+    await clickLink(page, BUTTON_TEXT_EXPORT);
+
+    // Wait for the summary request once the file manifest button is clicked.
+    const [request] = await Promise.all([
+      page.waitForRequest((request) =>
+        request.url().includes(API_ENDPOINT_SUMMARY)
+      ),
+      clickLink(page, BUTTON_TEXT_ANALYZE_IN_TERRA),
+    ]);
+
+    // Confirm summary request has dataset ID request param.
+    verifySummaryRequest(request);
   });
 });
 

e2e/anvil/anvil-dataset.spec.ts

@@ -1,5 +1,6 @@
 export const MUI_ALERT_ROOT = ".MuiAlert-root";
 export const MUI_BUTTON_GROUP_ROOT = ".MuiButtonGroup-root";
+export const MUI_FORM_CONTROL_ROOT = ".MuiFormControl-root";
 export const MUI_TABLE_CELL_ROOT = ".MuiTableCell-root";
 export const MUI_TABLE_ROOT = ".MuiTable-root";
 export const MUI_TABLE_ROW_ROOT = ".MuiTableRow-root";

e2e/features/common/constants.ts

@@ -36,14 +36,14 @@ export const exportConfig: ExportConfig = {
               children: [
                 {
                   component: MDX.Alert,
-                  viewBuilder: V.buildAlertDatasetTerraExportWarning,
+                  viewBuilder: V.buildAlertDatasetExportWarning,
                 } as ComponentConfig<typeof MDX.Alert, DatasetsResponse>,
               ],
               component: C.BackPageContentSingleColumn,
             } as ComponentConfig<typeof C.BackPageContentSingleColumn>,
           ],
           component: C.ConditionalComponent,
-          viewBuilder: V.renderDatasetTerraExportWarning,
+          viewBuilder: V.renderDatasetExportWarning,
         } as ComponentConfig<typeof C.ConditionalComponent, DatasetsResponse>,
         /* ------ */
         /* Dataset is accessible; render Terra export method */
@@ -63,7 +63,7 @@ export const exportConfig: ExportConfig = {
             ...exportSideColumn,
           ],
           component: C.ConditionalComponent,
-          viewBuilder: V.renderDatasetTerraExport,
+          viewBuilder: V.renderDatasetExport,
         } as ComponentConfig<typeof C.ConditionalComponent, DatasetsResponse>,
       ],
       route: ROUTE_EXPORT_TO_TERRA,

public/guides/exploring-studies.png

@@ -3,7 +3,6 @@ import {
   ExportConfig,
 } from "@databiosphere/findable-ui/lib/config/entities";
 import * as C from "../../../../app/components";
-import { ExportMethod } from "../../../../app/components/Export/components/AnVILExplorer/ExportMethod/exportMethod";
 import * as V from "../../../../app/viewModelBuilders/azul/anvil-cmg/common/viewModelBuilders";
 import { ROUTE_EXPORT_TO_TERRA, ROUTE_MANIFEST_DOWNLOAD } from "./constants";
 import { mainColumn as exportMainColumn } from "./exportMainColumn";
@@ -73,9 +72,9 @@ export const exportConfig: ExportConfig = {
         {
           children: [
             {
-              component: ExportMethod,
+              component: C.ExportMethod,
               viewBuilder: V.buildExportMethodTerra,
-            } as ComponentConfig<typeof ExportMethod>,
+            } as ComponentConfig<typeof C.ExportMethod>,
             {
               component: C.ExportMethod,
               viewBuilder: V.buildExportMethodManifestDownload,