Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' #1339

tongyu-microsoft · 2022-04-12T20:07:39Z

This PR adds warning messages in cohort editor for invalid input values; This PR also updates wording from 'Shift cohort' to 'Switch cohort'.

Description

Add warning messages in cohort editor for invalid input values: There was a bug created regarding to cohort input field, where manually putting in a value with keyboard does not work. The root cause for this bug is, we limit input value to be in between min value and max value if it is a numeric number. We don't allow users to input invalid values. In the case when user inputs an invalid value, we reset the value back to previous input value. This PR adds warning message to users so that users can understand the reason they are not able to input the value is because the input value is invalid.
We used to show "Shift cohort" in some places, while "Switch cohort" in others. This PR updates all the wordings from 'Shift cohort' to 'Switch cohort' to be consistent.

Areas changed

npm packages changed:

Python packages changed:

Tests

No new tests required.
New tests for the added feature are part of this PR.
I validated the changes manually.

Screenshots (if appropriate):

Documentation:

My change requires a change to the documentation.
I have updated the documentation accordingly.

libs/localization/src/lib/en.json

libs/core-ui/src/lib/Cohort/ManualCohortManagement/CohortEditorFilter.tsx

romanlutz · 2022-04-12T20:25:03Z

@tongyu-microsoft I have no objections, this looks great to me!
I was just curious if you'd considered using the built-in warnings for dropdowns: https://developer.microsoft.com/en-us/fluentui#/controls/web/dropdown

The nice thing about this is that the dropdown itself also goes red which is a nice visual suggestion to the user.

I guess this goes back to whether we are forcing users to change it (error) or whether it's more of a suggestion (warning). I believe it's an error, not just a warning, right?

tongyu-microsoft · 2022-04-12T20:28:51Z

@tongyu-microsoft I have no objections, this looks great to me! I was just curious if you'd considered using the built-in warnings for dropdowns: https://developer.microsoft.com/en-us/fluentui#/controls/web/dropdown

The nice thing about this is that the dropdown itself also goes red which is a nice visual suggestion to the user.

I guess this goes back to whether we are forcing users to change it (error) or whether it's more of a suggestion (warning). I believe it's an error, not just a warning, right?

Thanks for the comment! Yeah the built-in errors for Dropdown or TextField etc are nice! But here we are using component called SpinButton and I didn't find built-in warning/error properties in SpinButton

…onsible-ai-widgets into tongy/cohortInput

codecov-commenter · 2022-04-12T20:42:41Z

Codecov Report

Merging #1339 (3eacfa9) into main (f7d75d3) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1339   +/-   ##
=======================================
  Coverage   86.28%   86.28%           
=======================================
  Files         100      100           
  Lines        4675     4675           
=======================================
  Hits         4034     4034           
  Misses        641      641

Flag	Coverage Δ
unittests	`86.28% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f7d75d3...3eacfa9. Read the comment docs.

github-actions · 2022-04-12T20:55:49Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/tongy/cohortInput/dashboard/index.html

github-actions · 2022-04-12T21:10:10Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/tongy/cohortInput/dashboard/index.html

github-actions · 2022-04-12T21:29:49Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/tongy/cohortInput/dashboard/index.html

…'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort

…aking.ipynb (#1252) * [WIP] Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Fix the e2e test for notebook Signed-off-by: Gaurav Gupta <[email protected]> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <[email protected]> * Add missing file Signed-off-by: Gaurav Gupta <[email protected]> * Fix cohort namespace Signed-off-by: Gaurav Gupta <[email protected]> * minor fix to url for responsibleai package in setup.py (#1260) * Counterfactual Chart: Correct target description according to task_type (#1261) * Counterfactual Chart: Correct target description according to task_type Signed-off-by: Gaurav Gupta <[email protected]> * Change function name Signed-off-by: Gaurav Gupta <[email protected]> * Make Range lowercase Signed-off-by: Gaurav Gupta <[email protected]> * fix whitespace in values of adult census income dataset (#1263) * Add what-If scatter chart from highchart lib (#1262) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * add a builddebug yarn command to build UX locally which can be debugged in browser e2e (#1265) * allow rai text insights to work with RAI dashboard (#1269) * remove duplicate code in explanation dashboard (#1266) * Individual causal style responsive (#1268) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * Causal Style * Allow duplicating cohorts multiple times (#1274) * allow duplicating a cohort more than once * lintfix * Disable column header highlighting on hover in IndividualFeatureImportanceView (#1272) * disable column highlight on hover * lintfix * Rename new cohorts from "Unsaved" to "Temporary cohort" (#1273) * rename Unsaved to Temporary cohort * localize temp cohort * Counterfactual style refactor (#1275) * style refactor * test * test * test * fix * Don't change cursor on hover over cohort name * Fix (#1281) * fix cohort info styling (#1277) * fix readme link to fairness and interpretability example notebook (#1282) * add new RAI Utils package for common utilities shared across RAI packages (#1280) * Add ICE chart (#1283) * Fix * ice chart * ic * test * test * update docstring for explanation dashboard in regards to min number of rows (#1271) * make builds more reliable by adding retry logic to urlretrieve calls in notebooks (#1218) * upgrade pytest to 7.0.1, remove mock and updgrade pytest-mock to 3.6.1 (#1287) * remove deprecated codecov parameter (#1293) * Fix min/max special case in cohort filter creation with "in the range of" (#1279) * fix logic in the case that min or max are zero * lintfix * Rename 'Dashboard navigation' to 'Dashboard configuration' (#1291) * Rename 'Dashboard navigation' to 'Dashboard configuration' Signed-off-by: Gaurav Gupta <[email protected]> * Notebook change Signed-off-by: Gaurav Gupta <[email protected]> * Add raiutils to PR template (#1290) * fix heatmap bug (#1297) * Make "save and switch" work from cohort settings (#1276) * make save and switch work * fix naming * lintfix * adjustment according to Ilya's comment * lintfix * add retry logic to codecov step and only upload results for one python version (#1298) * add github action to release raiutils to pypi (#1294) * Add highchart for Dataset Explorer (#1286) * test * style * click * fix test * fix test * test * test * test * test * Update requirements-linting.txt to add flake8-pytest-style (#1296) * Fix sort abs (#1299) * Rename "base cohort" to "global cohort" (#1278) * change base cohort to global cohort * fix spelling * lintfix * fix codecov comment not appearing on PRs (#1302) * take absolute value of error calculation for regression scenario (#1301) * Limit individual feature importance selection to up to 5 (#1305) * update feature importance string * limit selection to up to 5 * add group count * remove message bar, show info icon instead * update e2e locator * fix E2E failure on feature importance * add ariaLabel for expand collapse button * add renderOnNewLayer props * Add error message for counterfactual panel (#1310) * add error message for counterfactual * update error message in camel case to fix build error * Add to_json() and from_json() methods to Cohort class (#1300) * Add to_json() and from_json() methods to Cohort class Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix linting Signed-off-by: Gaurav Gupta <[email protected]> * Add a highchart heatmap helper class (#1307) * add highchart heatmap helper class * add erroneously deleted line back * Fix cohort setting string (#1304) * Fix string * remove none * name * test * Fix all data style (#1303) * Add a feature flag for the new model overview experience (#1306) * add feature flag for new model overview experience and turn it off by default * remove useless constructor * Clean up charts code (#1313) * clean up chart code * remove arg * Expand the counterfactual flyout to cover the full page (#1315) Signed-off-by: Gaurav Gupta <[email protected]> * Bump minimist from 1.2.5 to 1.2.6 (#1292) * Bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> * minimist ^1.2.6 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * fix random node download failures by upgrading to latest github action with retry logic (#1317) * Add dataset cohort table to new ModelOverview experience (#1314) * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * Add installation instructions for raiwidgets to README (#1320) * refactor RAIInsights into RAIInsightsBase class for basic functionality (#1284) * Fix what if counterfactual header and description text misaligned (#1316) * align * e2e * add clear temporary cohort button to error analysis (#1322) * Raise UserConfigValidationException in case no model but valid model serializer (#1325) Signed-off-by: Gaurav Gupta <[email protected]> * Add test case for handling different types in causal (#1321) Signed-off-by: Gaurav Gupta <[email protected]> * show shift to an empty cohort in tree view as an empty node (#1318) * Bug fixing (#1326) * Move chart description up and remove scroll bar * Change string * Add box outlier for dataset explorer (#1323) * add outlier for dataset explorer * name * update string when no datapoint selected (#1331) * Fix Big empty space for featureImportance chart (#1328) * legend * removed invalid test case * constant * Disable save as new cohort button if nothing is selected in error tree (#1327) * Add disaggregated analysis table/heatmap (#1332) * disaggregated analysis changes only * lintfix * Change warning message to user exception for model type and task type mismatch (#1330) * Change warning message to user exception for model type and task type mismatch Signed-off-by: Gaurav Gupta <[email protected]> * Fix flake8 errors Signed-off-by: Gaurav Gupta <[email protected]> * Change the counterfactual text color from black to grey (#1337) Signed-off-by: Gaurav Gupta <[email protected]> * Limit each component description width up to 750px for readability (#1336) * limit description width up to 750px * export maxWidth from a common place * block empty cohort creation in RAI Dashboard (#1335) * Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort * Rename counterfactual style files to confirm with *.styles.ts (#1338) Signed-off-by: Gaurav Gupta <[email protected]> * Add disaggregated analysis table to Model Overview (#1341) * pull in changes for disaggregated analysis * add styles file * add textured NaN cells * module import for textured cells and grid y axis * lintfix * use combobox for dropdown rather than dropdown * lintfix * Rename causal style files to confirm with *.styles.ts (#1342) Signed-off-by: Gaurav Gupta <[email protected]> * update responsibleai to interpret-community 0.25.0 (#1343) * All component title and descriptions should be aligned (#1346) * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * Remove 5K limit blurb from local explanations tab (#1347) Signed-off-by: Gaurav Gupta <[email protected]> * Sort features by default in counterfactual flyout (#1312) * Sort features by default in counterfactual flyout Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> * attempt to fix test Signed-off-by: Gaurav Gupta <[email protected]> * Remove check Signed-off-by: Gaurav Gupta <[email protected]> * Bump moment from 2.28.0 to 2.29.2 (#1333) Bumps [moment](https://github.com/moment/moment) from 2.28.0 to 2.29.2. - [Release notes](https://github.com/moment/moment/releases) - [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md) - [Commits](moment/moment@2.28.0...2.29.2) --- updated-dependencies: - dependency-name: moment dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <[email protected]> * Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text (#1350) * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1344) * update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected (#1349) Co-authored-by: Vinutha Karanth <[email protected]> * update erroranalysis to 0.2.1 and remove some duplicate dependencies (#1334) * disable turbo checking for large amount of data (#1351) Signed-off-by: Ke Xu <[email protected]> * force re-render when chart type changes (#1354) Signed-off-by: Ke Xu <[email protected]> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1352) * Rename output column name in the counterfactual flyout (#1353) Signed-off-by: Gaurav Gupta <[email protected]> * Show column chart for categorical feature in data explorer (#1355) * Show column chart for categorical feature in data explorer * address comments * update fluentui (#1356) Signed-off-by: Ke Xu <[email protected]> * update code owner (#1308) * update code owner Signed-off-by: Ke Xu <[email protected]> * remove dup Signed-off-by: Ke Xu <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * update version to match studio (#1357) Signed-off-by: Ke Xu <[email protected]> * alignment fixes (#1359) * Add charts for new model overview experience (#1348) * rename OverallTable to DisaggregatedMetricsTable and move to core-ui * Copy the ModelPerformanceTab into model-assessment and rename to ModelOverview * reference OverallTable again in fairness widget * refactor core chart component out into core-ui * refactor out core chart component into core-ui * lintfix * undo tsconfig.lib.json change * fix chartAndConfigsId in tests * lintfix * add table for cohort metrics and add dropdown metric selector, add new metrics * add new metrics * undo unwanted changes * fix casing * add superscript 2 for r-squared * update tests to reflect new metrics * lintfix * add feature flag * fix mae * fix mae calculation * first version of new model overview table * get probability distribution box plot to work * add feature flag for new model overview experience and turn it off by default * add highchart heatmap helper class * remove custom styling * add erroneously deleted line back * remove useless constructor * modularize model overview * show outliers in box plot, fix positioning * remove showmetricsummary * refactor heatmap code into a common class * add featureDropdownRef to allow focusing * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * add chart config flyout (in progress) * add chart config flyout (in progress) * address feedback, use finalized color * adjust feature selection to disable options if limit is reached, add axis config buttons * select all via dropdown * lintfix * refactor box plot calculations and rendering * add style file * textured NaN cells, grid labels on y axis * standardize box plots to use fences * fix merge issues * unify box plot tooltip formatting code, fix bar chart sizing issue * small fixes * rearrange feature dropdown * lintfix * remove commented out code * remove box plot tooltip customization * lintfix * add a few unit tests * unit tests for smaller utilities, localization fixes, consistent flyout flow with confirm/cancel buttons * lintfix * fix chart config flyout update * fix test case * rename files for lint * file rename for lint * release rai-core-flask 0.3.0 (#1361) * upgrade python version used with flask CI to fix segfault error (#1363) * release raiwidgets and responsibleai v0.18.0 (#1360) * fix two bugs (#1364) * Add pre-built cohort into adult census notebook (#1243) * [WIP] Add pre-built cohort into adult census notebook Signed-off-by: Gaurav Gupta <[email protected]> * erroranalysis version bump in raiwidgets to 0.1.31 (#1245) * Make cohrtData empty list in case no pre-bdefined cohorts are injected (#1247) Signed-off-by: Gaurav Gupta <[email protected]> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add regression test for pre-defined cohorts in raiwidgets (#1249) Signed-off-by: Gaurav Gupta <[email protected]> * color (#1248) * Add feature importance box & bar chart (#1241) * refactor * build * build * temp * temp * temp * temp * box * cache * e2e * e2e * fix * e2e fix * e2e * fix e2e * widget * widget * fix * widget * e2e * e2e * e2e * test * test * PreBuilt cohorts UX changes (#1242) * Intial SDK implementation cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add basic validationf for cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add serialized version of cohort config to ResponsibleAiDashboard Signed-off-by: Gaurav Gupta <[email protected]> * Add more tests cohorts Signed-off-by: Gaurav Gupta <[email protected]> * fix broken builds due to pip upgrade which broke pip-tools (#1185) * refactor matrix filter and area state to be private static (#1179) * Change variable name Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add cohort data to dashboard e2e Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohorts filters Signed-off-by: Gaurav Gupta <[email protected]> * Document various data validation for cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add new interfaces for pre-built cohort Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add prebuilt cohort walking logic in UI and add more data validation scenarios Signed-off-by: Gaurav Gupta <[email protected]> * Add basic data validation checks Signed-off-by: Gaurav Gupta <[email protected]> * Add logic to translate the Index cohort filter Signed-off-by: Gaurav Gupta <[email protected]> * Remove commented out code Signed-off-by: Gaurav Gupta <[email protected]> * Add SDK validations for Index based cohort filter Signed-off-by: Gaurav Gupta <[email protected]> * Add code for validating classification outcome Signed-off-by: Gaurav Gupta <[email protected]> * Add error filter validations and add tests Signed-off-by: Gaurav Gupta <[email protected]> * Add fake cohorts for regression dataset Signed-off-by: Gaurav Gupta <[email protected]> * Add fake cohorts for multi-class classification dataset Signed-off-by: Gaurav Gupta <[email protected]> * Add handling of regression filter Signed-off-by: Gaurav Gupta <[email protected]> * Add support for classification outcome in UI Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for Predicted Y and True Y cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add UI code to handle prediced Y and true Y for pre-built cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add cohort validation with test data to raiwidgets Signed-off-by: Gaurav Gupta <[email protected]> * Add tests for validating Predicted/True Y cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add UI support for TrueY/PredictedY for classification Signed-off-by: Gaurav Gupta <[email protected]> * Rename cohort_filter_list to cohort_list Signed-off-by: Gaurav Gupta <[email protected]> * Rename UI varibles to match SDK Signed-off-by: Gaurav Gupta <[email protected]> * Fix duplicate cohort name Signed-off-by: Gaurav Gupta <[email protected]> * Add SDK cohorts to notebook Signed-off-by: Gaurav Gupta <[email protected]> * Add dataset validations and add categorical features Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for categorical_features Signed-off-by: Gaurav Gupta <[email protected]> * Fix sorted imports Signed-off-by: Gaurav Gupta <[email protected]> * Add code for translating categorical values Signed-off-by: Gaurav Gupta <[email protected]> * Move cohort processing to a separate file Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Refactor cohort translated function into different small functions Signed-off-by: Gaurav Gupta <[email protected]> * Change to lowercase for outcome Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Refactor cohort_list validations and converge pytest common functions into fixtures Signed-off-by: Gaurav Gupta <[email protected]> * Add conftest into raiwidgets tests Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for cohort list Signed-off-by: Gaurav Gupta <[email protected]> * Add cohortData test Signed-off-by: Gaurav Gupta <[email protected]> * Fix sorted imports Signed-off-by: Gaurav Gupta <[email protected]> * isort fix Signed-off-by: Gaurav Gupta <[email protected]> * Add UI unit tests for cohort translation Signed-off-by: Gaurav Gupta <[email protected]> * Add more checks in UI uni test Signed-off-by: Gaurav Gupta <[email protected]> * Add UI tests for regression cohorts Signed-off-by: Gaurav Gupta <[email protected]> * REmove notebook change Signed-off-by: Gaurav Gupta <[email protected]> * Fix typescript build Signed-off-by: Gaurav Gupta <[email protected]> * Change cohort filter values so that cohort filters non-zero points Signed-off-by: Gaurav Gupta <[email protected]> * Fix for empty cohort list Signed-off-by: Gaurav Gupta <[email protected]> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Propagate error strings instead of raising exceptions Signed-off-by: Gaurav Gupta <[email protected]> * Fix code issues Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <[email protected]> * Add missing file Signed-off-by: Gaurav Gupta <[email protected]> * fix notebook build failures due to pywinpty dependency release failing in python 3.6 (#1257) * fix notebook build failures due to pywinpty dependency release failing in python 3.6 * build pywinpty from conda instead * add lowerbound * fixup * fixup * Add supported models and data types to README.md responsibleai (#1259) Signed-off-by: Gaurav Gupta <[email protected]> * make getting-started notebook a markdown file showing APIs (#1223) * refactor tabs out of RAI dashboard into a separate component (#1256) * Add individual causal scatter chart (#1258) * temp * refactor * test * style fix * comment * minor fix to url for responsibleai package in setup.py (#1260) * Fix UX e2e tests and address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix eslint Signed-off-by: Gaurav Gupta <[email protected]> * Address review comments Signed-off-by: Gaurav Gupta <[email protected]> * Reset the number of samples in test dataset Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Bo Zhang <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Change cohort name Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Bo Zhang <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: tongy-msft <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <[email protected]> Co-authored-by: Vinutha Karanth <[email protected]>

…'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort

…aking.ipynb (#1252) * [WIP] Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Fix the e2e test for notebook Signed-off-by: Gaurav Gupta <[email protected]> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <[email protected]> * Add missing file Signed-off-by: Gaurav Gupta <[email protected]> * Fix cohort namespace Signed-off-by: Gaurav Gupta <[email protected]> * minor fix to url for responsibleai package in setup.py (#1260) * Counterfactual Chart: Correct target description according to task_type (#1261) * Counterfactual Chart: Correct target description according to task_type Signed-off-by: Gaurav Gupta <[email protected]> * Change function name Signed-off-by: Gaurav Gupta <[email protected]> * Make Range lowercase Signed-off-by: Gaurav Gupta <[email protected]> * fix whitespace in values of adult census income dataset (#1263) * Add what-If scatter chart from highchart lib (#1262) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * add a builddebug yarn command to build UX locally which can be debugged in browser e2e (#1265) * allow rai text insights to work with RAI dashboard (#1269) * remove duplicate code in explanation dashboard (#1266) * Individual causal style responsive (#1268) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * Causal Style * Allow duplicating cohorts multiple times (#1274) * allow duplicating a cohort more than once * lintfix * Disable column header highlighting on hover in IndividualFeatureImportanceView (#1272) * disable column highlight on hover * lintfix * Rename new cohorts from "Unsaved" to "Temporary cohort" (#1273) * rename Unsaved to Temporary cohort * localize temp cohort * Counterfactual style refactor (#1275) * style refactor * test * test * test * fix * Don't change cursor on hover over cohort name * Fix (#1281) * fix cohort info styling (#1277) * fix readme link to fairness and interpretability example notebook (#1282) * add new RAI Utils package for common utilities shared across RAI packages (#1280) * Add ICE chart (#1283) * Fix * ice chart * ic * test * test * update docstring for explanation dashboard in regards to min number of rows (#1271) * make builds more reliable by adding retry logic to urlretrieve calls in notebooks (#1218) * upgrade pytest to 7.0.1, remove mock and updgrade pytest-mock to 3.6.1 (#1287) * remove deprecated codecov parameter (#1293) * Fix min/max special case in cohort filter creation with "in the range of" (#1279) * fix logic in the case that min or max are zero * lintfix * Rename 'Dashboard navigation' to 'Dashboard configuration' (#1291) * Rename 'Dashboard navigation' to 'Dashboard configuration' Signed-off-by: Gaurav Gupta <[email protected]> * Notebook change Signed-off-by: Gaurav Gupta <[email protected]> * Add raiutils to PR template (#1290) * fix heatmap bug (#1297) * Make "save and switch" work from cohort settings (#1276) * make save and switch work * fix naming * lintfix * adjustment according to Ilya's comment * lintfix * add retry logic to codecov step and only upload results for one python version (#1298) * add github action to release raiutils to pypi (#1294) * Add highchart for Dataset Explorer (#1286) * test * style * click * fix test * fix test * test * test * test * test * Update requirements-linting.txt to add flake8-pytest-style (#1296) * Fix sort abs (#1299) * Rename "base cohort" to "global cohort" (#1278) * change base cohort to global cohort * fix spelling * lintfix * fix codecov comment not appearing on PRs (#1302) * take absolute value of error calculation for regression scenario (#1301) * Limit individual feature importance selection to up to 5 (#1305) * update feature importance string * limit selection to up to 5 * add group count * remove message bar, show info icon instead * update e2e locator * fix E2E failure on feature importance * add ariaLabel for expand collapse button * add renderOnNewLayer props * Add error message for counterfactual panel (#1310) * add error message for counterfactual * update error message in camel case to fix build error * Add to_json() and from_json() methods to Cohort class (#1300) * Add to_json() and from_json() methods to Cohort class Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix linting Signed-off-by: Gaurav Gupta <[email protected]> * Add a highchart heatmap helper class (#1307) * add highchart heatmap helper class * add erroneously deleted line back * Fix cohort setting string (#1304) * Fix string * remove none * name * test * Fix all data style (#1303) * Add a feature flag for the new model overview experience (#1306) * add feature flag for new model overview experience and turn it off by default * remove useless constructor * Clean up charts code (#1313) * clean up chart code * remove arg * Expand the counterfactual flyout to cover the full page (#1315) Signed-off-by: Gaurav Gupta <[email protected]> * Bump minimist from 1.2.5 to 1.2.6 (#1292) * Bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> * minimist ^1.2.6 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * fix random node download failures by upgrading to latest github action with retry logic (#1317) * Add dataset cohort table to new ModelOverview experience (#1314) * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * Add installation instructions for raiwidgets to README (#1320) * refactor RAIInsights into RAIInsightsBase class for basic functionality (#1284) * Fix what if counterfactual header and description text misaligned (#1316) * align * e2e * add clear temporary cohort button to error analysis (#1322) * Raise UserConfigValidationException in case no model but valid model serializer (#1325) Signed-off-by: Gaurav Gupta <[email protected]> * Add test case for handling different types in causal (#1321) Signed-off-by: Gaurav Gupta <[email protected]> * show shift to an empty cohort in tree view as an empty node (#1318) * Bug fixing (#1326) * Move chart description up and remove scroll bar * Change string * Add box outlier for dataset explorer (#1323) * add outlier for dataset explorer * name * update string when no datapoint selected (#1331) * Fix Big empty space for featureImportance chart (#1328) * legend * removed invalid test case * constant * Disable save as new cohort button if nothing is selected in error tree (#1327) * Add disaggregated analysis table/heatmap (#1332) * disaggregated analysis changes only * lintfix * Change warning message to user exception for model type and task type mismatch (#1330) * Change warning message to user exception for model type and task type mismatch Signed-off-by: Gaurav Gupta <[email protected]> * Fix flake8 errors Signed-off-by: Gaurav Gupta <[email protected]> * Change the counterfactual text color from black to grey (#1337) Signed-off-by: Gaurav Gupta <[email protected]> * Limit each component description width up to 750px for readability (#1336) * limit description width up to 750px * export maxWidth from a common place * block empty cohort creation in RAI Dashboard (#1335) * Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort * Rename counterfactual style files to confirm with *.styles.ts (#1338) Signed-off-by: Gaurav Gupta <[email protected]> * Add disaggregated analysis table to Model Overview (#1341) * pull in changes for disaggregated analysis * add styles file * add textured NaN cells * module import for textured cells and grid y axis * lintfix * use combobox for dropdown rather than dropdown * lintfix * Rename causal style files to confirm with *.styles.ts (#1342) Signed-off-by: Gaurav Gupta <[email protected]> * update responsibleai to interpret-community 0.25.0 (#1343) * All component title and descriptions should be aligned (#1346) * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * Remove 5K limit blurb from local explanations tab (#1347) Signed-off-by: Gaurav Gupta <[email protected]> * Sort features by default in counterfactual flyout (#1312) * Sort features by default in counterfactual flyout Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> * attempt to fix test Signed-off-by: Gaurav Gupta <[email protected]> * Remove check Signed-off-by: Gaurav Gupta <[email protected]> * Bump moment from 2.28.0 to 2.29.2 (#1333) Bumps [moment](https://github.com/moment/moment) from 2.28.0 to 2.29.2. - [Release notes](https://github.com/moment/moment/releases) - [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md) - [Commits](moment/moment@2.28.0...2.29.2) --- updated-dependencies: - dependency-name: moment dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <[email protected]> * Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text (#1350) * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1344) * update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected (#1349) Co-authored-by: Vinutha Karanth <[email protected]> * update erroranalysis to 0.2.1 and remove some duplicate dependencies (#1334) * disable turbo checking for large amount of data (#1351) Signed-off-by: Ke Xu <[email protected]> * force re-render when chart type changes (#1354) Signed-off-by: Ke Xu <[email protected]> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1352) * Rename output column name in the counterfactual flyout (#1353) Signed-off-by: Gaurav Gupta <[email protected]> * Show column chart for categorical feature in data explorer (#1355) * Show column chart for categorical feature in data explorer * address comments * update fluentui (#1356) Signed-off-by: Ke Xu <[email protected]> * update code owner (#1308) * update code owner Signed-off-by: Ke Xu <[email protected]> * remove dup Signed-off-by: Ke Xu <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * update version to match studio (#1357) Signed-off-by: Ke Xu <[email protected]> * alignment fixes (#1359) * Add charts for new model overview experience (#1348) * rename OverallTable to DisaggregatedMetricsTable and move to core-ui * Copy the ModelPerformanceTab into model-assessment and rename to ModelOverview * reference OverallTable again in fairness widget * refactor core chart component out into core-ui * refactor out core chart component into core-ui * lintfix * undo tsconfig.lib.json change * fix chartAndConfigsId in tests * lintfix * add table for cohort metrics and add dropdown metric selector, add new metrics * add new metrics * undo unwanted changes * fix casing * add superscript 2 for r-squared * update tests to reflect new metrics * lintfix * add feature flag * fix mae * fix mae calculation * first version of new model overview table * get probability distribution box plot to work * add feature flag for new model overview experience and turn it off by default * add highchart heatmap helper class * remove custom styling * add erroneously deleted line back * remove useless constructor * modularize model overview * show outliers in box plot, fix positioning * remove showmetricsummary * refactor heatmap code into a common class * add featureDropdownRef to allow focusing * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * add chart config flyout (in progress) * add chart config flyout (in progress) * address feedback, use finalized color * adjust feature selection to disable options if limit is reached, add axis config buttons * select all via dropdown * lintfix * refactor box plot calculations and rendering * add style file * textured NaN cells, grid labels on y axis * standardize box plots to use fences * fix merge issues * unify box plot tooltip formatting code, fix bar chart sizing issue * small fixes * rearrange feature dropdown * lintfix * remove commented out code * remove box plot tooltip customization * lintfix * add a few unit tests * unit tests for smaller utilities, localization fixes, consistent flyout flow with confirm/cancel buttons * lintfix * fix chart config flyout update * fix test case * rename files for lint * file rename for lint * release rai-core-flask 0.3.0 (#1361) * upgrade python version used with flask CI to fix segfault error (#1363) * release raiwidgets and responsibleai v0.18.0 (#1360) * fix two bugs (#1364) * Add pre-built cohort into adult census notebook (#1243) * [WIP] Add pre-built cohort into adult census notebook Signed-off-by: Gaurav Gupta <[email protected]> * erroranalysis version bump in raiwidgets to 0.1.31 (#1245) * Make cohrtData empty list in case no pre-bdefined cohorts are injected (#1247) Signed-off-by: Gaurav Gupta <[email protected]> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add regression test for pre-defined cohorts in raiwidgets (#1249) Signed-off-by: Gaurav Gupta <[email protected]> * color (#1248) * Add feature importance box & bar chart (#1241) * refactor * build * build * temp * temp * temp * temp * box * cache * e2e * e2e * fix * e2e fix * e2e * fix e2e * widget * widget * fix * widget * e2e * e2e * e2e * test * test * PreBuilt cohorts UX changes (#1242) * Intial SDK implementation cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add basic validationf for cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add serialized version of cohort config to ResponsibleAiDashboard Signed-off-by: Gaurav Gupta <[email protected]> * Add more tests cohorts Signed-off-by: Gaurav Gupta <[email protected]> * fix broken builds due to pip upgrade which broke pip-tools (#1185) * refactor matrix filter and area state to be private static (#1179) * Change variable name Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add cohort data to dashboard e2e Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohorts filters Signed-off-by: Gaurav Gupta <[email protected]> * Document various data validation for cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add new interfaces for pre-built cohort Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add prebuilt cohort walking logic in UI and add more data validation scenarios Signed-off-by: Gaurav Gupta <[email protected]> * Add basic data validation checks Signed-off-by: Gaurav Gupta <[email protected]> * Add logic to translate the Index cohort filter Signed-off-by: Gaurav Gupta <[email protected]> * Remove commented out code Signed-off-by: Gaurav Gupta <[email protected]> * Add SDK validations for Index based cohort filter Signed-off-by: Gaurav Gupta <[email protected]> * Add code for validating classification outcome Signed-off-by: Gaurav Gupta <[email protected]> * Add error filter validations and add tests Signed-off-by: Gaurav Gupta <[email protected]> * Add fake cohorts for regression dataset Signed-off-by: Gaurav Gupta <[email protected]> * Add fake cohorts for multi-class classification dataset Signed-off-by: Gaurav Gupta <[email protected]> * Add handling of regression filter Signed-off-by: Gaurav Gupta <[email protected]> * Add support for classification outcome in UI Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for Predicted Y and True Y cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add UI code to handle prediced Y and true Y for pre-built cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add cohort validation with test data to raiwidgets Signed-off-by: Gaurav Gupta <[email protected]> * Add tests for validating Predicted/True Y cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add UI support for TrueY/PredictedY for classification Signed-off-by: Gaurav Gupta <[email protected]> * Rename cohort_filter_list to cohort_list Signed-off-by: Gaurav Gupta <[email protected]> * Rename UI varibles to match SDK Signed-off-by: Gaurav Gupta <[email protected]> * Fix duplicate cohort name Signed-off-by: Gaurav Gupta <[email protected]> * Add SDK cohorts to notebook Signed-off-by: Gaurav Gupta <[email protected]> * Add dataset validations and add categorical features Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for categorical_features Signed-off-by: Gaurav Gupta <[email protected]> * Fix sorted imports Signed-off-by: Gaurav Gupta <[email protected]> * Add code for translating categorical values Signed-off-by: Gaurav Gupta <[email protected]> * Move cohort processing to a separate file Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Refactor cohort translated function into different small functions Signed-off-by: Gaurav Gupta <[email protected]> * Change to lowercase for outcome Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Refactor cohort_list validations and converge pytest common functions into fixtures Signed-off-by: Gaurav Gupta <[email protected]> * Add conftest into raiwidgets tests Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for cohort list Signed-off-by: Gaurav Gupta <[email protected]> * Add cohortData test Signed-off-by: Gaurav Gupta <[email protected]> * Fix sorted imports Signed-off-by: Gaurav Gupta <[email protected]> * isort fix Signed-off-by: Gaurav Gupta <[email protected]> * Add UI unit tests for cohort translation Signed-off-by: Gaurav Gupta <[email protected]> * Add more checks in UI uni test Signed-off-by: Gaurav Gupta <[email protected]> * Add UI tests for regression cohorts Signed-off-by: Gaurav Gupta <[email protected]> * REmove notebook change Signed-off-by: Gaurav Gupta <[email protected]> * Fix typescript build Signed-off-by: Gaurav Gupta <[email protected]> * Change cohort filter values so that cohort filters non-zero points Signed-off-by: Gaurav Gupta <[email protected]> * Fix for empty cohort list Signed-off-by: Gaurav Gupta <[email protected]> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Propagate error strings instead of raising exceptions Signed-off-by: Gaurav Gupta <[email protected]> * Fix code issues Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <[email protected]> * Add missing file Signed-off-by: Gaurav Gupta <[email protected]> * fix notebook build failures due to pywinpty dependency release failing in python 3.6 (#1257) * fix notebook build failures due to pywinpty dependency release failing in python 3.6 * build pywinpty from conda instead * add lowerbound * fixup * fixup * Add supported models and data types to README.md responsibleai (#1259) Signed-off-by: Gaurav Gupta <[email protected]> * make getting-started notebook a markdown file showing APIs (#1223) * refactor tabs out of RAI dashboard into a separate component (#1256) * Add individual causal scatter chart (#1258) * temp * refactor * test * style fix * comment * minor fix to url for responsibleai package in setup.py (#1260) * Fix UX e2e tests and address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix eslint Signed-off-by: Gaurav Gupta <[email protected]> * Address review comments Signed-off-by: Gaurav Gupta <[email protected]> * Reset the number of samples in test dataset Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Bo Zhang <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Change cohort name Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Bo Zhang <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: tongy-msft <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <[email protected]> Co-authored-by: Vinutha Karanth <[email protected]>

* Add more utilities into raiutils Signed-off-by: Gaurav Gupta <[email protected]> * Fix faulty file name Signed-off-by: Gaurav Gupta <[email protected]> * fix heatmap bug (#1297) * Make "save and switch" work from cohort settings (#1276) * make save and switch work * fix naming * lintfix * adjustment according to Ilya's comment * lintfix * add retry logic to codecov step and only upload results for one python version (#1298) * add github action to release raiutils to pypi (#1294) * Add highchart for Dataset Explorer (#1286) * test * style * click * fix test * fix test * test * test * test * test * Update requirements-linting.txt to add flake8-pytest-style (#1296) * Fix sort abs (#1299) * Rename "base cohort" to "global cohort" (#1278) * change base cohort to global cohort * fix spelling * lintfix * fix codecov comment not appearing on PRs (#1302) * take absolute value of error calculation for regression scenario (#1301) * Limit individual feature importance selection to up to 5 (#1305) * update feature importance string * limit selection to up to 5 * add group count * remove message bar, show info icon instead * update e2e locator * fix E2E failure on feature importance * add ariaLabel for expand collapse button * add renderOnNewLayer props * Add error message for counterfactual panel (#1310) * add error message for counterfactual * update error message in camel case to fix build error * Add to_json() and from_json() methods to Cohort class (#1300) * Add to_json() and from_json() methods to Cohort class Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix linting Signed-off-by: Gaurav Gupta <[email protected]> * Add a highchart heatmap helper class (#1307) * add highchart heatmap helper class * add erroneously deleted line back * Fix cohort setting string (#1304) * Fix string * remove none * name * test * Fix all data style (#1303) * Add a feature flag for the new model overview experience (#1306) * add feature flag for new model overview experience and turn it off by default * remove useless constructor * Clean up charts code (#1313) * clean up chart code * remove arg * Expand the counterfactual flyout to cover the full page (#1315) Signed-off-by: Gaurav Gupta <[email protected]> * Bump minimist from 1.2.5 to 1.2.6 (#1292) * Bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> * minimist ^1.2.6 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * fix random node download failures by upgrading to latest github action with retry logic (#1317) * Add dataset cohort table to new ModelOverview experience (#1314) * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * Add installation instructions for raiwidgets to README (#1320) * refactor RAIInsights into RAIInsightsBase class for basic functionality (#1284) * Fix what if counterfactual header and description text misaligned (#1316) * align * e2e * add clear temporary cohort button to error analysis (#1322) * Raise UserConfigValidationException in case no model but valid model serializer (#1325) Signed-off-by: Gaurav Gupta <[email protected]> * Add test case for handling different types in causal (#1321) Signed-off-by: Gaurav Gupta <[email protected]> * show shift to an empty cohort in tree view as an empty node (#1318) * Bug fixing (#1326) * Move chart description up and remove scroll bar * Change string * Add box outlier for dataset explorer (#1323) * add outlier for dataset explorer * name * update string when no datapoint selected (#1331) * Fix Big empty space for featureImportance chart (#1328) * legend * removed invalid test case * constant * Disable save as new cohort button if nothing is selected in error tree (#1327) * Add disaggregated analysis table/heatmap (#1332) * disaggregated analysis changes only * lintfix * Change warning message to user exception for model type and task type mismatch (#1330) * Change warning message to user exception for model type and task type mismatch Signed-off-by: Gaurav Gupta <[email protected]> * Fix flake8 errors Signed-off-by: Gaurav Gupta <[email protected]> * Change the counterfactual text color from black to grey (#1337) Signed-off-by: Gaurav Gupta <[email protected]> * Limit each component description width up to 750px for readability (#1336) * limit description width up to 750px * export maxWidth from a common place * block empty cohort creation in RAI Dashboard (#1335) * Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort * Rename counterfactual style files to confirm with *.styles.ts (#1338) Signed-off-by: Gaurav Gupta <[email protected]> * Add disaggregated analysis table to Model Overview (#1341) * pull in changes for disaggregated analysis * add styles file * add textured NaN cells * module import for textured cells and grid y axis * lintfix * use combobox for dropdown rather than dropdown * lintfix * Rename causal style files to confirm with *.styles.ts (#1342) Signed-off-by: Gaurav Gupta <[email protected]> * update responsibleai to interpret-community 0.25.0 (#1343) * All component title and descriptions should be aligned (#1346) * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * Remove 5K limit blurb from local explanations tab (#1347) Signed-off-by: Gaurav Gupta <[email protected]> * Sort features by default in counterfactual flyout (#1312) * Sort features by default in counterfactual flyout Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> * attempt to fix test Signed-off-by: Gaurav Gupta <[email protected]> * Remove check Signed-off-by: Gaurav Gupta <[email protected]> * Bump moment from 2.28.0 to 2.29.2 (#1333) Bumps [moment](https://github.com/moment/moment) from 2.28.0 to 2.29.2. - [Release notes](https://github.com/moment/moment/releases) - [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md) - [Commits](https://github.com/moment/moment/compare/2.28.0...2.29.2) --- updated-dependencies: - dependency-name: moment dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <[email protected]> * Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text (#1350) * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1344) * update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected (#1349) Co-authored-by: Vinutha Karanth <[email protected]> * update erroranalysis to 0.2.1 and remove some duplicate dependencies (#1334) * disable turbo checking for large amount of data (#1351) Signed-off-by: Ke Xu <[email protected]> * force re-render when chart type changes (#1354) Signed-off-by: Ke Xu <[email protected]> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1352) * Rename output column name in the counterfactual flyout (#1353) Signed-off-by: Gaurav Gupta <[email protected]> * Show column chart for categorical feature in data explorer (#1355) * Show column chart for categorical feature in data explorer * address comments * update fluentui (#1356) Signed-off-by: Ke Xu <[email protected]> * update code owner (#1308) * update code owner Signed-off-by: Ke Xu <[email protected]> * remove dup Signed-off-by: Ke Xu <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * update version to match studio (#1357) Signed-off-by: Ke Xu <[email protected]> * alignment fixes (#1359) * Add charts for new model overview experience (#1348) * rename OverallTable to DisaggregatedMetricsTable and move to core-ui * Copy the ModelPerformanceTab into model-assessment and rename to ModelOverview * reference OverallTable again in fairness widget * refactor core chart component out into core-ui * refactor out core chart component into core-ui * lintfix * undo tsconfig.lib.json change * fix chartAndConfigsId in tests * lintfix * add table for cohort metrics and add dropdown metric selector, add new metrics * add new metrics * undo unwanted changes * fix casing * add superscript 2 for r-squared * update tests to reflect new metrics * lintfix * add feature flag * fix mae * fix mae calculation * first version of new model overview table * get probability distribution box plot to work * add feature flag for new model overview experience and turn it off by default * add highchart heatmap helper class * remove custom styling * add erroneously deleted line back * remove useless constructor * modularize model overview * show outliers in box plot, fix positioning * remove showmetricsummary * refactor heatmap code into a common class * add featureDropdownRef to allow focusing * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * add chart config flyout (in progress) * add chart config flyout (in progress) * address feedback, use finalized color * adjust feature selection to disable options if limit is reached, add axis config buttons * select all via dropdown * lintfix * refactor box plot calculations and rendering * add style file * textured NaN cells, grid labels on y axis * standardize box plots to use fences * fix merge issues * unify box plot tooltip formatting code, fix bar chart sizing issue * small fixes * rearrange feature dropdown * lintfix * remove commented out code * remove box plot tooltip customization * lintfix * add a few unit tests * unit tests for smaller utilities, localization fixes, consistent flyout flow with confirm/cancel buttons * lintfix * fix chart config flyout update * fix test case * rename files for lint * file rename for lint * release rai-core-flask 0.3.0 (#1361) * upgrade python version used with flask CI to fix segfault error (#1363) * release raiwidgets and responsibleai v0.18.0 (#1360) * fix two bugs (#1364) * Add pre-built cohort into adult census notebook (#1243) * [WIP] Add pre-built cohort into adult census notebook Signed-off-by: Gaurav Gupta <[email protected]> * erroranalysis version bump in raiwidgets to 0.1.31 (#1245) * Make cohrtData empty list in case no pre-bdefined cohorts are injected (#1247) Signed-off-by: Gaurav Gupta <[email protected]> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add regression test for pre-defined cohorts in raiwidgets (#1249) Signed-off-by: Gaurav Gupta <[email protected]> * color (#1248) * Add feature importance box & bar chart (#1241) * refactor * build * build * temp * temp * temp * temp * box * cache * e2e * e2e * fix * e2e fix * e2e * fix e2e * widget * widget * fix * widget * e2e * e2e * e2e * test * test * PreBuilt cohorts UX changes (#1242) * Intial SDK implementation cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add basic validationf for cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add serialized version of cohort config to ResponsibleAiDashboard Signed-off-by: Gaurav Gupta <[email protected]> * Add more tests cohorts Signed-off-by: Gaurav Gupta <[email protected]> * fix broken builds due to pip upgrade which broke pip-tools (#1185) * refactor matrix filter and area state to be private static (#1179) * Change variable name Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add cohort data to dashboard e2e Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohorts filters Signed-off-by: Gaurav Gupta <[email protected]> * Document various data validation for cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add new interfaces for pre-built cohort Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add prebuilt cohort walking logic in UI and add more data validation scenarios Signed-off-by: Gaurav Gupta <[email protected]> * Add basic data validation checks Signed-off-by: Gaurav Gupta <[email protected]> * Add logic to translate the Index cohort filter Signed-off-by: Gaurav Gupta <[email protected]> * Remove commented out code Signed-off-by: Gaurav Gupta <[email protected]> * Add SDK validations for Index based cohort filter Signed-off-by: Gaurav Gupta <[email protected]> * Add code for validating classification outcome Signed-off-by: Gaurav Gupta <[email protected]> * Add error filter validations and add tests Signed-off-by: Gaurav Gupta <[email protected]> * Add fake cohorts for regression dataset Signed-off-by: Gaurav Gupta <[email protected]> * Add fake cohorts for multi-class classification dataset Signed-off-by: Gaurav Gupta <[email protected]> * Add handling of regression filter Signed-off-by: Gaurav Gupta <[email protected]> * Add support for classification outcome in UI Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for Predicted Y and True Y cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add UI code to handle prediced Y and true Y for pre-built cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add cohort validation with test data to raiwidgets Signed-off-by: Gaurav Gupta <[email protected]> * Add tests for validating Predicted/True Y cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add UI support for TrueY/PredictedY for classification Signed-off-by: Gaurav Gupta <[email protected]> * Rename cohort_filter_list to cohort_list Signed-off-by: Gaurav Gupta <[email protected]> * Rename UI varibles to match SDK Signed-off-by: Gaurav Gupta <[email protected]> * Fix duplicate cohort name Signed-off-by: Gaurav Gupta <[email protected]> * Add SDK cohorts to notebook Signed-off-by: Gaurav Gupta <[email protected]> * Add dataset validations and add categorical features Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for categorical_features Signed-off-by: Gaurav Gupta <[email protected]> * Fix sorted imports Signed-off-by: Gaurav Gupta <[email protected]> * Add code for translating categorical values Signed-off-by: Gaurav Gupta <[email protected]> * Move cohort processing to a separate file Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Refactor cohort translated function into different small functions Signed-off-by: Gaurav Gupta <[email protected]> * Change to lowercase for outcome Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Refactor cohort_list validations and converge pytest common functions into fixtures Signed-off-by: Gaurav Gupta <[email protected]> * Add conftest into raiwidgets tests Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for cohort list Signed-off-by: Gaurav Gupta <[email protected]> * Add cohortData test Signed-off-by: Gaurav Gupta <[email protected]> * Fix sorted imports Signed-off-by: Gaurav Gupta <[email protected]> * isort fix Signed-off-by: Gaurav Gupta <[email protected]> * Add UI unit tests for cohort translation Signed-off-by: Gaurav Gupta <[email protected]> * Add more checks in UI uni test Signed-off-by: Gaurav Gupta <[email protected]> * Add UI tests for regression cohorts Signed-off-by: Gaurav Gupta <[email protected]> * REmove notebook change Signed-off-by: Gaurav Gupta <[email protected]> * Fix typescript build Signed-off-by: Gaurav Gupta <[email protected]> * Change cohort filter values so that cohort filters non-zero points Signed-off-by: Gaurav Gupta <[email protected]> * Fix for empty cohort list Signed-off-by: Gaurav Gupta <[email protected]> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Propagate error strings instead of raising exceptions Signed-off-by: Gaurav Gupta <[email protected]> * Fix code issues Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <[email protected]> * Add missing file Signed-off-by: Gaurav Gupta <[email protected]> * fix notebook build failures due to pywinpty dependency release failing in python 3.6 (#1257) * fix notebook build failures due to pywinpty dependency release failing in python 3.6 * build pywinpty from conda instead * add lowerbound * fixup * fixup * Add supported models and data types to README.md responsibleai (#1259) Signed-off-by: Gaurav Gupta <[email protected]> * make getting-started notebook a markdown file showing APIs (#1223) * refactor tabs out of RAI dashboard into a separate component (#1256) * Add individual causal scatter chart (#1258) * temp * refactor * test * style fix * comment * minor fix to url for responsibleai package in setup.py (#1260) * Fix UX e2e tests and address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix eslint Signed-off-by: Gaurav Gupta <[email protected]> * Address review comments Signed-off-by: Gaurav Gupta <[email protected]> * Reset the number of samples in test dataset Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Bo Zhang <[email protected]> * Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb (#1252) * [WIP] Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Fix the e2e test for notebook Signed-off-by: Gaurav Gupta <[email protected]> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <[email protected]> * Add missing file Signed-off-by: Gaurav Gupta <[email protected]> * Fix cohort namespace Signed-off-by: Gaurav Gupta <[email protected]> * minor fix to url for responsibleai package in setup.py (#1260) * Counterfactual Chart: Correct target description according to task_type (#1261) * Counterfactual Chart: Correct target description according to task_type Signed-off-by: Gaurav Gupta <[email protected]> * Change function name Signed-off-by: Gaurav Gupta <[email protected]> * Make Range lowercase Signed-off-by: Gaurav Gupta <[email protected]> * fix whitespace in values of adult census income dataset (#1263) * Add what-If scatter chart from highchart lib (#1262) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * add a builddebug yarn command to build UX locally which can be debugged in browser e2e (#1265) * allow rai text insights to work with RAI dashboard (#1269) * remove duplicate code in explanation dashboard (#1266) * Individual causal style responsive (#1268) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * Causal Style * Allow duplicating cohorts multiple times (#1274) * allow duplicating a cohort more than once * lintfix * Disable column header highlighting on hover in IndividualFeatureImportanceView (#1272) * disable column highlight on hover * lintfix * Rename new cohorts from "Unsaved" to "Temporary cohort" (#1273) * rename Unsaved to Temporary cohort * localize temp cohort * Counterfactual style refactor (#1275) * style refactor * test * test * test * fix * Don't change cursor on hover over cohort name * Fix (#1281) * fix cohort info styling (#1277) * fix readme link to fairness and interpretability example notebook (#1282) * add new RAI Utils package for common utilities shared across RAI packages (#1280) * Add ICE chart (#1283) * Fix * ice chart * ic * test * test * update docstring for explanation dashboard in regards to min number of rows (#1271) * make builds more reliable by adding retry logic to urlretrieve calls in notebooks (#1218) * upgrade pytest to 7.0.1, remove mock and updgrade pytest-mock to 3.6.1 (#1287) * remove deprecated codecov parameter (#1293) * Fix min/max special case in cohort filter creation with "in the range of" (#1279) * fix logic in the case that min or max are zero * lintfix * Rename 'Dashboard navigation' to 'Dashboard configuration' (#1291) * Rename 'Dashboard navigation' to 'Dashboard configuration' Signed-off-by: Gaurav Gupta <[email protected]> * Notebook change Signed-off-by: Gaurav Gupta <[email protected]> * Add raiutils to PR template (#1290) * fix heatmap bug (#1297) * Make "save and switch" work from cohort settings (#1276) * make save and switch work * fix naming * lintfix * adjustment according to Ilya's comment * lintfix * add retry logic to codecov step and only upload results for one python version (#1298) * add github action to release raiutils to pypi (#1294) * Add highchart for Dataset Explorer (#1286) * test * style * click * fix test * fix test * test * test * test * test * Update requirements-linting.txt to add flake8-pytest-style (#1296) * Fix sort abs (#1299) * Rename "base cohort" to "global cohort" (#1278) * change base cohort to global cohort * fix spelling * lintfix * fix codecov comment not appearing on PRs (#1302) * take absolute value of error calculation for regression scenario (#1301) * Limit individual feature importance selection to up to 5 (#1305) * update feature importance string * limit selection to up to 5 * add group count * remove message bar, show info icon instead * update e2e locator * fix E2E failure on feature importance * add ariaLabel for expand collapse button * add renderOnNewLayer props * Add error message for counterfactual panel (#1310) * add error message for counterfactual * update error message in camel case to fix build error * Add to_json() and from_json() methods to Cohort class (#1300) * Add to_json() and from_json() methods to Cohort class Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix linting Signed-off-by: Gaurav Gupta <[email protected]> * Add a highchart heatmap helper class (#1307) * add highchart heatmap helper class * add erroneously deleted line back * Fix cohort setting string (#1304) * Fix string * remove none * name * test * Fix all data style (#1303) * Add a feature flag for the new model overview experience (#1306) * add feature flag for new model overview experience and turn it off by default * remove useless constructor * Clean up charts code (#1313) * clean up chart code * remove arg * Expand the counterfactual flyout to cover the full page (#1315) Signed-off-by: Gaurav Gupta <[email protected]> * Bump minimist from 1.2.5 to 1.2.6 (#1292) * Bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> * minimist ^1.2.6 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * fix random node download failures by upgrading to latest github action with retry logic (#1317) * Add dataset cohort table to new ModelOverview experience (#1314) * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * Add installation instructions for raiwidgets to README (#1320) * refactor RAIInsights into RAIInsightsBase class for basic functionality (#1284) * Fix what if counterfactual header and description text misaligned (#1316) * align * e2e * add clear temporary cohort button to error analysis (#1322) * Raise UserConfigValidationException in case no model but valid model serializer (#1325) Signed-off-by: Gaurav Gupta <[email protected]> * Add test case for handling different types in causal (#1321) Signed-off-by: Gaurav Gupta <[email protected]> * show shift to an empty cohort in tree view as an empty node (#1318) * Bug fixing (#1326) * Move chart description up and remove scroll bar * Change string * Add box outlier for dataset explorer (#1323) * add outlier for dataset explorer * name * update string when no datapoint selected (#1331) * Fix Big empty space for featureImportance chart (#1328) * legend * removed invalid test case * constant * Disable save as new cohort button if nothing is selected in error tree (#1327) * Add disaggregated analysis table/heatmap (#1332) * disaggregated analysis changes only * lintfix * Change warning message to user exception for model type and task type mismatch (#1330) * Change warning message to user exception for model type and task type mismatch Signed-off-by: Gaurav Gupta <[email protected]> * Fix flake8 errors Signed-off-by: Gaurav Gupta <[email protected]> * Change the counterfactual text color from black to grey (#1337) Signed-off-by: Gaurav Gupta <[email protected]> * Limit each component description width up to 750px for readability (#1336) * limit description width up to 750px * export maxWidth from a common place * block empty cohort creation in RAI Dashboard (#1335) * Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort * Rename counterfactual style files to confirm with *.styles.ts (#1338) Signed-off-by: Gaurav Gupta <[email protected]> * Add disaggregated analysis table to Model Overview (#1341) * pull in changes for disaggregated analysis * add styles file * add textured NaN cells * module import for textured cells and grid y axis * lintfix * use combobox for dropdown rather than dropdown * lintfix * Rename causal style files to confirm with *.styles.ts (#1342) Signed-off-by: Gaurav Gupta <[email protected]> * update responsibleai to interpret-community 0.25.0 (#1343) * All component title and descriptions should be aligned (#1346) * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * Remove 5K limit blurb from local explanations tab (#1347) Signed-off-by: Gaurav Gupta <[email protected]> * Sort features by default in counterfactual flyout (#1312) * Sort features by default in counterfactual flyout Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> * attempt to fix test Signed-off-by: Gaurav Gupta <[email protected]> * Remove check Signed-off-by: Gaurav Gupta <[email protected]> * Bump moment from 2.28.0 to 2.29.2 (#1333) Bumps [moment](https://github.com/moment/moment) from 2.28.0 to 2.29.2. - [Release notes](https://github.com/moment/moment/releases) - [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md) - [Commits](https://github.com/moment/moment/compare/2.28.0...2.29.2) --- updated-dependencies: - dependency-name: moment dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <[email protected]> * Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text (#1350) * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1344) * update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected (#1349) Co-authored-by: Vinutha Karanth <[email protected]> * update erroranalysis to 0.2.1 and remove some duplicate dependencies (#1334) * disable turbo checking for large amount of data (#1351) Signed-off-by: Ke Xu <[email protected]> * force re-render when chart type changes (#1354) Signed-off-by: Ke Xu <[email protected]> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1352) * Rename output column name in the counterfactual flyout (#1353) Signed-off-by: Gaurav Gupta <[email protected]> * Show column chart for categorical feature in data explorer (#1355) * Show column chart for categorical feature in data explorer * address comments * update fluentui (#1356) Signed-off-by: Ke Xu <[email protected]> * update code owner (#1308) * update code owner Signed-off-by: Ke Xu <[email protected]> * remove dup Signed-off-by: Ke Xu <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * update version to match studio (#1357) Signed-off-by: Ke Xu <[email protected]> * alignment fixes (#1359) * Add charts for new model overview experience (#1348) * rename OverallTable to DisaggregatedMetricsTable and move to core-ui * Copy the ModelPerformanceTab into model-assessment and rename to ModelOverview * reference OverallTable again in fairness widget * refactor core chart component out into core-ui * refactor out core chart component into core-ui * lintfix * undo tsconfig.lib.json change * fix chartAndConfigsId in tests * lintfix * add table for cohort metrics and add dropdown metric selector, add new metrics * add new metrics * undo unwanted changes * fix casing * add superscript 2 for r-squared * update tests to reflect new metrics * lintfix * add feature flag * fix mae * fix mae calculation * first version of new model overview table * get probability distribution box plot to work * add feature flag for new model overview experience and turn it off by default * add highchart heatmap helper class * remove custom styling * add erroneously deleted line back * remove useless constructor * modularize model overview * show outliers in box plot, fix positioning * remove showmetricsummary * refactor heatmap code into a common class * add featureDropdownRef to allow focusing * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * add chart config flyout (in progress) * add chart config flyout (in progress) * address feedback, use finalized color * adjust feature selection to disable options if limit is reached, add axis config buttons * select all via dropdown * lintfix * refactor box plot calculations and rendering * add style file * textured NaN cells, grid labels on y axis * standardize box plots to use fences * fix merge issues * unify box plot tooltip formatting code, fix bar chart sizing issue * small fixes * rearrange feature dropdown * lintfix * remove commented out code * remove box plot tooltip customization * lintfix * add a few unit tests * unit tests for smaller utilities, localization fixes, consistent flyout flow with confirm/cancel buttons * lintfix * fix chart config flyout update * fix test case * rename files for lint * file rename for lint * release rai-core-flask 0.3.0 (#1361) * upgrade python version used with flask CI to fix segfault error (#1363) * release raiwidgets and responsibleai v0.18.0 (#1360) * fix two bugs (#1364) * Add pre-built cohort into adult census notebook (#1243) * [WIP] Add pre-built cohort into adult census notebook Signed-off-by: Gaurav Gupta <[email protected]> * erroranalysis version bump in raiwidgets to 0.1.31 (#1245) * Make cohrtData empty list in case no pre-bdefined cohorts are injected (#1247) Signed-off-by: Gaurav Gupta <[email protected]> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add regression test for pre-defined cohorts in raiwidgets (#1249) Signed-off-by: Gaurav Gupta <[email protected]> * color (#1248) * Add feature importance box & bar chart (#1241) * refactor * build * build * temp * temp * temp * temp * box * cache * e2e * e2e * fix * e2e fix * e2e * fix e2e * widget * widget * fix * widget * e2e * e2e * e2e * test * test * PreBuilt cohorts UX changes (#1242) * Intial SDK implementation cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add basic validationf for cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add serialized version of cohort config to ResponsibleAiDashboard Signed-off-by: Gaurav Gupta <[email protected]> * Add more tests cohorts Signed-off-by: Gaurav Gupta <[email protected]> * fix broken builds due to pip upgrade which broke pip-tools (#1185) * refactor matrix filter and area state to be private static (#1179) * Change variable name Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add cohort data to dashboard e2e Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohorts filters Signed-off-by: Gaurav Gupta <[email protected]> * Document various data validation for cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add new interfaces for pre-built cohort Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add prebuilt cohort walking logic in UI and add more data validation scenarios Signed-off-by: Gaurav Gupta <[email protected]> * Add basic data validation checks Signed-off-by: Gaurav Gupta <[email protected]> * Add logic to translate the Index cohort filter Signed-off-by: Gaurav Gupta <[email protected]> * Remove commented out code Signed-off-by: Gaurav Gupta <[email protected]> * Add SDK validations for Index based cohort filter Signed-off-by: Gaurav Gupta <[email protected]> * Add code for validating classification outcome Signed-off-by: Gaurav Gupta <[email protected]> * Add error filter validations and add tests Signed-off-by: Gaurav Gupta <[email protected]> * Add fake cohorts for regression dataset Signed-off-by: Gaurav Gupta <[email protected]> * Add fake cohorts for multi-class classification dataset Signed-off-by: Gaurav Gupta <[email protected]> * Add handling of regression filter Signed-off-by: Gaurav Gupta <[email protected]> * Add support for classification outcome in UI Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for Predicted Y and True Y cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add UI code to handle prediced Y and true Y for pre-built cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add cohort validation with test data to raiwidgets Signed-off-by: Gaurav Gupta <[email protected]> * Add tests for validating Predicted/True Y cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add UI support for TrueY/PredictedY for classification Signed-off-by: Gaurav Gupta <[email protected]> * Rename cohort_filter_list to cohort_list Signed-off-by: Gaurav Gupta <[email protected]> * Rename UI varibles to match SDK Signed-off-by: Gaurav Gupta <[email protected]> * Fix duplicate cohort name Signed-off-by: Gaurav Gupta <[email protected]> * Add SDK cohorts to notebook Signed-off-by: Gaurav Gupta <[email protected]> * Add dataset validations and add categorical features Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for categorical_features Signed-off-by: Gaurav Gupta <[email protected]> * Fix sorted imports Signed-off-by: Gaurav Gupta <[email protected]> * Add code for translating categorical values Signed-off-by: Gaurav Gupta <[email protected]> * Move cohort processing to a separate file Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Refactor cohort translated function into different small functions Signed-off-by: Gaurav Gupta <[email protected]> * Change to lowercase for outcome Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Refactor cohort_list validations and converge pytest common functions into fixtures Signed-off-by: Gaurav Gupta <[email protected]> * Add conftest into raiwidgets tests Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for cohort list Signed-off-by: Gaurav Gupta <[email protected]> * Add cohortData test Signed-off-by: Gaurav Gupta <[email protected]> * Fix sorted imports Signed-off-by: Gaurav Gupta <[email protected]> * isort fix Signed-off-by: Gaurav Gupta <[email protected]> * Add UI unit tests for cohort translation Signed-off-by: Gaurav Gupta <[email protected]> * Add more checks in UI uni test Signed-off-by: Gaurav Gupta <[email protected]> * Add UI tests for regression cohorts Signed-off-by: Gaurav Gupta <[email protected]> * REmove notebook change Signed-off-by: Gaurav Gupta <[email protected]> * Fix typescript build Signed-off-by: Gaurav Gupta <[email protected]> * Change cohort filter values so that cohort filters non-zero points Signed-off-by: Gaurav Gupta <[email protected]> * Fix for empty cohort list Signed-off-by: Gaurav Gupta <[email protected]> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Propagate error strings instead of raising exceptions Signed-off-by: Gaurav Gupta <[email protected]> * Fix code issues Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <[email protected]> * Add missing file Signed-off-by: Gaurav Gupta <[email protected]> * fix notebook build failures due to pywinpty dependency release failing in python 3.6 (#1257) * fix notebook build failures due to pywinpty dependency release failing in python 3.6 * build pywinpty from conda instead * add lowerbound * fixup * fixup * Add supported models and data types to README.md responsibleai (#1259) Signed-off-by: Gaurav Gupta <[email protected]> * make getting-started notebook a markdown file showing APIs (#1223) * refactor tabs out of RAI dashboard into a separate component (#1256) * Add individual causal scatter chart (#1258) * temp * refactor * test * style fix * comment * minor fix to url for responsibleai package in setup.py (#1260) * Fix UX e2e tests and address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix eslint Signed-off-by: Gaurav Gupta <[email protected]> * Address review comments Signed-off-by: Gaurav Gupta <[email protected]> * Reset the number of samples in test dataset Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Bo Zhang <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Change cohort name Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Bo Zhang <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: tongy-msft <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <[email protected]> Co-authored-by: Vinutha Karanth <[email protected]> * Add more context to download print in fetch_dataset (#1368) * Add missing desired range to counterfactual chart (#1366) Signed-off-by: Gaurav Gupta <[email protected]> * Add a summary for disaggregated analysis showing ratio and difference (#1367) * add fairness metric table, update existing tables (order, N/A values, grid axis label lengths) * lintfix * merge fixes * addressed eslint issue for null value by creating a dedicated file, localized N/A, updated tooltip text to be wrapped (+ associated unit tests) * keep feature flag * Update PR template to be more intuitive (#1116) * Updating PR template to make sure there's at least 1 box that can be ticked. * replace checkboxes with GitHub emojis for checkmark and x to prevent GitHub from interpreting it as a task list * update PR template based on feedback * lintfix * Update CODEOWNERS file (#1369) * Use global cohort for disaggregated analysis (#1370) * use global cohort for disaggregated analysis and warn user about it * lintfix * Change procedural loops to functions map() and filter() (#1255) * Change procedural loops to function map() and filter() Signed-off-by: Gaurav Gupta <[email protected]> * Fix review comment Signed-off-by: Gaurav Gupta <[email protected]> * Add accuracy as multiclass metric (#1371) * support accuracy for multiclass classification * lintfix * update code owner (#1372) Signed-off-by: Ke Xu <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Rename _validate_model_analysis_input_parameters() to _validate_rai_insights_input_parameters (#1373) Signed-off-by: Gaurav Gupta <[email protected]> * Cache and serialize the counterfactual explainer (#1374) * Cache and serialize the counterfactual explainer Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> * Add a top-level pivot to model overview (#1375) * show only disaggregated analysis OR only dataset cohorts * lintfix * basec -> basic typo * Add a toggle to model overview to turn heatmap colors on and off (#1376) * add visual display toggle * lintfix * simplify onVisualDisplayToggleChange method * color replacements * fluentui colors * localization, style file * lintfix * Create rai-e2e package for tests shared with AML (#1378) * update Signed-off-by: vinutha karanth <[email protected]> * update Signed-off-by: vinutha karanth <[email protected]> * index update Signed-off-by: vinutha karanth <[email protected]> * remove files from wid-e2e Signed-off-by: vinutha karanth <[email protected]> * update datashape Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * add pyspark support to tree surrogate model in error analysis (#1251) * Create e2e package for tests shared with AML (#1383) * new buildable folder Signed-off-by: vinutha karanth <[email protected]> * remove rai-e2e lib Signed-off-by: vinutha karanth <[email protected]> * add license Signed-off-by: vinutha karanth <[email protected]> * update Signed-off-by: vinutha karanth <[email protected]> * Add feature and metric configuration flyouts to model overview (#1381) * show only disaggregated analysis OR only dataset cohorts * lintfix * basec -> basic typo * feature config * merge fixes * metric config * lintfix * wrap metric descriptions * lintfix * move constant to constants file to avoid circular dependency, lintfix * move inline code into functions * Add probability distribution line chart to model overview (#1382) * line chart for prob distribution * lintfix * fix race condition in tree view update code which can cause root stats to be ignored on selected cohort in RAI dashboard static view (#1386) * String updates for model overview (#1387) * string updates for model overview * lintfix * Handle the loading of causal models more gracefully (#1377) * Handle the loading of causal models more gracefully Signed-off-by: Gaurav Gupta <[email protected]> * Address code review commensts and fix test Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comment Signed-off-by: Gaurav Gupta <[email protected]> * Some fixes for data explorer, heatmap, counterfactual components (#1389) * add x axis and update heatmap * cf custom point color should be consistent * cf custom point name should be displayed * disable causal treatment whatif in static view (#1391) * fix big data set for counter factual chart (#1393) Signed-off-by: Ke Xu <[email protected]> * alignment fixes for causal (#1392) * Fix feature selection state carry-over bug in model overview (#1390) * Provide feature flight support to dashboard, test environment, and Python widget (#1385) * Python part of exposing flights * add feature flighting * fix path for test case * update snapshot * add none constant, contrib guide update, and unit tests for feature_flights param * flake8 * isort * release raiwidgets and responsibleai v0.18.1 (#1413) * Add warning in counterfactual manager when unable to load explainer (#1412) * Add warning in counterfactual manager when unable to load explainer Signed-off-by: Gaurav Gupta <[email protected]> * Fix linting Signed-off-by: Gaurav Gupta <[email protected]> * Bug fixes on 'Set value' not copying over feature values correctly in what if counterfactual panel (#1416) * fix cf set value bug * remove comment * fix error on machines with pyspark installed where passed dataframe is not spark pandas (#1415) * add postbuild branch trigger (#1417) * Fix causal UI strings according to classification/regression tasks (#1419) * Fix causal UI strings according to classification/regression tasks Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint error Signed-off-by: Gaurav Gupta <[email protected]> * Fix UI test Signed-off-by: Gaurav Gupta <[email protected]> * Fix description for model overview (#1425) * fix description for model overview * keep new description for new model overview * fix failing to create error report when filter_features is empty list (#1421) * filter out missing values from what if dropdown to prevent explanation dashboard from crashing (#1418) * Remove |Set Value| blurb in case it is not availble in counterfactual panel (#1426) Signed-off-by: Gaurav Gupta <[email protected]> * Add y-axis description to counterfactual feature importance chart (#1423) Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: xuke444 <[email protected]> * Add the user class name to causal UI strings (#1422) * Fix causal UI strings according to classification/regression tasks Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint error Signed-off-by: Gaurav Gupta <[email protected]> * Fix UI test Signed-off-by: Gaurav Gupta <[email protected]> * Add the user class name to causal UI strings Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: xuke444 <[email protected]> * fix math.min / max for array size more than 10^7 (#1427) Signed-off-by: Ke Xu <[email protected]> * upgrade pytest and lightgbm to try to fix random pytest segfault test failures (#1424) s * fix flaky notebook causing build failures by adding retry logic (#1431) * Upper bound SciKit-Learn to address freeze in causal (#1432)  ## Description Replaces #1429 to address #1430 . Causal analysis is getting stuck with the latest release of SciKit-Learn. This contains: - Test case which gets stuck with SciKit-Learn 1.1.0 - Upper bound on SciKit-Learn in `requirements.txt` ## Checklist  - [x] I have added screenshots above for all UI changes. - [x] Documentation was updated if it was needed. - [x] New tests were added or changes were manually verified. Signed-off-by: Richard Edgar <[email protected]> * fix dependency chart axis updating with incorrect values in explanation dashboard (#1437) * fix codecov and widget test screenshot uploads (#1428) * release raiwidgets and responsibleai v0.18.2 (#1439) * fix (#1441) Signed-off-by: vinutha karanth <[email protected]> * Fix cohort name conflict and not run few tests for AML (#1442) * fix Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * Few e2e tests changes to accommodate AML static tests (#1445) * update Signed-off-by: vinutha karanth <[email protected]> * update Signed-off-by: vinutha karanth <[email protected]> * Fix locators logic for string features - data explorer and model statistics components (#1446) * update Signed-off-by: vinutha karanth <[email protected]> * update Signed-off-by: vinutha karanth <[email protected]> * fix Signed-off-by: vinutha karanth <[email protected]> * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * fix Signed-off-by: vinutha karanth <[email protected]> * Add more unittests RAI dashboard input class (#1448) * Add unit tests for ResponsibleAIDashboardInput Signed-off-by: Gaurav Gupta <[email protected]> * Add more tests Signed-off-by: Gaurav Gupta <[email protected]> * Fix imports Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Bo Zhang <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: tongy-msft <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <[email protected]> Co-authored-by: Vinutha Karanth <[email protected]> Co-authored-by: Richard Edgar <[email protected]>

tongyu-microsoft and others added 3 commits April 12, 2022 12:58

add error message for invalid value

ef3c886

update shift cohort to switch cohort

2cff753

Merge branch 'main' into tongy/cohortInput

cc96e44

imatiach-msft reviewed Apr 12, 2022

View reviewed changes

libs/localization/src/lib/en.json Show resolved Hide resolved

imatiach-msft reviewed Apr 12, 2022

View reviewed changes

libs/localization/src/lib/en.json Show resolved Hide resolved

imatiach-msft approved these changes Apr 12, 2022

View reviewed changes

imatiach-msft reviewed Apr 12, 2022

View reviewed changes

libs/core-ui/src/lib/Cohort/ManualCohortManagement/CohortEditorFilter.tsx Show resolved Hide resolved

tongyu-microsoft added 2 commits April 12, 2022 13:39

merge main

b6972ca

Merge branch 'tongy/cohortInput' of https://github.com/microsoft/resp…

3eacfa9

…onsible-ai-widgets into tongy/cohortInput

tongyu-microsoft enabled auto-merge (squash) April 12, 2022 20:49

tongyu-microsoft merged commit 9c30636 into main Apr 12, 2022

tongyu-microsoft deleted the tongy/cohortInput branch April 12, 2022 22:08

imatiach-msft mentioned this pull request Apr 19, 2022

release raiwidgets and responsibleai v0.18.0 #1360

Merged

19 tasks

gaugup pushed a commit that referenced this pull request Apr 21, 2022

Add warning message in cohort editor for invalid input value; Update …

753c35d

…'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort

gaugup pushed a commit that referenced this pull request May 26, 2022

Add warning message in cohort editor for invalid input value; Update …

d536a01

…'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' #1339

Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' #1339

tongyu-microsoft commented Apr 12, 2022 •

edited

Loading

romanlutz commented Apr 12, 2022

tongyu-microsoft commented Apr 12, 2022 •

edited

Loading

codecov-commenter commented Apr 12, 2022 •

edited

Loading

github-actions bot commented Apr 12, 2022

github-actions bot commented Apr 12, 2022

github-actions bot commented Apr 12, 2022

Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' #1339

Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' #1339

Conversation

tongyu-microsoft commented Apr 12, 2022 • edited Loading

Description

Areas changed

Tests

Screenshots (if appropriate):

Documentation:

romanlutz commented Apr 12, 2022

tongyu-microsoft commented Apr 12, 2022 • edited Loading

codecov-commenter commented Apr 12, 2022 • edited Loading

Codecov Report

github-actions bot commented Apr 12, 2022

github-actions bot commented Apr 12, 2022

github-actions bot commented Apr 12, 2022

tongyu-microsoft commented Apr 12, 2022 •

edited

Loading

tongyu-microsoft commented Apr 12, 2022 •

edited

Loading

codecov-commenter commented Apr 12, 2022 •

edited

Loading