Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add more utilities into raiutils (#1295)
* Add more utilities into raiutils Signed-off-by: Gaurav Gupta <[email protected]> * Fix faulty file name Signed-off-by: Gaurav Gupta <[email protected]> * fix heatmap bug (#1297) * Make "save and switch" work from cohort settings (#1276) * make save and switch work * fix naming * lintfix * adjustment according to Ilya's comment * lintfix * add retry logic to codecov step and only upload results for one python version (#1298) * add github action to release raiutils to pypi (#1294) * Add highchart for Dataset Explorer (#1286) * test * style * click * fix test * fix test * test * test * test * test * Update requirements-linting.txt to add flake8-pytest-style (#1296) * Fix sort abs (#1299) * Rename "base cohort" to "global cohort" (#1278) * change base cohort to global cohort * fix spelling * lintfix * fix codecov comment not appearing on PRs (#1302) * take absolute value of error calculation for regression scenario (#1301) * Limit individual feature importance selection to up to 5 (#1305) * update feature importance string * limit selection to up to 5 * add group count * remove message bar, show info icon instead * update e2e locator * fix E2E failure on feature importance * add ariaLabel for expand collapse button * add renderOnNewLayer props * Add error message for counterfactual panel (#1310) * add error message for counterfactual * update error message in camel case to fix build error * Add to_json() and from_json() methods to Cohort class (#1300) * Add to_json() and from_json() methods to Cohort class Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix linting Signed-off-by: Gaurav Gupta <[email protected]> * Add a highchart heatmap helper class (#1307) * add highchart heatmap helper class * add erroneously deleted line back * Fix cohort setting string (#1304) * Fix string * remove none * name * test * Fix all data style (#1303) * Add a feature flag for the new model overview experience (#1306) * add feature flag for new model overview experience and turn it off by default * remove useless constructor * Clean up charts code (#1313) * clean up chart code * remove arg * Expand the counterfactual flyout to cover the full page (#1315) Signed-off-by: Gaurav Gupta <[email protected]> * Bump minimist from 1.2.5 to 1.2.6 (#1292) * Bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> * minimist ^1.2.6 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * fix random node download failures by upgrading to latest github action with retry logic (#1317) * Add dataset cohort table to new ModelOverview experience (#1314) * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * Add installation instructions for raiwidgets to README (#1320) * refactor RAIInsights into RAIInsightsBase class for basic functionality (#1284) * Fix what if counterfactual header and description text misaligned (#1316) * align * e2e * add clear temporary cohort button to error analysis (#1322) * Raise UserConfigValidationException in case no model but valid model serializer (#1325) Signed-off-by: Gaurav Gupta <[email protected]> * Add test case for handling different types in causal (#1321) Signed-off-by: Gaurav Gupta <[email protected]> * show shift to an empty cohort in tree view as an empty node (#1318) * Bug fixing (#1326) * Move chart description up and remove scroll bar * Change string * Add box outlier for dataset explorer (#1323) * add outlier for dataset explorer * name * update string when no datapoint selected (#1331) * Fix Big empty space for featureImportance chart (#1328) * legend * removed invalid test case * constant * Disable save as new cohort button if nothing is selected in error tree (#1327) * Add disaggregated analysis table/heatmap (#1332) * disaggregated analysis changes only * lintfix * Change warning message to user exception for model type and task type mismatch (#1330) * Change warning message to user exception for model type and task type mismatch Signed-off-by: Gaurav Gupta <[email protected]> * Fix flake8 errors Signed-off-by: Gaurav Gupta <[email protected]> * Change the counterfactual text color from black to grey (#1337) Signed-off-by: Gaurav Gupta <[email protected]> * Limit each component description width up to 750px for readability (#1336) * limit description width up to 750px * export maxWidth from a common place * block empty cohort creation in RAI Dashboard (#1335) * Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort * Rename counterfactual style files to confirm with *.styles.ts (#1338) Signed-off-by: Gaurav Gupta <[email protected]> * Add disaggregated analysis table to Model Overview (#1341) * pull in changes for disaggregated analysis * add styles file * add textured NaN cells * module import for textured cells and grid y axis * lintfix * use combobox for dropdown rather than dropdown * lintfix * Rename causal style files to confirm with *.styles.ts (#1342) Signed-off-by: Gaurav Gupta <[email protected]> * update responsibleai to interpret-community 0.25.0 (#1343) * All component title and descriptions should be aligned (#1346) * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * Remove 5K limit blurb from local explanations tab (#1347) Signed-off-by: Gaurav Gupta <[email protected]> * Sort features by default in counterfactual flyout (#1312) * Sort features by default in counterfactual flyout Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> * attempt to fix test Signed-off-by: Gaurav Gupta <[email protected]> * Remove check Signed-off-by: Gaurav Gupta <[email protected]> * Bump moment from 2.28.0 to 2.29.2 (#1333) Bumps [moment](https://github.com/moment/moment) from 2.28.0 to 2.29.2. - [Release notes](https://github.com/moment/moment/releases) - [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md) - [Commits](https://github.com/moment/moment/compare/2.28.0...2.29.2) --- updated-dependencies: - dependency-name: moment dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <[email protected]> * Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text (#1350) * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1344) * update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected (#1349) Co-authored-by: Vinutha Karanth <[email protected]> * update erroranalysis to 0.2.1 and remove some duplicate dependencies (#1334) * disable turbo checking for large amount of data (#1351) Signed-off-by: Ke Xu <[email protected]> * force re-render when chart type changes (#1354) Signed-off-by: Ke Xu <[email protected]> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1352) * Rename output column name in the counterfactual flyout (#1353) Signed-off-by: Gaurav Gupta <[email protected]> * Show column chart for categorical feature in data explorer (#1355) * Show column chart for categorical feature in data explorer * address comments * update fluentui (#1356) Signed-off-by: Ke Xu <[email protected]> * update code owner (#1308) * update code owner Signed-off-by: Ke Xu <[email protected]> * remove dup Signed-off-by: Ke Xu <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * update version to match studio (#1357) Signed-off-by: Ke Xu <[email protected]> * alignment fixes (#1359) * Add charts for new model overview experience (#1348) * rename OverallTable to DisaggregatedMetricsTable and move to core-ui * Copy the ModelPerformanceTab into model-assessment and rename to ModelOverview * reference OverallTable again in fairness widget * refactor core chart component out into core-ui * refactor out core chart component into core-ui * lintfix * undo tsconfig.lib.json change * fix chartAndConfigsId in tests * lintfix * add table for cohort metrics and add dropdown metric selector, add new metrics * add new metrics * undo unwanted changes * fix casing * add superscript 2 for r-squared * update tests to reflect new metrics * lintfix * add feature flag * fix mae * fix mae calculation * first version of new model overview table * get probability distribution box plot to work * add feature flag for new model overview experience and turn it off by default * add highchart heatmap helper class * remove custom styling * add erroneously deleted line back * remove useless constructor * modularize model overview * show outliers in box plot, fix positioning * remove showmetricsummary * refactor heatmap code into a common class * add featureDropdownRef to allow focusing * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * add chart config flyout (in progress) * add chart config flyout (in progress) * address feedback, use finalized color * adjust feature selection to disable options if limit is reached, add axis config buttons * select all via dropdown * lintfix * refactor box plot calculations and rendering * add style file * textured NaN cells, grid labels on y axis * standardize box plots to use fences * fix merge issues * unify box plot tooltip formatting code, fix bar chart sizing issue * small fixes * rearrange feature dropdown * lintfix * remove commented out code * remove box plot tooltip customization * lintfix * add a few unit tests * unit tests for smaller utilities, localization fixes, consistent flyout flow with confirm/cancel buttons * lintfix * fix chart config flyout update * fix test case * rename files for lint * file rename for lint * release rai-core-flask 0.3.0 (#1361) * upgrade python version used with flask CI to fix segfault error (#1363) * release raiwidgets and responsibleai v0.18.0 (#1360) * fix two bugs (#1364) * Add pre-built cohort into adult census notebook (#1243) * [WIP] Add pre-built cohort into adult census notebook Signed-off-by: Gaurav Gupta <[email protected]> * erroranalysis version bump in raiwidgets to 0.1.31 (#1245) * Make cohrtData empty list in case no pre-bdefined cohorts are injected (#1247) Signed-off-by: Gaurav Gupta <[email protected]> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add regression test for pre-defined cohorts in raiwidgets (#1249) Signed-off-by: Gaurav Gupta <[email protected]> * color (#1248) * Add feature importance box & bar chart (#1241) * refactor * build * build * temp * temp * temp * temp * box * cache * e2e * e2e * fix * e2e fix * e2e * fix e2e * widget * widget * fix * widget * e2e * e2e * e2e * test * test * PreBuilt cohorts UX changes (#1242) * Intial SDK implementation cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add basic validationf for cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add serialized version of cohort config to ResponsibleAiDashboard Signed-off-by: Gaurav Gupta <[email protected]> * Add more tests cohorts Signed-off-by: Gaurav Gupta <[email protected]> * fix broken builds due to pip upgrade which broke pip-tools (#1185) * refactor matrix filter and area state to be private static (#1179) * Change variable name Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add cohort data to dashboard e2e Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohorts filters Signed-off-by: Gaurav Gupta <[email protected]> * Document various data validation for cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add new interfaces for pre-built cohort Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add prebuilt cohort walking logic in UI and add more data validation scenarios Signed-off-by: Gaurav Gupta <[email protected]> * Add basic data validation checks Signed-off-by: Gaurav Gupta <[email protected]> * Add logic to translate the Index cohort filter Signed-off-by: Gaurav Gupta <[email protected]> * Remove commented out code Signed-off-by: Gaurav Gupta <[email protected]> * Add SDK validations for Index based cohort filter Signed-off-by: Gaurav Gupta <[email protected]> * Add code for validating classification outcome Signed-off-by: Gaurav Gupta <[email protected]> * Add error filter validations and add tests Signed-off-by: Gaurav Gupta <[email protected]> * Add fake cohorts for regression dataset Signed-off-by: Gaurav Gupta <[email protected]> * Add fake cohorts for multi-class classification dataset Signed-off-by: Gaurav Gupta <[email protected]> * Add handling of regression filter Signed-off-by: Gaurav Gupta <[email protected]> * Add support for classification outcome in UI Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for Predicted Y and True Y cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add UI code to handle prediced Y and true Y for pre-built cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add cohort validation with test data to raiwidgets Signed-off-by: Gaurav Gupta <[email protected]> * Add tests for validating Predicted/True Y cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add UI support for TrueY/PredictedY for classification Signed-off-by: Gaurav Gupta <[email protected]> * Rename cohort_filter_list to cohort_list Signed-off-by: Gaurav Gupta <[email protected]> * Rename UI varibles to match SDK Signed-off-by: Gaurav Gupta <[email protected]> * Fix duplicate cohort name Signed-off-by: Gaurav Gupta <[email protected]> * Add SDK cohorts to notebook Signed-off-by: Gaurav Gupta <[email protected]> * Add dataset validations and add categorical features Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for categorical_features Signed-off-by: Gaurav Gupta <[email protected]> * Fix sorted imports Signed-off-by: Gaurav Gupta <[email protected]> * Add code for translating categorical values Signed-off-by: Gaurav Gupta <[email protected]> * Move cohort processing to a separate file Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Refactor cohort translated function into different small functions Signed-off-by: Gaurav Gupta <[email protected]> * Change to lowercase for outcome Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Refactor cohort_list validations and converge pytest common functions into fixtures Signed-off-by: Gaurav Gupta <[email protected]> * Add conftest into raiwidgets tests Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for cohort list Signed-off-by: Gaurav Gupta <[email protected]> * Add cohortData test Signed-off-by: Gaurav Gupta <[email protected]> * Fix sorted imports Signed-off-by: Gaurav Gupta <[email protected]> * isort fix Signed-off-by: Gaurav Gupta <[email protected]> * Add UI unit tests for cohort translation Signed-off-by: Gaurav Gupta <[email protected]> * Add more checks in UI uni test Signed-off-by: Gaurav Gupta <[email protected]> * Add UI tests for regression cohorts Signed-off-by: Gaurav Gupta <[email protected]> * REmove notebook change Signed-off-by: Gaurav Gupta <[email protected]> * Fix typescript build Signed-off-by: Gaurav Gupta <[email protected]> * Change cohort filter values so that cohort filters non-zero points Signed-off-by: Gaurav Gupta <[email protected]> * Fix for empty cohort list Signed-off-by: Gaurav Gupta <[email protected]> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Propagate error strings instead of raising exceptions Signed-off-by: Gaurav Gupta <[email protected]> * Fix code issues Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <[email protected]> * Add missing file Signed-off-by: Gaurav Gupta <[email protected]> * fix notebook build failures due to pywinpty dependency release failing in python 3.6 (#1257) * fix notebook build failures due to pywinpty dependency release failing in python 3.6 * build pywinpty from conda instead * add lowerbound * fixup * fixup * Add supported models and data types to README.md responsibleai (#1259) Signed-off-by: Gaurav Gupta <[email protected]> * make getting-started notebook a markdown file showing APIs (#1223) * refactor tabs out of RAI dashboard into a separate component (#1256) * Add individual causal scatter chart (#1258) * temp * refactor * test * style fix * comment * minor fix to url for responsibleai package in setup.py (#1260) * Fix UX e2e tests and address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix eslint Signed-off-by: Gaurav Gupta <[email protected]> * Address review comments Signed-off-by: Gaurav Gupta <[email protected]> * Reset the number of samples in test dataset Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Bo Zhang <[email protected]> * Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb (#1252) * [WIP] Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Fix the e2e test for notebook Signed-off-by: Gaurav Gupta <[email protected]> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <[email protected]> * Add missing file Signed-off-by: Gaurav Gupta <[email protected]> * Fix cohort namespace Signed-off-by: Gaurav Gupta <[email protected]> * minor fix to url for responsibleai package in setup.py (#1260) * Counterfactual Chart: Correct target description according to task_type (#1261) * Counterfactual Chart: Correct target description according to task_type Signed-off-by: Gaurav Gupta <[email protected]> * Change function name Signed-off-by: Gaurav Gupta <[email protected]> * Make Range lowercase Signed-off-by: Gaurav Gupta <[email protected]> * fix whitespace in values of adult census income dataset (#1263) * Add what-If scatter chart from highchart lib (#1262) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * add a builddebug yarn command to build UX locally which can be debugged in browser e2e (#1265) * allow rai text insights to work with RAI dashboard (#1269) * remove duplicate code in explanation dashboard (#1266) * Individual causal style responsive (#1268) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * Causal Style * Allow duplicating cohorts multiple times (#1274) * allow duplicating a cohort more than once * lintfix * Disable column header highlighting on hover in IndividualFeatureImportanceView (#1272) * disable column highlight on hover * lintfix * Rename new cohorts from "Unsaved" to "Temporary cohort" (#1273) * rename Unsaved to Temporary cohort * localize temp cohort * Counterfactual style refactor (#1275) * style refactor * test * test * test * fix * Don't change cursor on hover over cohort name * Fix (#1281) * fix cohort info styling (#1277) * fix readme link to fairness and interpretability example notebook (#1282) * add new RAI Utils package for common utilities shared across RAI packages (#1280) * Add ICE chart (#1283) * Fix * ice chart * ic * test * test * update docstring for explanation dashboard in regards to min number of rows (#1271) * make builds more reliable by adding retry logic to urlretrieve calls in notebooks (#1218) * upgrade pytest to 7.0.1, remove mock and updgrade pytest-mock to 3.6.1 (#1287) * remove deprecated codecov parameter (#1293) * Fix min/max special case in cohort filter creation with "in the range of" (#1279) * fix logic in the case that min or max are zero * lintfix * Rename 'Dashboard navigation' to 'Dashboard configuration' (#1291) * Rename 'Dashboard navigation' to 'Dashboard configuration' Signed-off-by: Gaurav Gupta <[email protected]> * Notebook change Signed-off-by: Gaurav Gupta <[email protected]> * Add raiutils to PR template (#1290) * fix heatmap bug (#1297) * Make "save and switch" work from cohort settings (#1276) * make save and switch work * fix naming * lintfix * adjustment according to Ilya's comment * lintfix * add retry logic to codecov step and only upload results for one python version (#1298) * add github action to release raiutils to pypi (#1294) * Add highchart for Dataset Explorer (#1286) * test * style * click * fix test * fix test * test * test * test * test * Update requirements-linting.txt to add flake8-pytest-style (#1296) * Fix sort abs (#1299) * Rename "base cohort" to "global cohort" (#1278) * change base cohort to global cohort * fix spelling * lintfix * fix codecov comment not appearing on PRs (#1302) * take absolute value of error calculation for regression scenario (#1301) * Limit individual feature importance selection to up to 5 (#1305) * update feature importance string * limit selection to up to 5 * add group count * remove message bar, show info icon instead * update e2e locator * fix E2E failure on feature importance * add ariaLabel for expand collapse button * add renderOnNewLayer props * Add error message for counterfactual panel (#1310) * add error message for counterfactual * update error message in camel case to fix build error * Add to_json() and from_json() methods to Cohort class (#1300) * Add to_json() and from_json() methods to Cohort class Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix linting Signed-off-by: Gaurav Gupta <[email protected]> * Add a highchart heatmap helper class (#1307) * add highchart heatmap helper class * add erroneously deleted line back * Fix cohort setting string (#1304) * Fix string * remove none * name * test * Fix all data style (#1303) * Add a feature flag for the new model overview experience (#1306) * add feature flag for new model overview experience and turn it off by default * remove useless constructor * Clean up charts code (#1313) * clean up chart code * remove arg * Expand the counterfactual flyout to cover the full page (#1315) Signed-off-by: Gaurav Gupta <[email protected]> * Bump minimist from 1.2.5 to 1.2.6 (#1292) * Bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> * minimist ^1.2.6 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * fix random node download failures by upgrading to latest github action with retry logic (#1317) * Add dataset cohort table to new ModelOverview experience (#1314) * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * Add installation instructions for raiwidgets to README (#1320) * refactor RAIInsights into RAIInsightsBase class for basic functionality (#1284) * Fix what if counterfactual header and description text misaligned (#1316) * align * e2e * add clear temporary cohort button to error analysis (#1322) * Raise UserConfigValidationException in case no model but valid model serializer (#1325) Signed-off-by: Gaurav Gupta <[email protected]> * Add test case for handling different types in causal (#1321) Signed-off-by: Gaurav Gupta <[email protected]> * show shift to an empty cohort in tree view as an empty node (#1318) * Bug fixing (#1326) * Move chart description up and remove scroll bar * Change string * Add box outlier for dataset explorer (#1323) * add outlier for dataset explorer * name * update string when no datapoint selected (#1331) * Fix Big empty space for featureImportance chart (#1328) * legend * removed invalid test case * constant * Disable save as new cohort button if nothing is selected in error tree (#1327) * Add disaggregated analysis table/heatmap (#1332) * disaggregated analysis changes only * lintfix * Change warning message to user exception for model type and task type mismatch (#1330) * Change warning message to user exception for model type and task type mismatch Signed-off-by: Gaurav Gupta <[email protected]> * Fix flake8 errors Signed-off-by: Gaurav Gupta <[email protected]> * Change the counterfactual text color from black to grey (#1337) Signed-off-by: Gaurav Gupta <[email protected]> * Limit each component description width up to 750px for readability (#1336) * limit description width up to 750px * export maxWidth from a common place * block empty cohort creation in RAI Dashboard (#1335) * Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort * Rename counterfactual style files to confirm with *.styles.ts (#1338) Signed-off-by: Gaurav Gupta <[email protected]> * Add disaggregated analysis table to Model Overview (#1341) * pull in changes for disaggregated analysis * add styles file * add textured NaN cells * module import for textured cells and grid y axis * lintfix * use combobox for dropdown rather than dropdown * lintfix * Rename causal style files to confirm with *.styles.ts (#1342) Signed-off-by: Gaurav Gupta <[email protected]> * update responsibleai to interpret-community 0.25.0 (#1343) * All component title and descriptions should be aligned (#1346) * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * Remove 5K limit blurb from local explanations tab (#1347) Signed-off-by: Gaurav Gupta <[email protected]> * Sort features by default in counterfactual flyout (#1312) * Sort features by default in counterfactual flyout Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> * attempt to fix test Signed-off-by: Gaurav Gupta <[email protected]> * Remove check Signed-off-by: Gaurav Gupta <[email protected]> * Bump moment from 2.28.0 to 2.29.2 (#1333) Bumps [moment](https://github.com/moment/moment) from 2.28.0 to 2.29.2. - [Release notes](https://github.com/moment/moment/releases) - [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md) - [Commits](https://github.com/moment/moment/compare/2.28.0...2.29.2) --- updated-dependencies: - dependency-name: moment dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <[email protected]> * Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text (#1350) * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1344) * update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected (#1349) Co-authored-by: Vinutha Karanth <[email protected]> * update erroranalysis to 0.2.1 and remove some duplicate dependencies (#1334) * disable turbo checking for large amount of data (#1351) Signed-off-by: Ke Xu <[email protected]> * force re-render when chart type changes (#1354) Signed-off-by: Ke Xu <[email protected]> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1352) * Rename output column name in the counterfactual flyout (#1353) Signed-off-by: Gaurav Gupta <[email protected]> * Show column chart for categorical feature in data explorer (#1355) * Show column chart for categorical feature in data explorer * address comments * update fluentui (#1356) Signed-off-by: Ke Xu <[email protected]> * update code owner (#1308) * update code owner Signed-off-by: Ke Xu <[email protected]> * remove dup Signed-off-by: Ke Xu <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * update version to match studio (#1357) Signed-off-by: Ke Xu <[email protected]> * alignment fixes (#1359) * Add charts for new model overview experience (#1348) * rename OverallTable to DisaggregatedMetricsTable and move to core-ui * Copy the ModelPerformanceTab into model-assessment and rename to ModelOverview * reference OverallTable again in fairness widget * refactor core chart component out into core-ui * refactor out core chart component into core-ui * lintfix * undo tsconfig.lib.json change * fix chartAndConfigsId in tests * lintfix * add table for cohort metrics and add dropdown metric selector, add new metrics * add new metrics * undo unwanted changes * fix casing * add superscript 2 for r-squared * update tests to reflect new metrics * lintfix * add feature flag * fix mae * fix mae calculation * first version of new model overview table * get probability distribution box plot to work * add feature flag for new model overview experience and turn it off by default * add highchart heatmap helper class * remove custom styling * add erroneously deleted line back * remove useless constructor * modularize model overview * show outliers in box plot, fix positioning * remove showmetricsummary * refactor heatmap code into a common class * add featureDropdownRef to allow focusing * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * add chart config flyout (in progress) * add chart config flyout (in progress) * address feedback, use finalized color * adjust feature selection to disable options if limit is reached, add axis config buttons * select all via dropdown * lintfix * refactor box plot calculations and rendering * add style file * textured NaN cells, grid labels on y axis * standardize box plots to use fences * fix merge issues * unify box plot tooltip formatting code, fix bar chart sizing issue * small fixes * rearrange feature dropdown * lintfix * remove commented out code * remove box plot tooltip customization * lintfix * add a few unit tests * unit tests for smaller utilities, localization fixes, consistent flyout flow with confirm/cancel buttons * lintfix * fix chart config flyout update * fix test case * rename files for lint * file rename for lint * release rai-core-flask 0.3.0 (#1361) * upgrade python version used with flask CI to fix segfault error (#1363) * release raiwidgets and responsibleai v0.18.0 (#1360) * fix two bugs (#1364) * Add pre-built cohort into adult census notebook (#1243) * [WIP] Add pre-built cohort into adult census notebook Signed-off-by: Gaurav Gupta <[email protected]> * erroranalysis version bump in raiwidgets to 0.1.31 (#1245) * Make cohrtData empty list in case no pre-bdefined cohorts are injected (#1247) Signed-off-by: Gaurav Gupta <[email protected]> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add regression test for pre-defined cohorts in raiwidgets (#1249) Signed-off-by: Gaurav Gupta <[email protected]> * color (#1248) * Add feature importance box & bar chart (#1241) * refactor * build * build * temp * temp * temp * temp * box * cache * e2e * e2e * fix * e2e fix * e2e * fix e2e * widget * widget * fix * widget * e2e * e2e * e2e * test * test * PreBuilt cohorts UX changes (#1242) * Intial SDK implementation cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add basic validationf for cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add serialized version of cohort config to ResponsibleAiDashboard Signed-off-by: Gaurav Gupta <[email protected]> * Add more tests cohorts Signed-off-by: Gaurav Gupta <[email protected]> * fix broken builds due to pip upgrade which broke pip-tools (#1185) * refactor matrix filter and area state to be private static (#1179) * Change variable name Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add cohort data to dashboard e2e Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohorts filters Signed-off-by: Gaurav Gupta <[email protected]> * Document various data validation for cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add new interfaces for pre-built cohort Signed-off-by: Gaurav Gupta <[email protected]> * Add more cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add prebuilt cohort walking logic in UI and add more data validation scenarios Signed-off-by: Gaurav Gupta <[email protected]> * Add basic data validation checks Signed-off-by: Gaurav Gupta <[email protected]> * Add logic to translate the Index cohort filter Signed-off-by: Gaurav Gupta <[email protected]> * Remove commented out code Signed-off-by: Gaurav Gupta <[email protected]> * Add SDK validations for Index based cohort filter Signed-off-by: Gaurav Gupta <[email protected]> * Add code for validating classification outcome Signed-off-by: Gaurav Gupta <[email protected]> * Add error filter validations and add tests Signed-off-by: Gaurav Gupta <[email protected]> * Add fake cohorts for regression dataset Signed-off-by: Gaurav Gupta <[email protected]> * Add fake cohorts for multi-class classification dataset Signed-off-by: Gaurav Gupta <[email protected]> * Add handling of regression filter Signed-off-by: Gaurav Gupta <[email protected]> * Add support for classification outcome in UI Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for Predicted Y and True Y cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add UI code to handle prediced Y and true Y for pre-built cohort filters Signed-off-by: Gaurav Gupta <[email protected]> * Add cohort validation with test data to raiwidgets Signed-off-by: Gaurav Gupta <[email protected]> * Add tests for validating Predicted/True Y cohorts Signed-off-by: Gaurav Gupta <[email protected]> * Add UI support for TrueY/PredictedY for classification Signed-off-by: Gaurav Gupta <[email protected]> * Rename cohort_filter_list to cohort_list Signed-off-by: Gaurav Gupta <[email protected]> * Rename UI varibles to match SDK Signed-off-by: Gaurav Gupta <[email protected]> * Fix duplicate cohort name Signed-off-by: Gaurav Gupta <[email protected]> * Add SDK cohorts to notebook Signed-off-by: Gaurav Gupta <[email protected]> * Add dataset validations and add categorical features Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for categorical_features Signed-off-by: Gaurav Gupta <[email protected]> * Fix sorted imports Signed-off-by: Gaurav Gupta <[email protected]> * Add code for translating categorical values Signed-off-by: Gaurav Gupta <[email protected]> * Move cohort processing to a separate file Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Refactor cohort translated function into different small functions Signed-off-by: Gaurav Gupta <[email protected]> * Change to lowercase for outcome Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Refactor cohort_list validations and converge pytest common functions into fixtures Signed-off-by: Gaurav Gupta <[email protected]> * Add conftest into raiwidgets tests Signed-off-by: Gaurav Gupta <[email protected]> * Add validations for cohort list Signed-off-by: Gaurav Gupta <[email protected]> * Add cohortData test Signed-off-by: Gaurav Gupta <[email protected]> * Fix sorted imports Signed-off-by: Gaurav Gupta <[email protected]> * isort fix Signed-off-by: Gaurav Gupta <[email protected]> * Add UI unit tests for cohort translation Signed-off-by: Gaurav Gupta <[email protected]> * Add more checks in UI uni test Signed-off-by: Gaurav Gupta <[email protected]> * Add UI tests for regression cohorts Signed-off-by: Gaurav Gupta <[email protected]> * REmove notebook change Signed-off-by: Gaurav Gupta <[email protected]> * Fix typescript build Signed-off-by: Gaurav Gupta <[email protected]> * Change cohort filter values so that cohort filters non-zero points Signed-off-by: Gaurav Gupta <[email protected]> * Fix for empty cohort list Signed-off-by: Gaurav Gupta <[email protected]> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Propagate error strings instead of raising exceptions Signed-off-by: Gaurav Gupta <[email protected]> * Fix code issues Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <[email protected]> * Add missing file Signed-off-by: Gaurav Gupta <[email protected]> * fix notebook build failures due to pywinpty dependency release failing in python 3.6 (#1257) * fix notebook build failures due to pywinpty dependency release failing in python 3.6 * build pywinpty from conda instead * add lowerbound * fixup * fixup * Add supported models and data types to README.md responsibleai (#1259) Signed-off-by: Gaurav Gupta <[email protected]> * make getting-started notebook a markdown file showing APIs (#1223) * refactor tabs out of RAI dashboard into a separate component (#1256) * Add individual causal scatter chart (#1258) * temp * refactor * test * style fix * comment * minor fix to url for responsibleai package in setup.py (#1260) * Fix UX e2e tests and address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Fix eslint Signed-off-by: Gaurav Gupta <[email protected]> * Address review comments Signed-off-by: Gaurav Gupta <[email protected]> * Reset the number of samples in test dataset Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Bo Zhang <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Change cohort name Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Bo Zhang <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: tongy-msft <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <[email protected]> Co-authored-by: Vinutha Karanth <[email protected]> * Add more context to download print in fetch_dataset (#1368) * Add missing desired range to counterfactual chart (#1366) Signed-off-by: Gaurav Gupta <[email protected]> * Add a summary for disaggregated analysis showing ratio and difference (#1367) * add fairness metric table, update existing tables (order, N/A values, grid axis label lengths) * lintfix * merge fixes * addressed eslint issue for null value by creating a dedicated file, localized N/A, updated tooltip text to be wrapped (+ associated unit tests) * keep feature flag * Update PR template to be more intuitive (#1116) * Updating PR template to make sure there's at least 1 box that can be ticked. * replace checkboxes with GitHub emojis for checkmark and x to prevent GitHub from interpreting it as a task list * update PR template based on feedback * lintfix * Update CODEOWNERS file (#1369) * Use global cohort for disaggregated analysis (#1370) * use global cohort for disaggregated analysis and warn user about it * lintfix * Change procedural loops to functions map() and filter() (#1255) * Change procedural loops to function map() and filter() Signed-off-by: Gaurav Gupta <[email protected]> * Fix review comment Signed-off-by: Gaurav Gupta <[email protected]> * Add accuracy as multiclass metric (#1371) * support accuracy for multiclass classification * lintfix * update code owner (#1372) Signed-off-by: Ke Xu <[email protected]> Co-authored-by: Roman Lutz <[email protected]> * Rename _validate_model_analysis_input_parameters() to _validate_rai_insights_input_parameters (#1373) Signed-off-by: Gaurav Gupta <[email protected]> * Cache and serialize the counterfactual explainer (#1374) * Cache and serialize the counterfactual explainer Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> * Add a top-level pivot to model overview (#1375) * show only disaggregated analysis OR only dataset cohorts * lintfix * basec -> basic typo * Add a toggle to model overview to turn heatmap colors on and off (#1376) * add visual display toggle * lintfix * simplify onVisualDisplayToggleChange method * color replacements * fluentui colors * localization, style file * lintfix * Create rai-e2e package for tests shared with AML (#1378) * update Signed-off-by: vinutha karanth <[email protected]> * update Signed-off-by: vinutha karanth <[email protected]> * index update Signed-off-by: vinutha karanth <[email protected]> * remove files from wid-e2e Signed-off-by: vinutha karanth <[email protected]> * update datashape Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * add pyspark support to tree surrogate model in error analysis (#1251) * Create e2e package for tests shared with AML (#1383) * new buildable folder Signed-off-by: vinutha karanth <[email protected]> * remove rai-e2e lib Signed-off-by: vinutha karanth <[email protected]> * add license Signed-off-by: vinutha karanth <[email protected]> * update Signed-off-by: vinutha karanth <[email protected]> * Add feature and metric configuration flyouts to model overview (#1381) * show only disaggregated analysis OR only dataset cohorts * lintfix * basec -> basic typo * feature config * merge fixes * metric config * lintfix * wrap metric descriptions * lintfix * move constant to constants file to avoid circular dependency, lintfix * move inline code into functions * Add probability distribution line chart to model overview (#1382) * line chart for prob distribution * lintfix * fix race condition in tree view update code which can cause root stats to be ignored on selected cohort in RAI dashboard static view (#1386) * String updates for model overview (#1387) * string updates for model overview * lintfix * Handle the loading of causal models more gracefully (#1377) * Handle the loading of causal models more gracefully Signed-off-by: Gaurav Gupta <[email protected]> * Address code review commensts and fix test Signed-off-by: Gaurav Gupta <[email protected]> * Fix code review comment Signed-off-by: Gaurav Gupta <[email protected]> * Some fixes for data explorer, heatmap, counterfactual components (#1389) * add x axis and update heatmap * cf custom point color should be consistent * cf custom point name should be displayed * disable causal treatment whatif in static view (#1391) * fix big data set for counter factual chart (#1393) Signed-off-by: Ke Xu <[email protected]> * alignment fixes for causal (#1392) * Fix feature selection state carry-over bug in model overview (#1390) * Provide feature flight support to dashboard, test environment, and Python widget (#1385) * Python part of exposing flights * add feature flighting * fix path for test case * update snapshot * add none constant, contrib guide update, and unit tests for feature_flights param * flake8 * isort * release raiwidgets and responsibleai v0.18.1 (#1413) * Add warning in counterfactual manager when unable to load explainer (#1412) * Add warning in counterfactual manager when unable to load explainer Signed-off-by: Gaurav Gupta <[email protected]> * Fix linting Signed-off-by: Gaurav Gupta <[email protected]> * Bug fixes on 'Set value' not copying over feature values correctly in what if counterfactual panel (#1416) * fix cf set value bug * remove comment * fix error on machines with pyspark installed where passed dataframe is not spark pandas (#1415) * add postbuild branch trigger (#1417) * Fix causal UI strings according to classification/regression tasks (#1419) * Fix causal UI strings according to classification/regression tasks Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint error Signed-off-by: Gaurav Gupta <[email protected]> * Fix UI test Signed-off-by: Gaurav Gupta <[email protected]> * Fix description for model overview (#1425) * fix description for model overview * keep new description for new model overview * fix failing to create error report when filter_features is empty list (#1421) * filter out missing values from what if dropdown to prevent explanation dashboard from crashing (#1418) * Remove |Set Value| blurb in case it is not availble in counterfactual panel (#1426) Signed-off-by: Gaurav Gupta <[email protected]> * Add y-axis description to counterfactual feature importance chart (#1423) Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: xuke444 <[email protected]> * Add the user class name to causal UI strings (#1422) * Fix causal UI strings according to classification/regression tasks Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint error Signed-off-by: Gaurav Gupta <[email protected]> * Fix UI test Signed-off-by: Gaurav Gupta <[email protected]> * Add the user class name to causal UI strings Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: xuke444 <[email protected]> * fix math.min / max for array size more than 10^7 (#1427) Signed-off-by: Ke Xu <[email protected]> * upgrade pytest and lightgbm to try to fix random pytest segfault test failures (#1424) s * fix flaky notebook causing build failures by adding retry logic (#1431) * Upper bound SciKit-Learn to address freeze in causal (#1432) <!--- Provide a general summary of your changes in the Title above --> ## Description Replaces #1429 to address #1430 . Causal analysis is getting stuck with the latest release of SciKit-Learn. This contains: - Test case which gets stuck with SciKit-Learn 1.1.0 - Upper bound on SciKit-Learn in `requirements.txt` ## Checklist <!--- Make sure to satisfy all the criteria listed below. --> - [x] I have added screenshots above for all UI changes. - [x] Documentation was updated if it was needed. - [x] New tests were added or changes were manually verified. Signed-off-by: Richard Edgar <[email protected]> * fix dependency chart axis updating with incorrect values in explanation dashboard (#1437) * fix codecov and widget test screenshot uploads (#1428) * release raiwidgets and responsibleai v0.18.2 (#1439) * fix (#1441) Signed-off-by: vinutha karanth <[email protected]> * Fix cohort name conflict and not run few tests for AML (#1442) * fix Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * Few e2e tests changes to accommodate AML static tests (#1445) * update Signed-off-by: vinutha karanth <[email protected]> * update Signed-off-by: vinutha karanth <[email protected]> * Fix locators logic for string features - data explorer and model statistics components (#1446) * update Signed-off-by: vinutha karanth <[email protected]> * update Signed-off-by: vinutha karanth <[email protected]> * fix Signed-off-by: vinutha karanth <[email protected]> * update Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * fix Signed-off-by: vinutha karanth <[email protected]> * Add more unittests RAI dashboard input class (#1448) * Add unit tests for ResponsibleAIDashboardInput Signed-off-by: Gaurav Gupta <[email protected]> * Add more tests Signed-off-by: Gaurav Gupta <[email protected]> * Fix imports Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments Signed-off-by: Gaurav Gupta <[email protected]> * Address code review comments Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Bo Zhang <[email protected]> Co-authored-by: Roman Lutz <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: tongy-msft <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <[email protected]> Co-authored-by: Vinutha Karanth <[email protected]> Co-authored-by: Richard Edgar <[email protected]>
- Loading branch information
1 parent
8f5bf16
commit 1edd227