diff --git a/CHANGES.md b/CHANGES.md index 2ee3946a..7c5177da 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,5 +1,13 @@ ## Changelog +### 1.6.0 (2019-12-19) + * charts integration + * the ability to look at data in line, bar, stacked bar & pie charts + * the ability to group & aggregate data within the charts + * direct ipython iframes to correlations & charts pages with pre-selected inputs + * the ability to access instances from code by data id `dtale.get_instance(data_id)` + * view all active data instances `dtale.instances()` + ### 1.5.1 (2019-12-12) * conversion of new flask instance for each `dtale.show` call to serving all data associated with one parent process under the same flask instance unless otherwise specified by the user (the `force` parameter) diff --git a/README.md b/README.md index 6975e27e..2b9a2078 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ [![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/Title.png)](https://github.com/man-group/dtale) -[Live Demo](http://andrewschonfeld.pythonanywhere.com/dtale/main) +[Live Demo](http://andrewschonfeld.pythonanywhere.com) ----------------- @@ -24,7 +24,7 @@ D-Tale was born out a conversion from SAS to Python. What was originally a perl - [Dimensions/Main Menu](#dimensionsmain-menu) - [Selecting/Deselecting Columns](#selectingdeselecting-columns) - [Menu functions w/ no columns selected](#menu-functions-w-no-columns-selected) - - [Describe](#describe), [Coverage](#coverage), [Correlations](#correlations), [Heat Map](#heat-map), [Instances](#instances), [About](#about), [Resize](#resize), [Iframe-Mode/Full-Mode](#iframe-modefull-mode), [Shutdown](#shutdown) + - [Describe](#describe), [Charts](#charts), [Correlations](#correlations), [Heat Map](#heat-map), [Instances](#instances), [About](#about), [Resize](#resize), [Iframe-Mode/Full-Mode](#iframe-modefull-mode), [Shutdown](#shutdown) - [Menu functions w/ column(s) selected](#menu-functions-w-columns-selected) - [Move To Front](#move-to-front), [Lock](#lock), [Unlock](#unlock), [Sorting](#sorting), [Formats](#formats), [Histogram](#histogram) - [Menu functions within a Jupyter Notebook](#menu-functions-within-a-jupyter-notebook) @@ -34,6 +34,7 @@ D-Tale was born out a conversion from SAS to Python. What was originally a perl - [Linting](#linting) - [Formatting JS](#formatting-js) - [Docker Development](#docker-development) +- [Startup Behavior](#startup-behavior) - [Documentation](#documentation) - [Requirements](#requirements) - [Acknowledgements](#acknowledgements) @@ -97,9 +98,13 @@ d.kill() d.open_browser() # There is also some helpful metadata about the process -d._port # the process's port +d._data_id # the process's data identifier d._url # the url to access the process +d2 = dtale.get_instance(d._data_id) # returns a new reference to the instance running at that data_id + +dtale.instances() # returns a dictionary of all instances available, this would be { 1: ... } + ``` ### Jupyter Notebook @@ -115,9 +120,9 @@ If you are running ipython<=5.0 then you also have the ability to adjust the siz One thing of note is that alot of the modal popups you see in the standard browser version will now open separate browser windows for spacial convienence: -|Column Menus|Correlations|Describe|Histogram|Coverage|Instances| +|Column Menus|Correlations|Describe|Histogram|Charts|Instances| |:------:|:------:|:------:|:------:|:------:|:------:| -|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/Column_menu.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/correlations_popup.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/describe_popup.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/histogram_popup.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/coverage_popup.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/instances_popup.png)| +|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/Column_menu.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/correlations_popup.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/describe_popup.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/histogram_popup.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/charts_popup.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/instances_popup.png)| ### Command-line Base CLI options (run `dtale --help` to see all options available) @@ -129,6 +134,7 @@ Base CLI options (run `dtale --help` to see all options available) |`--debug`|turn on Flask's "debug" mode for your D-Tale instance| |`--no-reaper`|flag to turn off auto-reaping subprocess (kill D-Tale instances after an hour of inactivity), good for long-running displays | |`--open-browser`|flag to automatically open up your server's default browser to your D-Tale instance| +|`--force`|flag to force D-Tale to try an kill any pre-existing process at the port you've specified so it can use it| Loading data from **arctic** ```bash @@ -242,17 +248,28 @@ Apply a simple pandas `query` to your data (link to pandas documentation include |--------|:------:| |![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/Filter_apply.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/Post_filter.png)| -#### Coverage -Check for coverage gaps on column(s) by way of other column(s) as group(s) - - Select column(s) in "Group(s)" & "Col(s)" - - date-type columns you can also specify a frequency of D, W, M, Q, Y - - Select multiple values in "Cols(s)" and/or "Groups(s)" by holdings the SHIFT key as you click - - Click "Load" - - The output will be the counts of non-nan records in "Col(s)" grouped by your selections in "Group(s)" +#### Charts +Build custom charts based off your data. -|Daily|Daily Regional| -|-----|:-------------:| -|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/Coverage_daily.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/Coverage_daily_regions.png)| + - To build a chart you must pick a value for X & Y inputs which effectively drive what data is along the X & Y axes + - If your data along the x-axis has duplicates you have three options: + - specify a group, which will create series for each group + - specify an aggregation, you can choose from one of the following: Count, First, Last, Mean, Median, Minimum, MAximum, Standard Deviation, Variance, Mean Absolute Deviation, Product of All Items, Sum + - specify both a group & an aggregation + - Click the "Load" button which will load the data and display the default cahrt type "line" + - You now have the ability to toggle between different chart types: line, bar, stacked bar & pie + - If you have specified a group then you have the ability between showing all series in one chart and breaking each series out into its own chart "Chart per Group" + +Here are some examples with the following inputs: X: date, Y: Col0, Group: security_id, Aggregation: Mean, Query: `security_id in (100000, 100001) and date >= '20181220' and date <= '20181231'` + +|Chart Type|Chart|Chart per Group| +|:------:|:------:|:------:| +|line|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/charts_line.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/charts_line_pg.png)| +|bar|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/charts_bar.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/charts_bar_pg.png)| +|stacked|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/charts_stacked.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/charts_stacked_pg.png)| +|pie|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/charts_pie.png)|![](https://raw.githubusercontent.com/man-group/dtale/master/docs/images/charts_pie_pg.png)| + +This is a very powerful feature with many more features that could be offered (word cloud, different statistical aggregations, etc...) so please submit issues :) #### Correlations Shows a pearson correlation matrix of all numeric columns against all other numeric columns @@ -485,6 +502,80 @@ $ python ``` Then view your D-Tale instance in your browser using the link that gets printed +## Startup Behavior + +Here's a little background on how the `dtale.show()` function works: + - by default it will look for ports between 40000 & 49000, but you can change that range by specifying the environment variables DTALE_MIN_PORT & DTALE_MAX_PORT + - think of sessions as python consoles or jupyter notebooks + +1) Session 1 executes `dtale.show(df)` our state is: + +|Session|Port|Active Data IDs|URL(s)| +|:-----:|:-----:|:-----:|:-----:| +|1|40000|1|http://localhost:40000/dtale/main/1| + +2) Session 1 executes `dtale.show(df)` our state is: + +|Session|Port|Active Data IDs|URL(s)| +|:-----:|:-----:|:-----:|:-----:| +|1|40000|1,2|http://localhost:40000/dtale/main/[1,2]| + +2) Session 2 executes `dtale.show(df)` our state is: + +|Session|Port|Active Data IDs|URL(s)| +|:-----:|:-----:|:-----:|:-----:| +|1|40000|1,2|http://localhost:40000/dtale/main/[1,2]| +|2|40001|1|http://localhost:40001/dtale/main/1| + +3) Session 1 executes `dtale.show(df, port=40001, force=True)` our state is: + +|Session|Port|Active Data IDs|URL(s)| +|:-----:|:-----:|:-----:|:-----:| +|1|40001|1,2,3|http://localhost:40001/dtale/main/[1,2,3]| + +4) Session 3 executes `dtale.show(df)` our state is: + +|Session|Port|Active Data IDs|URL(s)| +|:-----:|:-----:|:-----:|:-----:| +|1|40001|1,2,3|http://localhost:40001/dtale/main/[1,2,3]| +|3|40000|1|http://localhost:40000/dtale/main/1| + +5) Session 2 executes `dtale.show(df)` our state is: + +|Session|Port|Active Data IDs|URL(s)| +|:-----:|:-----:|:-----:|:-----:| +|1|40001|1,2,3|http://localhost:40001/dtale/main/[1,2,3]| +|3|40000|1|http://localhost:40000/dtale/main/1| +|2|40002|1|http://localhost:40002/dtale/main/1| + +6) Session 4 executes `dtale.show(df, port=8080)` our state is: + +|Session|Port|Active Data IDs|URL(s)| +|:-----:|:-----:|:-----:|:-----:| +|1|40001|1,2,3|http://localhost:40001/dtale/main/[1,2,3]| +|3|40000|1|http://localhost:40000/dtale/main/1| +|2|40002|1|http://localhost:40002/dtale/main/1| +|4|8080|1|http://localhost:8080/dtale/main/1| + +7) Session 1 executes `dtale.get_instance(1).kill()` our state is: + +|Session|Port|Active Data IDs|URL(s)| +|:-----:|:-----:|:-----:|:-----:| +|1|40001|2,3|http://localhost:40001/dtale/main/[2,3]| +|3|40000|1|http://localhost:40000/dtale/main/1| +|2|40002|1|http://localhost:40002/dtale/main/1| +|4|8080|1|http://localhost:8080/dtale/main/1| + +7) Session 5 sets DTALE_MIN_RANGE to 30000 and DTALE_MAX_RANGE 39000 and executes `dtale.show(df)` our state is: + +|Session|Port|Active Data ID(s)|URL(s)| +|:-----:|:-----:|:-----:|:-----:| +|1|40001|2,3|http://localhost:40001/dtale/main/[2,3]| +|3|40000|1|http://localhost:40000/dtale/main/1| +|2|40002|1|http://localhost:40002/dtale/main/1| +|4|8080|1|http://localhost:8080/dtale/main/1| +|5|30000|1|http://localhost:30000/dtale/main/1| + ## Documentation Have a look at the [detailed documentation](https://dtale.readthedocs.io). diff --git a/docker/2_7/Dockerfile b/docker/2_7/Dockerfile index 1933cc93..f8843fed 100644 --- a/docker/2_7/Dockerfile +++ b/docker/2_7/Dockerfile @@ -44,4 +44,4 @@ WORKDIR /app RUN set -eux \ ; . /root/.bashrc \ - ; easy_install dtale-1.5.1-py2.7.egg + ; easy_install dtale-1.6.0-py2.7.egg diff --git a/docker/3_6/Dockerfile b/docker/3_6/Dockerfile index 9d530a19..e612a5a9 100644 --- a/docker/3_6/Dockerfile +++ b/docker/3_6/Dockerfile @@ -44,4 +44,4 @@ WORKDIR /app RUN set -eux \ ; . /root/.bashrc \ - ; easy_install dtale-1.5.1-py3.7.egg + ; easy_install dtale-1.6.0-py3.7.egg diff --git a/docs/images/Coverage_daily.png b/docs/images/Coverage_daily.png deleted file mode 100644 index 8f65300e..00000000 Binary files a/docs/images/Coverage_daily.png and /dev/null differ diff --git a/docs/images/Coverage_daily_regions.png b/docs/images/Coverage_daily_regions.png deleted file mode 100644 index 145b5b6a..00000000 Binary files a/docs/images/Coverage_daily_regions.png and /dev/null differ diff --git a/docs/images/charts_bar.png b/docs/images/charts_bar.png new file mode 100644 index 00000000..f3a6fb22 Binary files /dev/null and b/docs/images/charts_bar.png differ diff --git a/docs/images/charts_bar_pg.png b/docs/images/charts_bar_pg.png new file mode 100644 index 00000000..db06daae Binary files /dev/null and b/docs/images/charts_bar_pg.png differ diff --git a/docs/images/charts_line.png b/docs/images/charts_line.png new file mode 100644 index 00000000..53436aa0 Binary files /dev/null and b/docs/images/charts_line.png differ diff --git a/docs/images/charts_line_pg.png b/docs/images/charts_line_pg.png new file mode 100644 index 00000000..86353c7b Binary files /dev/null and b/docs/images/charts_line_pg.png differ diff --git a/docs/images/charts_pie.png b/docs/images/charts_pie.png new file mode 100644 index 00000000..f906506e Binary files /dev/null and b/docs/images/charts_pie.png differ diff --git a/docs/images/charts_pie_pg.png b/docs/images/charts_pie_pg.png new file mode 100644 index 00000000..158c4804 Binary files /dev/null and b/docs/images/charts_pie_pg.png differ diff --git a/docs/images/charts_popup.png b/docs/images/charts_popup.png new file mode 100644 index 00000000..2cf77f7b Binary files /dev/null and b/docs/images/charts_popup.png differ diff --git a/docs/images/charts_stacked.png b/docs/images/charts_stacked.png new file mode 100644 index 00000000..8d18326d Binary files /dev/null and b/docs/images/charts_stacked.png differ diff --git a/docs/images/charts_stacked_pg.png b/docs/images/charts_stacked_pg.png new file mode 100644 index 00000000..8c7279b5 Binary files /dev/null and b/docs/images/charts_stacked_pg.png differ diff --git a/docs/images/coverage_popup.png b/docs/images/coverage_popup.png deleted file mode 100644 index 60d45253..00000000 Binary files a/docs/images/coverage_popup.png and /dev/null differ diff --git a/docs/source/conf.py b/docs/source/conf.py index e7f11072..01a5b573 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -63,9 +63,9 @@ # built documents. # # The short X.Y version. -version = u'1.5.1' +version = u'1.6.0' # The full version, including alpha/beta/rc tags. -release = u'1.5.1' +release = u'1.6.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. diff --git a/docs/twitter/._holidays.csv b/docs/twitter/._holidays.csv new file mode 100644 index 00000000..eb980420 Binary files /dev/null and b/docs/twitter/._holidays.csv differ diff --git a/docs/twitter/holidays.csv b/docs/twitter/holidays.csv new file mode 100755 index 00000000..b2ad9d19 --- /dev/null +++ b/docs/twitter/holidays.csv @@ -0,0 +1,9 @@ +1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103 +0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00 +0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,1.00,1.00,1.00,0.00,1.00,1.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,1.00,0.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,1.00,0.00,0.00,1.00,1.00,0.00,1.00,1.00,0.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,1.00,1.00,0.00,1.00,1.00,0.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,1.00,0.00,0.00,1.00,0.00 +0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,1.00,1.00,0.00,0.00,1.00,1.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,1.00,1.00,0.00,0.00,1.00,0.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00 +0.00,1.00,1.00,1.00,0.00,1.00,1.00,1.00,0.00,1.00,1.00,1.00,0.00,1.00,1.00,1.00,0.00,0.00,1.00,0.00,0.00,0.00,1.00,1.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,1.00,1.00,0.00,0.00,1.00,0.00,0.00,0.00,1.00,0.00,0.00,0.00,1.00,1.00,0.00,1.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,1.00,1.00,0.00,1.00,1.00,1.00,0.00,0.00,1.00,1.00,1.00,0.00,1.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,1.00,0.00,0.00,1.00,0.00 +0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,0.00,1.00,0.00,0.00,0.00,0.00,1.00,0.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,0.00,0.00,0.00 +0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,0.00,1.00,0.00,0.00,0.00,0.00,1.00,0.00,0.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,1.00,1.00,0.00,1.00,0.00,1.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,1.00,1.00,0.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,1.00,1.00,0.00,1.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,0.00,1.00,0.00,0.00,1.00,0.00,0.00,0.00,1.00,0.00 +0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00 +1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00 \ No newline at end of file diff --git a/dtale/__init__.py b/dtale/__init__.py index e2132546..b3f0a80f 100644 --- a/dtale/__init__.py +++ b/dtale/__init__.py @@ -3,4 +3,4 @@ dtale = Blueprint('dtale', __name__, url_prefix='/dtale') # flake8: NOQA -from dtale.app import show # isort:skip +from dtale.app import show, get_instance, instances # isort:skip \ No newline at end of file diff --git a/dtale/app.py b/dtale/app.py index e458fd58..7c5ccf98 100644 --- a/dtale/app.py +++ b/dtale/app.py @@ -22,7 +22,7 @@ from dtale.cli.clickutils import retrieve_meta_info_and_version, setup_logging from dtale.utils import (build_shutdown_url, build_url, dict_merge, get_host, running_with_flask_debug, swag_from) -from dtale.views import cleanup, is_up, kill, startup +from dtale.views import DATA, DtaleData, cleanup, is_up, kill, startup if PY3: import _thread @@ -168,9 +168,9 @@ def get_send_file_max_age(self, name): def build_app(url, reaper_on=True, hide_shutdown=False, host=None): """ - Builds Flask application encapsulating endpoints for D-Tale's front-end + Builds :class:`flask:flask.Flask` application encapsulating endpoints for D-Tale's front-end - :return: Flask application + :return: :class:`flask:flask.Flask` application :rtype: :class:`dtale.app.DtaleFlask` """ @@ -211,7 +211,7 @@ def build_app(url, reaper_on=True, hide_shutdown=False, host=None): @swag_from('swagger/dtale/root.yml') def root(): """ - Flask routes which redirect to dtale/main + :class:`flask:flask.Flask` routes which redirect to dtale/main :return: 302 - flask.redirect('/dtale/main') """ @@ -220,7 +220,7 @@ def root(): @app.route('/favicon.ico') def favicon(): """ - Flask routes which returns favicon + :class:`flask:flask.Flask` routes which returns favicon :return: image/png """ @@ -229,7 +229,7 @@ def favicon(): @app.errorhandler(404) def page_not_found(e=None): """ - Flask routes which returns favicon + :class:`flask:flask.Flask` routes which returns favicon :param e: exception :return: text/html with exception information @@ -241,7 +241,7 @@ def page_not_found(e=None): @app.errorhandler(500) def internal_server_error(e=None): """ - Flask route which returns favicon + :class:`flask:flask.Flask` route which returns favicon :param e: exception :return: text/html with exception information @@ -269,7 +269,7 @@ def shutdown_server(): @swag_from('swagger/dtale/shutdown.yml') def shutdown(): """ - Flask route for initiating server shutdown + :class:`flask:flask.Flask` route for initiating server shutdown :return: text/html with server shutdown message """ @@ -290,7 +290,7 @@ def before_request(): @swag_from('swagger/dtale/site-map.yml') def site_map(): """ - Flask route listing all available flask endpoints + :class:`flask:flask.Flask` route listing all available flask endpoints :return: JSON of all flask enpoints [ [endpoint1, function path1], @@ -317,7 +317,7 @@ def has_no_empty_params(rule): @swag_from('swagger/dtale/version-info.yml') def version_info(): """ - Flask route for retrieving version information about D-Tale + :class:`flask:flask.Flask` route for retrieving version information about D-Tale :return: text/html version information """ @@ -328,7 +328,7 @@ def version_info(): @swag_from('swagger/dtale/health.yml') def health_check(): """ - Flask route for checking if D-Tale is up and running + :class:`flask:flask.Flask` route for checking if D-Tale is up and running :return: text/html 'ok' """ @@ -338,6 +338,18 @@ def health_check(): def initialize_process_props(host=None, port=None, force=False): + """ + Helper function to initalize global state corresponding to the host & port being used for your + :class:`flask:flask.Flask` process + + :param host: hostname to use otherwise it will default to the output of :func:`python:socket.gethostname` + :type host: str, optional + :param port: port to use otherwise default to the output of :func:dtale.app.find_free_port + :type port: str, optional + :param force: boolean flag to determine whether to ignore the :func:dtale.app.find_free_port function + :type force: bool + :return: + """ global ACTIVE_HOST, ACTIVE_PORT if force: @@ -367,9 +379,17 @@ def initialize_process_props(host=None, port=None, force=False): def find_free_port(): """ - Searches for free port on executing server for running Flask process + Searches for free port on executing server to run the :class:`flask:flask.Flask` process. Checks ports in range + specified using environment variables: - :return: string port number + DTALE_MIN_PORT (default: 40000) + DTALE_MAX_PORT (default: 49000) + + The range limitation is required for usage in tools such as jupyterhub. Will raise an exception if an open + port cannot be found. + + :return: port number + :rtype: int """ def is_port_in_use(port): @@ -393,7 +413,7 @@ def is_port_in_use(port): def show(data=None, host=None, port=None, name=None, debug=False, subprocess=True, data_loader=None, reaper_on=True, open_browser=False, notebook=False, force=False, **kwargs): """ - Entry point for kicking off D-Tale Flask process from python process + Entry point for kicking off D-Tale :class:`flask:flask.Flask` process from python process :param data: data which D-Tale will display :type data: :class:`pandas:pandas.DataFrame` or :class:`pandas:pandas.Series` @@ -404,7 +424,7 @@ def show(data=None, host=None, port=None, name=None, debug=False, subprocess=Tru :type port: str, optional :param name: optional label to assign a D-Tale process :type name: str, optional - :param debug: will turn on Flask debug functionality, defaults to False + :param debug: will turn on :class:`flask:flask.Flask` debug functionality, defaults to False :type debug: bool, optional :param subprocess: run D-Tale as a subprocess of your current process, defaults to True :type subprocess: bool, optional @@ -467,5 +487,31 @@ def _start(): instance.notebook() else: _start() + logger.info('D-Tale started at: {}'.format(url)) return instance + + +def instances(): + """ + Returns a dictionary of data IDs & :class:dtale.views.DtaleData objects pertaining to all the current pieces of + data being viewed + + :return: dict + """ + return {data_id: DtaleData(data_id, build_url(ACTIVE_PORT, host=ACTIVE_HOST)) for data_id in DATA} + + +def get_instance(data_id): + """ + Returns a :class:dtale.views.DtaleData object for the data_id passed as input, will return None if the data_id + does not exist + + :param data_id: integer string identifier for a D-Tale process's data + :type data_id: str + :return: :class:dtale.views.DtaleData + """ + data_id_str = str(data_id) + if data_id_str in DATA: + return DtaleData(data_id_str, build_url(ACTIVE_PORT, host=ACTIVE_HOST)) + return None diff --git a/dtale/cli/script.py b/dtale/cli/script.py index ea1635b8..3882d1fe 100644 --- a/dtale/cli/script.py +++ b/dtale/cli/script.py @@ -10,7 +10,7 @@ logger = getLogger(__name__) -@click.command(name='main', help='Run dtale from command-line') +@click.command(name='main', help='Run D-Tale from command-line') @click.option('--host', type=str, help='hostname or IP address of process') @click.option('--port', type=int, help='port number of process') @click.option('--debug', is_flag=True, help="flag to switch on Flask's debug mode") diff --git a/dtale/static/css/main.css b/dtale/static/css/main.css index a43d3dba..92b5a68a 100644 --- a/dtale/static/css/main.css +++ b/dtale/static/css/main.css @@ -8019,67 +8019,67 @@ a.text-dark:focus, a.text-dark:hover { } .Select__multi-value { - background-color: #52bbef; border-radius: 2px; - border: 1px solid #2a91d1; color: white; - display: inline-block; + display: inline-flex; font-size: 0.9em; margin-left: 5px; vertical-align: top; } -.Select--multi .Select__multi-value__remove, -.Select--multi .Select__multi-value__label { +.Select__multi-value__label { display: inline-block; } -.Select--multi .Select__multi-value__label { +.Select__multi-value__label { border-bottom-right-radius: 2px; border-top-right-radius: 2px; cursor: default; - color: white; + color: black; vertical-align: middle; } -.Select--multi a.Select__multi-value__label { +a.Select__multi-value__label { color: white; cursor: pointer; text-decoration: none; } -.Select--multi a.Select__multi-value__label:hover { +Select__multi-value__label:hover { text-decoration: underline; } -.Select--multi .Select__multi-value__remove { +.Select__multi-value__remove { cursor: pointer; border-bottom-left-radius: 2px; border-top-left-radius: 2px; - vertical-align: text-top; + color: black; } -.Select--multi .Select__multi-value__remove:hover, .Select--multi .Select__multi-value__remove:focus { +.Select__multi-value__remove:hover, +.Select__multi-value__remove:focus { background-color: #3bb2ed; color: #f2f2f2; } -.Select--multi .Select__multi-value__remove:active { +.Select__multi-value__remove:active { background-color: #2a91d1; } -.Select--multi.is-disabled .Select__multi-value { +.Select.is-disabled .Select__multi-value { background-color: #ebedee; border: 1px solid #cfd4d7; color: #404040; } -.Select--multi.is-disabled .Select__multi-value__remove { +.Select.is-disabled .Select__multi-value__remove { cursor: not-allowed; border-right: 1px solid #cfd4d7; } -.Select--multi.is-disabled .Select__multi-value__remove:hover, .Select--multi.is-disabled .Select__multi-value__remove:focus, .Select--multi.is-disabled .Select__multi-value__remove:active { +.Select.is-disabled .Select__multi-value__remove:hover, +.Select.is-disabled .Select__multi-value__remove:focus, +.Select.is-disabled .Select__multi-value__remove:active { background-color: #ebedee; } diff --git a/dtale/swagger/dtale/views/chart-data.yml b/dtale/swagger/dtale/views/chart-data.yml new file mode 100644 index 00000000..f7e3fe4a --- /dev/null +++ b/dtale/swagger/dtale/views/chart-data.yml @@ -0,0 +1,82 @@ +summary: Builds chart data based off of inputs for x, y, group & aggregation +description: | + * > + Aggregations are created using the [pandas.core.groupby.GroupBy](https://pandas.pydata.org/pandas-docs/stable/user_guide/groupby.html) + and the different aggregations available: count, first, last mean, median, min, max, std, var, mad, prod, sum + * Call this api passing a [pandas.DataFrame.query](https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#indexing-query) in your query string to perform coverage on certain data +tags: + - D-Tale API +parameters: + - name: data_id + in: path + required: true + schema: + type: string + description: identifier used to get data associated with this API call + - name: x + in: query + required: true + schema: + type: string + description: column to be used as x-axis of chart + - name: y + in: query + required: true + schema: + type: string + description: column to be used as y-axis of chart + - name: group + in: query + required: false + schema: + type: string + description: comma-separaed list of columns to group on + - name: agg + in: query + required: false + schema: + type: string + description: name of aggregation to be applied to y-axis data + enum: [count, first, last mean, median, min, max, std, var, mad, prod, sum] + - name: query + in: query + schema: + type: string + description: pandas dataframe query for filtering data down before correlations are generated +responses: + 200: + description: JSON structure of coverage data + content: + application/json: + schema: + oneOf: + - properties: + data: + type: object + description: object of all series within the chart (keys are series names, 'all' if only one series) + properties: + all: + type: object + properties: + x: + type: array + description: values for x-axis + y: + type: array + description: values for y-axis + min: + type: number + description: minimum y-axis value amongst all series + max: + type: number + description: maximum y-axis value amongst all series + - properties: + error: + type: string + description: Exception summary + traceback: + type: string + description: Exception traceback + success: + type: boolean + default: false diff --git a/dtale/swagger/dtale/views/correlations-ts.yml b/dtale/swagger/dtale/views/correlations-ts.yml index 517f2b8c..007dc613 100644 --- a/dtale/swagger/dtale/views/correlations-ts.yml +++ b/dtale/swagger/dtale/views/correlations-ts.yml @@ -5,6 +5,12 @@ description: | tags: - D-Tale API parameters: + - name: data_id + in: path + required: true + schema: + type: string + description: identifier used to get data associated with this API call - name: cols in: query required: true @@ -30,19 +36,23 @@ responses: schema: oneOf: - properties: - ':col1:col2': + data: type: object description: correlation data for col1 & col2 properties: - data: - type: array - items: - type: object - properties: - corr: + all: + type: object + properties: + x: + type: array + description: values for x-axis + items: type: float description: correlation - date: + y: + type: array + description: values for y-axis + items: type: string description: date of correlation max: diff --git a/dtale/swagger/dtale/views/correlations.yml b/dtale/swagger/dtale/views/correlations.yml index 0bd40c78..8ccb8639 100644 --- a/dtale/swagger/dtale/views/correlations.yml +++ b/dtale/swagger/dtale/views/correlations.yml @@ -5,6 +5,12 @@ description: | tags: - D-Tale API parameters: + - name: data_id + in: path + required: true + schema: + type: string + description: identifier used to get data associated with this API call - name: query in: query schema: diff --git a/dtale/swagger/dtale/views/coverage.yml b/dtale/swagger/dtale/views/coverage.yml deleted file mode 100644 index b9175ef0..00000000 --- a/dtale/swagger/dtale/views/coverage.yml +++ /dev/null @@ -1,100 +0,0 @@ -summary: Gathers coverage information(counts) for a column grouped by other column(s) -description: | - * > - Groups data by group column (if group column is datetime then a frequency (D, W, M, Q, Y) can be specified as well - using [pandas.DataFrame.groupby](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.groupby.html) - * > - Counts are created using the [pandas.core.groupby.GroupBy.count](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.core.groupby.GroupBy.count.html) - which excludes missing values(nan) - * Call this api passing a [pandas.DataFrame.query](https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#indexing-query) in your query string to perform coverage on certain data -tags: - - D-Tale API -parameters: - - name: col - in: query - required: true - schema: - type: string - description: column of pandas dataframe to record coverage on - - name: filters (deprecated) - in: query - schema: - type: array - description: > - breadcrumb trail of filters when grouping on datetime column. For example, if the original frequency of your - datetime group is Yearly, then you can do the following drill downs Yearly -> Quarterly -> Monthly -> Weekly -> Daily - items: - type: object - properties: - name: - type: string - description: name of datetime column used in group - prevFreq: - type: string - enum: [Y, Q, M, W, D] - description: frequency you are performing the drill-down on - freq: - type: string - enum: [Y, Q, M, W, D] - description: sub-frequency you want to convert to, `prevFreq` must be greater than `freq` - date: - type: string - description: date string of the data point for `prevFreq` you want broken up into `freq` - - name: group - in: query - required: true - schema: - type: array - description: list of columns (and frequency if column is datetime) to group on - items: - type: object - properties: - name: - type: string - description: name of column in dataframe - required: true - freq: - type: string - enum: [Y, Q, M, W, D] - description: if column is datetime you have the additional option to specify a frequency - default: D - - name: query - in: query - schema: - type: string - description: pandas dataframe query for filtering data down before correlations are generated -responses: - 200: - description: JSON structure of coverage data - content: - application/json: - schema: - oneOf: - - properties: - data: - type: object - description: coverage information - properties: - col: - type: array - items: - type: integer - description: count of non-missing records for column at specific group value - labels: - type: array - items: - type: string - decription: string representation of group values (should be same length as *col* property) - success: - type: boolean - default: true - - properties: - error: - type: string - description: Exception summary - traceback: - type: string - description: Exception traceback - success: - type: boolean - default: false diff --git a/dtale/swagger/dtale/views/data.yml b/dtale/swagger/dtale/views/data.yml index 64fee376..4fb88773 100644 --- a/dtale/swagger/dtale/views/data.yml +++ b/dtale/swagger/dtale/views/data.yml @@ -4,6 +4,12 @@ description: | tags: - D-Tale API parameters: + - name: data_id + in: path + required: true + schema: + type: string + description: identifier used to get data associated with this API call - name: ids in: query schema: diff --git a/dtale/swagger/dtale/views/describe.yml b/dtale/swagger/dtale/views/describe.yml index e314cce0..053a6a91 100644 --- a/dtale/swagger/dtale/views/describe.yml +++ b/dtale/swagger/dtale/views/describe.yml @@ -3,6 +3,19 @@ description: | * [pandas.Series.describe](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.describe.html) tags: - D-Tale API +parameters: + - name: data_id + in: path + required: true + schema: + type: string + description: identifier used to get data associated with this API call + - name: column + in: path + required: true + schema: + type: string + description: column you want to describe responses: 200: description: JSON object containing describe information, unique values and success flag diff --git a/dtale/swagger/dtale/views/dtypes.yml b/dtale/swagger/dtale/views/dtypes.yml index d8d1d04a..7110fb74 100644 --- a/dtale/swagger/dtale/views/dtypes.yml +++ b/dtale/swagger/dtale/views/dtypes.yml @@ -1,6 +1,13 @@ summary: Fetch list of column names and dtypes tags: - D-Tale API +parameters: + - name: data_id + in: path + required: true + schema: + type: string + description: identifier used to get data associated with this API call responses: 200: description: JSON object containing column name/dtype pairs and success flag diff --git a/dtale/swagger/dtale/views/histogram.yml b/dtale/swagger/dtale/views/histogram.yml index d847daa0..91e10d3f 100644 --- a/dtale/swagger/dtale/views/histogram.yml +++ b/dtale/swagger/dtale/views/histogram.yml @@ -5,6 +5,12 @@ description: | tags: - D-Tale API parameters: + - name: data_id + in: path + required: true + schema: + type: string + description: identifier used to get data associated with this API call - name: col in: query required: true diff --git a/dtale/swagger/dtale/views/main.yml b/dtale/swagger/dtale/views/main.yml index d727fd10..5ef41716 100644 --- a/dtale/swagger/dtale/views/main.yml +++ b/dtale/swagger/dtale/views/main.yml @@ -1,6 +1,13 @@ summary: Main route which is hit from browsers returning HTML structure for housing all AJAX operations tags: - D-Tale API +parameters: + - name: data_id + in: path + required: true + schema: + type: string + description: identifier used to get data associated with this API call responses: 200: description: HTML from jinja template containing base DOM and JS script imports diff --git a/dtale/swagger/dtale/views/processes.yml b/dtale/swagger/dtale/views/processes.yml index 3a999e9e..01fa370a 100644 --- a/dtale/swagger/dtale/views/processes.yml +++ b/dtale/swagger/dtale/views/processes.yml @@ -1,6 +1,13 @@ summary: Fetch list of running D-Tale processes tags: - D-Tale API +parameters: + - name: data_id + in: path + required: true + schema: + type: string + description: identifier used to get data associated with this API call responses: 200: description: JSON object containing array of instance objects and success flag diff --git a/dtale/swagger/dtale/views/scatter.yml b/dtale/swagger/dtale/views/scatter.yml index 5cb007cb..e6cb1a12 100644 --- a/dtale/swagger/dtale/views/scatter.yml +++ b/dtale/swagger/dtale/views/scatter.yml @@ -5,6 +5,12 @@ description: | tags: - D-Tale API parameters: + - name: data_id + in: path + required: true + schema: + type: string + description: identifier used to get data associated with this API call - name: cols in: query required: true diff --git a/dtale/swagger/dtale/views/test-filter.yml b/dtale/swagger/dtale/views/test-filter.yml index 3c6bf3ae..0bcf09af 100644 --- a/dtale/swagger/dtale/views/test-filter.yml +++ b/dtale/swagger/dtale/views/test-filter.yml @@ -4,6 +4,12 @@ description: | tags: - D-Tale API parameters: + - name: data_id + in: path + required: true + schema: + type: string + description: identifier used to get data associated with this API call - name: query in: query schema: diff --git a/dtale/swagger/dtale/views/update-settings.yml b/dtale/swagger/dtale/views/update-settings.yml index 14418007..cbe3e40a 100644 --- a/dtale/swagger/dtale/views/update-settings.yml +++ b/dtale/swagger/dtale/views/update-settings.yml @@ -2,6 +2,12 @@ summary: Push updates to stateful settings associated with the port in request ( tags: - D-Tale API parameters: + - name: data_id + in: path + required: true + schema: + type: string + description: identifier used to get data associated with this API call - name: settings in: query schema: diff --git a/dtale/utils.py b/dtale/utils.py index cc1f3826..f46d00ee 100644 --- a/dtale/utils.py +++ b/dtale/utils.py @@ -40,10 +40,26 @@ def running_with_flask_debug(): def get_host(host=None): + """ + Returns host input if it exists otherwise the output of :func:`python:socket.gethostname` + + :param host: hostname, can start with 'http://', 'https://' or just the hostname itself + :type host: str, optional + :return: str + """ return host or socket.gethostname() def build_url(port, host=None): + """ + Returns full url combining host(if not specified will use the output of :func:`python:socket.gethostname`) & port + + :param port: integer string for the port to be used by the :class:`flask:flask.Flask` process + :type port: str + :param host: hostname, can start with 'http://', 'https://' or just the hostname itself + :type host: str, optional + :return: str + """ final_host = get_host(host) if final_host.startswith('http'): return '{}:{}'.format(final_host, port) @@ -137,13 +153,20 @@ def json_string(x, nan_display=''): """ convert value to string to be used within JSON output + If a :class:`python.UnicodeEncodeError` occurs then :func:`python:str.encode` will be called on input + :param x: value to be converted to string :param nan_display: if `x` is :attr:`numpy:numpy.nan` then return this value :return: string value :rtype: str """ if x: - return str(x) + try: + return str(x) + except UnicodeEncodeError: + return x.encode('utf-8') + except BaseException as ex: + logger.exception(ex) return nan_display @@ -226,7 +249,9 @@ def json_timestamp(x, nan_display=''): :rtype: bigint """ try: - return int((time.mktime(x.timetuple()) + (old_div(x.microsecond, 1000000.0))) * 1000) + output = (pd.Timestamp(x) if isinstance(x, np.datetime64) else x) + output = int((time.mktime(output.timetuple()) + (old_div(output.microsecond, 1000000.0))) * 1000) + return output except BaseException: return nan_display @@ -283,6 +308,9 @@ def format_dict(self, lst): def format_dicts(self, lsts): return [self.format_dict(l) for l in lsts] + def format_lists(self, df): + return {name: [f(v, nan_display=self.nan_display) for v in df[name].values] for _idx, name, f in self.fmts} + def classify_type(type_name): """ diff --git a/dtale/views.py b/dtale/views.py index b6905f3e..871092af 100644 --- a/dtale/views.py +++ b/dtale/views.py @@ -12,7 +12,6 @@ import pandas as pd import requests from future.utils import string_types -from pandas.tseries.offsets import Day, MonthBegin, QuarterBegin, YearBegin from dtale import dtale from dtale.cli.clickutils import retrieve_meta_info_and_version @@ -51,10 +50,21 @@ def in_ipython_frontend(): def kill(base): + """ + This function fires a request to this instance's 'shutdown' route to kill it + + """ requests.get(build_shutdown_url(base)) def is_up(base): + """ + This function checks to see if instance's :mod:`flask:flask.Flask` process is up by hitting 'health' route. + + Using `verify=False` will allow us to validate instances being served up over SSL + + :return: `True` if :mod:`flask:flask.Flask` process is up and running, `False` otherwise + """ try: return requests.get('{}/health'.format(base), verify=False).ok except BaseException: @@ -66,8 +76,15 @@ class DtaleData(object): Wrapper class to abstract the global state of a D-Tale process while allowing a user to programatically interact with a running D-Tale instance - :param port: integer string for a D-Tale process's port - :type port: str + :param data_id: integer string identifier for a D-Tale process's data + :type data_id: str + :param url: endpoint for instances :class:`flask:flask.Flask` process + :type url: str + + Attributes: + _data_id data identifier + _url :class:`flask:flask.Flask` endpoint + _notebook_handle reference to the most recent :class:`ipython:IPython.display.DisplayHandle` created :Example: @@ -103,11 +120,15 @@ def data(self, data): startup(self._url, data=data, data_id=self._data_id) def main_url(self): + """ + Helper function creating main :class:`flask:flask.Flask` route using instance's url & data_id + :return: str + """ return '{}/dtale/main/{}'.format(self._url, self._data_id) def kill(self): """ - This function fires a request to this instance's 'shutdown' route to kill it + Helper function to pass instance's endpoint to :func:dtale.views.kill """ kill(self._url) @@ -121,11 +142,7 @@ def open_browser(self): def is_up(self): """ - This function checks to see if instance's :mod:`flask:flask.Flask` process is up by hitting 'health' route. - - Using `verify=False` will allow us to validate instances being served up over SSL - - :return: `True` if :mod:`flask:flask.Flask` process is up and running, `False` otherwise + Helper function to pass instance's endpoint to :func:dtale.views.is_up """ return is_up(self._url) @@ -154,13 +171,19 @@ def __repr__(self): return '' return self.main_url() - def _build_iframe(self, width='100%', height=350): + def _build_iframe(self, route='/dtale/iframe/', params=None, width='100%', height=350): """ Helper function to build an :class:`ipython:IPython.display.IFrame` if that module exists within your environment + :param route: the :class:`flask:flask.Flask` route to hit on D-Tale + :type route: str, optional + :param params: properties & values passed as query parameters to the route + :type params: dict, optional :param width: width of the ipython cell + :type width: str or int, optional :param height: height of the ipython cell + :type height: str or int, optional :return: :class:`ipython:IPython.display.IFrame` """ try: @@ -168,10 +191,13 @@ def _build_iframe(self, width='100%', height=350): except ImportError: logger.info('in order to use this function, please install IPython') return None - iframe_url = '{}/dtale/iframe/{}'.format(self._url, self._data_id) + iframe_url = '{}{}{}'.format(self._url, route, self._data_id) + if params is not None: + formatted_params = ['{}={}'.format(k, ','.join(make_list(params[k]))) for k in sorted(params)] + iframe_url = '{}?{}'.format(iframe_url, '&'.join(formatted_params)) return IFrame(iframe_url, width=width, height=height) - def notebook(self, width='100%', height=350): + def notebook(self, route='/dtale/iframe/', params=None, width='100%', height=350): """ Helper function which checks to see if :mod:`flask:flask.Flask` process is up and running and then tries to build an :class:`ipython:IPython.display.IFrame` and run :meth:`ipython:IPython.display.display` on it so @@ -180,8 +206,14 @@ def notebook(self, width='100%', height=350): A reference to the :class:`ipython:IPython.display.DisplayHandle` is stored in _notebook_handle for updating if you are running ipython>=5.0 + :param route: the :class:`flask:flask.Flask` route to hit on D-Tale + :type route: str, optional + :param params: properties & values passed as query parameters to the route + :type params: dict, optional :param width: width of the ipython cell + :type width: str or int, optional :param height: height of the ipython cell + :type height: str or int, optional """ try: from IPython.display import display @@ -192,10 +224,55 @@ def notebook(self, width='100%', height=350): while not self.is_up(): time.sleep(0.01) - self._notebook_handle = display(self._build_iframe(width=width, height=height), display_id=True) + self._notebook_handle = display( + self._build_iframe(route=route, params=params, width=width, height=height), display_id=True + ) if self._notebook_handle is None: self._notebook_handle = True + def notebook_correlations(self, col1, col2, width='100%', height=350): + """ + Helper function to build an `ipython:IPython.display.IFrame` pointing at the correlations popup + + :param col1: column on left side of correlation + :type col1: str + :param col2: column on right side of correlation + :type col2: str + :param width: width of the ipython cell + :type width: str or int, optional + :param height: height of the ipython cell + :type height: str or int, optional + :return: :class:`ipython:IPython.display.IFrame` + """ + self.notebook('/dtale/popup/correlations/', params=dict(col1=col1, col2=col2), width=width, height=height) + + def notebook_charts(self, x, y, group=None, aggregation=None, width='100%', height=350): + """ + Helper function to build an `ipython:IPython.display.IFrame` pointing at the charts popup + + :param x: column to be used as x-axis of chart + :type x: str + :param y: column to be used as y-axis of chart + :type y: str + :param group: comma-separated string of columns to group chart data by + :type group: str, optional + :param aggregation: points to a specific function that can be applied to + :func: pandas.core.groupby.DataFrameGroupBy. Possible values are: count, first, last mean, + median, min, max, std, var, mad, prod, sum + :type aggregation: str, optional + :param width: width of the ipython cell + :type width: str or int, optional + :param height: height of the ipython cell + :type height: str or int, optional + :return: :class:`ipython:IPython.display.IFrame` + """ + params = dict(x=x, y=y) + if group: + params['group'] = ','.join(make_list(group)) + if aggregation: + params['aggregation'] = aggregation + self.notebook('/dtale/popup/charts/', params=params, width=width, height=height) + def adjust_cell_dimensions(self, width='100%', height=350): """ If you are running ipython>=5.0 then this will update the most recent notebook cell you displayed D-Tale in @@ -342,8 +419,10 @@ def _view_main(data_id, iframe=False): """ Helper function rendering main HTML which will also build title and store whether we are viewing from an