Merge pull request #1433 from saulpw/v2.9dev

v2.9 Release

.circleci/config.yml

@@ -18,16 +18,17 @@ jobs:
       - run: sudo chown -R circleci:circleci /usr/local/share
       - run: sudo chown -R circleci:circleci /usr/local/lib
       - run: sudo chown -R circleci:circleci /usr/local/include/
+      - run: touch ~/.visidatarc
       - run: echo 'export PYTHONPATH=~/visidata:~/visidata/visidata' >> $BASH_ENV
       - run: source $BASH_ENV
       - run: locale
       - run:
           name: dependencies
           command: |
             pip3 install -r requirements.txt
-      - run:
-          command: |
-            dev/test.sh
+            pip3 install pytest
+      - run: pytest -sv visidata/tests/test_commands.py
+      - run: dev/test.sh
 
   test-3.7:
     <<: *test-template

CHANGELOG.md

@@ -1,5 +1,90 @@
 # VisiData version history
 
+# 2.9 (2022-04-XX)
+
+- [ux] add confirming modal dialog
+    - only "y" required to confirm
+- add XDG support (thanks @jck for the PR #1420)
+    - `options.config` default is now `"$XDG_CONFIG_HOME"/visidata/config.py` if `$XDG_CONFIG_HOME` is set and `config.py` exists. If not, falls back to the standard `$HOME/.visidatarc`.
+    - vendor [appdirs.py](https://github.com/ActiveState/appdirs/blob/master/appdirs.py)
+- [cmdlog] support variables in .vdj (requested by @jungle-boogie #1364)
+    - in the .vdj, write variables like so: `${variableName}`
+    - then on the CLI: `vd -p foo.vdj variableName=bar`
+- [loaders arrow] new Apache Arrow IPC loader/saver (requested by @d-miketa #1369) (requires `pyarrow`)
+    - add `.arrow` (file) and .arrows (streaming) formats
+    - add more native `parquet` loader via `pyarrow`
+- preliminary "windowing" for referencing x rows before and y rows after in an expression (requested by @maxigit #1399, @MMesch #1129, @samuelludwig #1210)
+    - press `w` (longname: `addcol-window`) followed by two numbers: the number of rows to aggregate *before* and *after* the current row.
+    - there will be a new column, `foo_window`, where each row contains a list of aggregated rows. after that, e.g. `=` `sum(foo_window)` to get a running total for each row
+- add `setcol-format-enum` which takes e.g. `A=apple B=banana` and uses that as a mapping when formatting a column.
+- vendor `https://github.com/saulpw/unzip-http`; allows the downloading of individual files from a .zip file over http without downloading the entire archive (requires `urllib3` package)
+- add `save-source` to save a root sheet directly to its source
+- add `setcol-formatter` to specify formatting function used for Column (default: `generic` formats based on type and fmtstr).  Can be `json` or `python` or a custom formatter
+
+
+## Improvements
+
+- [cli] when `-v` or `-h` VisiData now does not read config or do anything else (requested by @geekscrapy #1340)
+- [cmdlog] set `.vdj` to be the default cmdlog format
+- [replay] allow column names to be numbers (reported by @frosencrantz #1309)
+    - if wishing to reference a column index, required to be an int in a .vdj cmdlog
+- [cmdlog] when saving cmdlogs, type column indices as integers, and column names as strings
+    - If replaying, and *col* is an `int`, the CmdLog will index by position.
+    - If *col* is a `str` it will index by column name.
+- [display] preview first n elements of a list/dict cell
+- [regex] add unbound `addcol-<regex>` commands
+- [rtl] improvements to right-to-left text display (requested by @dotancohen #1392)
+- [man] have `vd --help` open the .txt manpage by default (requested by @halloleo #1332)
+- [mouse] invert scroll wheel direction (requested by @marcobra #1351)
+- [performance] improvements to plotting of empty canvas, multiline display, and draw-ing functions
+- [plugins] notify when plugin update available (thanks @geekscrapy for PR #1355)
+
+## Bugfixes
+
+- [aggregators] fail on setting an unknown aggregator in **Columns Sheet** (reported by @geekscrapy #1299)
+- [aggregators] handle `delete-cell` case for aggregators column in **Columns Sheet** (reported by @geekscrapy #1299)
+- [aggregators] fix quartile aggregators (reported by @pnfnp #1312)
+- [aggregators] fix copying of aggregators when duplicating a sheet (reported by @frosencrantz #1373)
+- [canvas] do not use "other" label when there are exactly 9 columns being plotted (reported by @tdussa #1198)
+- [cli] fix `+:subsheet:col:row:` when `load_lazy` is False
+- [delete-row] clear deleted rows from `selectedRows` (reported by @geekscrapy #1284)
+- [exec-longname] output warning, if longname does not exist
+- [expr] prefer visible columns over hidden columns (reported by @frosencrantz #1360)
+- [freeze-sheet] carry over column attributes for freeze-sheet (reported by @frosencrantz #1373)
+- [import-python] use command-specific history (reported by @frosencrantz #1243)
+- [IndexSheet] fix renaming of sheet names from an IndexSheet (reported by @aborruso #1339)
+- [input] handle history for non-string input values (reported by @frosencrantz #1371)
+- [loaders pandas] fix (`dup-selected`) `"`of selected rows for **Pandas Sheet** (reported and fixed by @jasonwrith #1315 #1316)
+- [loaders usv] swap delimiters (reported by @frosencrantz #1383)
+- [loaders usv] save delimiter override options (reported by @frosencrantz #1383)
+- [loaders usv] fix saving header with usv row delimiter (reported by @frosencrantz #1383)
+- [loaders xlsx] fix clipboard on XlsxSheets (reported by @jungle-boogie #1348)
+- [macros] fix macro-record keystroke setting (reported by @fatherofinvention #1280)
+- [mouse] stay disabled after input (reported by @holderbp #1401)
+- [mouse] fix for pypy3 (thanks @LaPingvino for PR
+- [quitguard] refinement of quit-sheet protection (reported by @geekscrapy #1037, #1381)
+- [save-selected] get sheet names for saving from selected rows (reported by @aborruso #1339)
+- [shell] strip trailing whitespace in `z;` output (reported by @justin2004 #1370)
+- [tty] fix bug where piping async output into stdin broke visidata keyboard input (reported by @ovikk13 #1347)
+- [undo] fix issue where undoing a reload blanks the current sheet; do not set undos for reload sheet (#1302)
+- [unset-option] fix issue where Exception is raised on the next undo-able command run after `unset-option` (reported by @ajkerrigan #1267)
+- [windows] require `windows-curses` installation on Windows (thanks @ajkerrigan for PR #1407; reported by @schiltz3 #1268, @aagha #1406)
+
+## API
+
+- add Column.formatter (generic, json, python)
+- add SqliteQuerySheet to globals
+- `vd.loadConfigFile()` no longer needs a filename argument, and will use `options.config` by default (#211)
+- use `newline="` for csv.writer (thanks @daviewales for PR #1368)
+- make `ItemColumn` a proper class for inheritance
+- add `openJoin` and `openMelt` to allow overriding by plugin sheetsS
+- addColumn takes `*cols` (reported by @pyglot #1414)
+
+## Deprecated
+
+- deprecate old vdmenu system
+    - remove `Shift+V` command
+
 # 2.8 (2021-12-15)
 
 ## Improvements

dev/checklists/manual-tests.md

@@ -93,3 +93,4 @@
     - save name makes sense
     - fails if not offered a directory
     - succeeds if offered a directory
+21. Test macro-record.