Inconsistent behavior between --exclude-where and --query for missing columns

[victorlin]

Current Behavior

• --exclude-where nonexistent_column='value' ignores the missing column silently
• --query nonexistent_column!='value' raises a pandas UndefinedVariableError

Expected behavior

--exclude-where nonexistent_column='value' and --query nonexistent_column!='value' should produce the same effect.

How to reproduce

Setup:

echo 'strain\tcountry\tdate
SEQ1\tA\t2018-03-24
SEQ2\tB\t2018-03-25' > metadata.tsv

Run with --exclude-where:

$ augur filter \
  --metadata metadata.tsv \
  --exclude-where nonexistent_column='value' \
  --output-strains out.txt
0 strains were dropped during filtering
2 strains passed all filters

Run with --query:

$ augur filter \
  --metadata metadata.tsv \
  --query nonexistent_column!='value' \
  --output-strains out.txt
...
pandas.core.computation.ops.UndefinedVariableError: name 'nonexistent_column' is not defined

Possible solution

Both exclusion options should be consistent, though behavior could be any of the following:

1. Pass silently (current behavior for --exclude-where)
2. Pass with warning
3. Error (current behavior for --query, but raise a custom FilterException to hide internal pandas implementation)

Your environment: if running Nextstrain locally

• augur 14.0.0

[victorlin]

Related to #754, which is about --group-by behavior with missing columns.

[victorlin]

Oops, I misread #754. I see it is also mentions filter options briefly, so this issue is actually a subset of that one.