WIP: oreo remote conf

[vidma]

• Circuit breakers
  • finish circuit breakers integration (track ksu.lds_names)
• Remote conf
  • override global stats switch(es) remotely: dsl/extractors: compute_stats
  • http client + specs
  • init with global conf, so usage could be simpler ?
  • later: cache requests/responses by LDS name (if multiple PDS of same LDS) ?
  • later: finish handling the case for new/unknown metrics (unknown by backend stats or columns)
• Fixes for API integration:
  • header: Accept
  • logicalDatasources -> logicalDataSources
  • in resp it could be both ldsName and ldsPath
• write small integration test with all the tools above
  • check if override global stats conf remotely works: input/output stats
  • make sure to disable stats for in-memory DS
• double-check/update PySpark integration
  • circuitbreaker python wrapper code (exists, just add to kensu-py)
  • anything else
• numpy / ndarray - might be slow? maybe stats should be deprecated/disabled here by default (as there's no direct read)? maybe there's write?
• bigquery
• pandas: simple inefficient
• later: pandas: make it faster, but more complex
• requests: api.py: count: len(json)
• psychopg2
  • output stats via pandas extractor (but need tweaks)
• matplotlib
  * stats via pandas extractor, but modified afterwards. need something special here
• aiflow
  • bigquery

unclear:

• bigquery (part of): one unclear special case
• boto3: custom end-user reported stats
  • not covered/known by backend as might not relate to schema nor known metric names, how should remote enable/disable work here?
  • just keep it always enabled?
  • kensu_put(): check what is put_body.metadata['stats'] ?
• ML: glounts -not clear. seem to be via pandas/numpy (DF, Series, ndarray) or no stats
• ML: what about sklearn ML metrics ?

these seem to have no stats:

• ML: sklearn (no stats there, it seems?)
• json, pickle, pysftp, pandas_gbq: no stats