feature: add req parsers for configs #2248
Conversation
| np_deps: Dict[str, str], | ||
| sys_deps: Dict[str, str], | ||
| ) -> Optional[str]: | ||
| from packaging.requirements import InvalidRequirement, Requirement |
There was a problem hiding this comment.
only change to the original here, as packaging is not available by default in environments, having the import trickle down to the top-level causes issues.
|
Let me make sure we properly override these internally and if so, this should be good to go. |
| YML_SPLIT_LINE = re.compile(r"(?:=\s)?(<=|>=|~=|==|<|>|=)") | ||
|
|
||
|
|
||
| def req_parser(config_value: str) -> Dict[str, Any]: |
There was a problem hiding this comment.
is it possible for extensions to define and override these parsers? that way this implementation can be decoupled with metaflow-netflixext
There was a problem hiding this comment.
There is no special mechanism right now. If this was something in core, the override we do for things like Parameter would work fine but here this is not. I don't think this will actually work right now (looking at it) because the plugin pypi doesn't exist internally (it's not loaded). I'm going to have to play around with it a bit. We could of course add a similar plugin category PARSERS (like everything else) and resolve it in the plugins. Bringing it back up here though will be a bit trickier.
moving from https://github.com/romain-intel/metaflow-dep-parsers
adds
req_parserfor@pypiandyml_parserfor@condato enable passing dependencies as config values.