Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP][Flytekit] Add variable_map in LiteralResolver for FlyteRemote.get Error on Dataclass/Pydantic Models #3031

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

[WIP][Flytekit] Add variable_map in LiteralResolver for FlyteRemote.get Error on Dataclass/Pydantic Models #3031

Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
a88827c
feat: add variable_map in data response
mao3267 Dec 24, 2024
e0b8f3a
fix: make lint
mao3267 Jan 4, 2025
4c73d6b
Merge branch 'master' of https://github.com/mao3267/flytekit into add…
mao3267 Jan 4, 2025
3a61ac9
Merge branch 'master' of https://github.com/mao3267/flytekit into add…
mao3267 Jan 15, 2025
6072099
fix: add null check for variable map
mao3267 Jan 17, 2025
270105d
fix: changed incorrectly
mao3267 Jan 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 17 additions & 3 deletions flytekit/remote/remote.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -358,8 +358,14 @@ def get(
data_response = self.client.get_data(flyte_uri)

if data_response.HasField("literal_map"):
from flytekit.models.interface import VariableMap

lm = LiteralMap.from_flyte_idl(data_response.literal_map)
return LiteralsResolver(lm.literals)
if data_response.variable_map is not None:
vm = VariableMap.from_flyte_idl(data_response.variable_map)
return LiteralsResolver(lm.literals, vm.variables)
else:
return LiteralsResolver(lm.literals)
elif data_response.HasField("literal"):
return Literal.from_flyte_idl(data_response.literal)
elif data_response.HasField("pre_signed_urls"):
Expand Down Expand Up @@ -2518,11 +2524,19 @@ def _assign_inputs_and_outputs(
):
"""Helper for assigning synced inputs and outputs to an execution object."""
input_literal_map = self._get_input_literal_map(execution_data)
execution._inputs = LiteralsResolver(input_literal_map.literals, interface.inputs, self.context)
# 如果 interface.inputs 为空,则只返回 literals
if not interface.inputs:
execution._inputs = input_literal_map.literals
else:
execution._inputs = LiteralsResolver(input_literal_map.literals, interface.inputs, self.context)

if execution.is_done and not execution.error:
output_literal_map = self._get_output_literal_map(execution_data)
execution._outputs = LiteralsResolver(output_literal_map.literals, interface.outputs, self.context)
# 如果 interface.outputs 为空,则只返回 literals
if not interface.outputs:
execution._outputs = output_literal_map.literals
else:
execution._outputs = LiteralsResolver(output_literal_map.literals, interface.outputs, self.context)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider consolidating input/output handling logic

Consider consolidating the input/output handling logic to avoid code duplication. The same pattern is repeated for both inputs and outputs.

Code suggestion
Check the AI-generated fix before applying
 -        # 如果 interface.inputs 为空,则只返回 literals
 -        if not interface.inputs:
 -            execution._inputs = input_literal_map.literals
 -        else:
 -            execution._inputs = LiteralsResolver(input_literal_map.literals, interface.inputs, self.context)
 -
 -        if execution.is_done and not execution.error:
 -            output_literal_map = self._get_output_literal_map(execution_data)
 -            # 如果 interface.outputs 为空,则只返回 literals
 -            if not interface.outputs:
 -                execution._outputs = output_literal_map.literals
 -            else:
 -                execution._outputs = LiteralsResolver(output_literal_map.literals, interface.outputs, self.context)
 +        def _assign_literals(literal_map, interface_vars, target_attr):
 +            if not interface_vars:
 +                setattr(execution, target_attr, literal_map.literals)
 +            else:
 +                setattr(execution, target_attr, LiteralsResolver(literal_map.literals, interface_vars, self.context))
 +
 +        _assign_literals(input_literal_map, interface.inputs, '_inputs')
 +
 +        if execution.is_done and not execution.error:
 +            output_literal_map = self._get_output_literal_map(execution_data)
 +            _assign_literals(output_literal_map, interface.outputs, '_outputs')

Code Review Run #7c00db

Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

return execution

def _get_input_literal_map(self, execution_data: ExecutionDataResponse) -> literal_models.LiteralMap:
Expand Down
Loading