[BUG] Map_task should support launch plans, but get type errors

[pimdh]

Describe the bug

The API suggests that map_task supports a launchplan. However, if I try the following, I get typing errors:

from flytekit import LaunchPlan, map_task, task, workflow


@task
def my_task(x: int) -> int:
    return x * 2


@workflow
def my_wf(x: int) -> int:
    return my_task(x)


my_lp = LaunchPlan.create("my_lp", my_wf)


@workflow
def map_wf(nums: list[int]) -> list[int]:
    return map_task(my_lp)(x=nums)

Upon registration, I get the following errors:

        Details: failed to compile workflow for [resource_type:WORKFLOW project:"scratch" domain:"main" name:"flyte_playground.map_launchplan.map_wf" version:"767711eb9510a25f9b7512cd"] with err: Collected Errors: 2
        Error 0: Code: MismatchingTypes, Node Id: end-node, Description: The output variable 'my_lp.o0' has type [simple:INTEGER], but it's assigned to the input variable 'o0' which has type type [collection_type:{simple:INTEGER}].
        Error 1: Code: MismatchingTypes, Node Id: n0, Description: The output variable 'nums' has type [collection_type:{simple:INTEGER}], but it's assigned to the input variable 'my_lp.x' which has type type [simple:INTEGER].

May be related to #5684

Expected behavior

Should be registered fine.

Additional context to reproduce

Flytekit 1.14.6

Screenshots

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[kumare3]

This is not a bug, This is not a supported feature in flyte. This is available in Union today - https://docs.union.ai/byoc/user-guide/core-concepts/launch-plans/mapping-over-launch-plans#mapping-over-launch-plans

Reason is, the fundamental way, launchplans are mapped over needs a large state storage system, which does not fit default array_nodes storage system. Union has a different engine, which can store data efficiently and in higher resolution, does surpassing these limits. Its a large lift to support in oss and we do not currently plan to support - but we will definitely visit this in a few months, as we upstream some of our learnings from Union.

[pimdh]

Thanks for the reply!
Maybe the type hints/docstrings of map_task can be updated to reflect that launchplans are not supported?
https://docs.flyte.org/en/latest/api/flytekit/generated/flytekit.map_task.html#flytekit.map_task
Thanks again!

[eapolinario]

Coalescing the discussions about this topic in #5684.