Skip to content

Lazy fields that point to a specific Job #800

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tclose opened this issue Apr 28, 2025 · 1 comment
Open

Lazy fields that point to a specific Job #800

tclose opened this issue Apr 28, 2025 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@tclose
Copy link
Contributor

tclose commented Apr 28, 2025

Lazy fields that point to a specific Job, i.e. instead of a Node, to allow execution of a node with partially resolved state arrays when the upstream hasn't completed yet, or partially errored. Note that these would not replace the current "lazy node out" fields, but be generated from them at runtime and passed to the jobs that arise from the node when they are created.

When a state is initialised and arrays are split over, if you have a LazyJobOutField (or better name), even if the full array isn't ready yet you can create an array filled with LazyJobOutField objects, which then can be resolved independently allowing downstream jobs to be executed when its immediate predecessors have finished, even if the predecessors of its sibling jobs have failed.

@tclose tclose self-assigned this Apr 28, 2025
@tclose tclose converted this from a draft issue Apr 28, 2025
@tclose tclose added the enhancement New feature or request label Apr 29, 2025
@tclose tclose moved this from Proposed to temp in Pydra Roadmap Apr 29, 2025
@tclose tclose moved this from temp to To do (v1.0) in Pydra Roadmap Apr 29, 2025
@effigies
Copy link
Contributor

I think this should definitely be 1.1 if not 1.0. It feels like a potentially big can of worms, and I don't know how I'd tackle it myself, so I'm hesitant to put it on 1.0.

@tclose tclose moved this from Triage to v1.1-v1.2 in Pydra Roadmap May 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: v1.1-v1.2
Development

No branches or pull requests

2 participants