Skip to content

feat(loop): add optional overlap support to allow concurrent loop executions #2771

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 32 commits into
base: master
Choose a base branch
from

Conversation

Lumabots
Copy link

Summary

Information

  • This PR fixes an issue.
  • This PR adds something new (e.g. new method or parameters).
  • This PR is a breaking change (e.g. methods or parameters removed/renamed).
  • This PR is not a code change (e.g. documentation, README, typehinting,
    examples, ...).

Checklist

  • I have searched the open pull requests for duplicates.
  • If code changes were made then they have been tested.
    • I have updated the documentation to reflect the changes.
  • If type: ignore comments were used, a comment is also left explaining why.
  • I have updated the changelog to include these changes.

Lumabots and others added 10 commits April 18, 2025 16:44
…xecutions

Added a new overlap parameter to the loop decorator and Loop class to control whether loop iterations can run concurrently. When set to True, the next iteration will not wait for the previous one to finish, allowing overlapping executions—useful for long-running tasks that should not delay subsequent runs. Defaults to False to preserve current behavior.

Signed-off-by: Lumouille <[email protected]>
Signed-off-by: Lumouille <[email protected]>
Signed-off-by: Lumouille <[email protected]>
Exception in callback Future.set_result(True)
handle: <TimerHandle when=203424.768941542 Future.set_result(True)>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/asyncio/events.py", line 84, in _run
    self._context.run(self._callback, *self._args)
asyncio.exceptions.InvalidStateError: invalid state


Signed-off-by: Lumouille <[email protected]>
Signed-off-by: Lumouille <[email protected]>
@Lumabots
Copy link
Author

sorry i renamed the branch and it deleted the other one

@Lumabots Lumabots changed the title Overlap feat(loop): add optional overlap support to allow concurrent loop executions Apr 30, 2025
Copy link
Member

@plun1331 plun1331 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably add an additional parameter for limiting the number of active tasks

Lumabots and others added 2 commits May 1, 2025 11:17
Co-authored-by: plun1331 <[email protected]>
Signed-off-by: Lumouille <[email protected]>
Co-authored-by: plun1331 <[email protected]>
Signed-off-by: Lumouille <[email protected]>
@Lumabots
Copy link
Author

Lumabots commented May 1, 2025

We should probably add an additional parameter for limiting the number of active tasks

maybe:
def init(self, ..., overlap: bool | int = True):

If overlap is:

True: allow overlapping tasks with no limit (default).

False: do not allow overlapping (wait for task to finish).

An int: allow overlapping up to n concurrent tasks.

@Lumabots
Copy link
Author

Lumabots commented May 1, 2025

so i did some change but i'll debug that tonight + if someone can tell me about my implementation in case overlap if false, should i stick with await coro ? or else its good like so

@Lumabots
Copy link
Author

Lumabots commented May 1, 2025

not ready for review

@Lumabots
Copy link
Author

Lumabots commented May 1, 2025

well my new fix breaks a bit the current_loop number, if anyone has an idea
Screenshot 2025-05-01 at 16 48 18

@Lumabots
Copy link
Author

Lumabots commented May 1, 2025

    @tasks.loop(seconds=5, overlap=2)
    async def test_loop(self):
        logger.debug(
            f"test loop {self.test_loop.current_loop}, Next loop {self.test_loop.next_iteration}, tasks {self.test_loop.get_task()}"
        )
        await asyncio.sleep(15)
        logger.debug(f"test loop done {self.test_loop.current_loop}")

@plun1331
Copy link
Member

plun1331 commented May 1, 2025

well my new fix breaks a bit the current_loop number, if anyone has an idea Screenshot 2025-05-01 at 16 48 18

probably because all the loops have the same state, so after 3 loops run they'll all thing they're on 3

Copy link
Member

@plun1331 plun1331 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, one extra thing, tasks should be removed from _tasks after they complete (otherwise the list will just grow infinitely)

@Lumabots
Copy link
Author

Lumabots commented May 1, 2025

i mean i dont really think thats an issue, since its overlapping there is no way to keep the current loop inside the same thing, and if i do make that for inside the loop the current loop doesnt update then in every other usage than the loop it itself the number will not be accurate.

well my new fix breaks a bit the current_loop number, if anyone has an idea Screenshot 2025-05-01 at 16 48 18

probably because all the loops have the same state, so after 3 loops run they'll all thing they're on 3

yep ik that, but im working on something for inside the loop we do have the same current loop and outside we have the general index. If not needed i can just let this behavour

@plun1331
Copy link
Member

plun1331 commented May 1, 2025

I think we just change the docs to specify that it represents the number of loops that have been started

@plun1331 plun1331 requested review from NeloBlivion and JustaSqu1d and removed request for NeloBlivion May 1, 2025 16:20
@Lumabots
Copy link
Author

Lumabots commented May 1, 2025

so i did that inside the loop it continue to give the same one, but if you use another command it return the real one, what do you think about that

Signed-off-by: Lumouille <[email protected]>
@plun1331
Copy link
Member

plun1331 commented May 1, 2025

if you can get that to work then that would be fine

@Lumabots
Copy link
Author

Lumabots commented May 1, 2025

Screenshot 2025-05-01 at 17 22 47 so this is the result inside the loop for @tasks.loop(seconds=5, overlap=True) async def test_loop(self): logger.debug( f"test loop {self.test_loop.current_loop}, Next loop {self.test_loop.next_iteration}, tasks {self.test_loop.get_task()}" ) await asyncio.sleep(15) logger.debug(f"test loop done {self.test_loop.current_loop}")

but inside my command if i do
await ctx.respond(f"test, {self.test_loop.current_loop}")

it will return the number of loop totals like before

@Lumabots
Copy link
Author

Lumabots commented May 1, 2025

Oh, one extra thing, tasks should be removed from _tasks after they complete (otherwise the list will just grow infinitely)

is this enough ?

task = asyncio.create_task(
    (
        self.coro(*args, **kwargs)
        if self.overlap is True
        else run_with_semaphore()
    ),
    name=f"pycord-loop-{self.coro.__name__}-{self._current_loop}",
)
task.add_done_callback(self._tasks.remove)
self._tasks.append(task)

@Lumabots
Copy link
Author

Lumabots commented May 1, 2025

should be ready for review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants