You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
By default `use()` will return a function instance with a sync interface. You can pass `use_async=True` to have it return an `AsyncFunction` that provides an async interface.
outputs =await flux_dev(prompt="a cat wearing an amusing hat")
642
+
643
+
for output in outputs:
644
+
print(Path(output))
645
+
646
+
asyncio.run(main())
647
+
```
648
+
649
+
If the model returns an iterator an `AsyncIterator` implementation will be used:
650
+
651
+
```py
652
+
import asyncio
653
+
import replicate
654
+
655
+
asyncdefmain():
656
+
claude = replicate.use("anthropic/claude-3.5-haiku", use_async=True)
657
+
output =await claude(prompt="say hello")
658
+
659
+
# Stream the response as it comes in.
660
+
asyncfor token in output:
661
+
print(token)
662
+
663
+
# Wait until model has completed. This will return either a `list` or a `str` depending
664
+
# on whether the model uses AsyncIterator or ConcatenateAsyncIterator. You can check this
665
+
# on the model schema by looking for `x-cog-display: concatenate`.
666
+
print(await output)
667
+
668
+
asyncio.run(main())
669
+
```
670
+
631
671
### Typing
632
672
633
673
By default `use()` knows nothing about the interface of the model. To provide a better developer experience we provide two methods to add type annotations to the function returned by the `use()` helper.
@@ -666,11 +706,10 @@ In future we hope to provide tooling to generate and provide these models as pac
666
706
667
707
There are several key things still outstanding:
668
708
669
-
1. Support for asyncio.
670
-
2. Support for streaming text when available (rather than polling)
671
-
3. Support for streaming files when available (rather than polling)
672
-
4. Support for cleaning up downloaded files.
673
-
5. Support for streaming logs using `OutputIterator`.
709
+
1. Support for streaming text when available (rather than polling)
710
+
2. Support for streaming files when available (rather than polling)
711
+
3. Support for cleaning up downloaded files.
712
+
4. Support for streaming logs using `OutputIterator`.
0 commit comments