How to enable concurrency translatation or batch for customTranslate?

[poor-circle]

I‘m using a custom LLM translator and found the translatation is very slow because there are many text translating one by one. Once translation's latency is about 3-4s.

How can I enable concurrency translatation? my config is:

[Custom]
Url=http://localhost:6800
EnableShortDelay=True
DisableSpamChecks=True

[poor-circle]

The readme said: "It will never send out more than 1 request at a time (no concurrency!)."
So there is no concurrency even if i disable spam checks?

[poor-circle]

It seems that we should overload MaxTranslationsPerRequest and MaxConcurrency in CustomTranslateEndpoint.cs and load it from config？