-
-
Notifications
You must be signed in to change notification settings - Fork 364
rdagent fin_factor error #788
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
The same problem |
2 similar comments
The same problem |
The same problem |
@TPLin22 @WinstonLiyt @SunsetWolf please fix this issue |
update: maybe it's because the default embedding model is also openai, i'm trying to replace it with a free model, will update soon |
The problem is that the response text returned by |
yes the output by deepseek is a dict list, so before json.loads, it needs some fix, also another problem is the return of a boolean value, deepseek default returns like a True or False which should be replaced to lower case, just wonder why the official repo didn't add the compatibility if the config supports deepseek |
so convert the dict list in deepseek to json dict in rdagent, or conversely? |
🐛 Bug Description
To Reproduce
Steps to reproduce the behavior:
run: rdagent fin_factor
my .env file like this:
BACKEND=rdagent.oai.backend.LiteLLMAPIBackend
CHAT_MODEL=deepseek/deepseek-chat
DEEPSEEK_API_KEY=sk-
using deepseek as backend model
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
2025-04-14 12:12:27.066 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:342 - litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
2025-04-14 12:12:27.068 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:343 - Retrying 10th time...
Implementing: 10%|██████████████▋ | 1/10 [04:19<38:58, 259.78s/it]
Workflow Progress: 50%|███████████████████████████████████████████████▌ | 2/4 [09:44<09:44, 292.05s/step, loop_index=0, step_index=2, step_name=coding]
Traceback (most recent call last):
File "/home/roger/anaconda3/envs/rdagent/bin/rdagent", line 8, in
sys.exit(app())
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/app/cli.py", line 48, in app
fire.Fire(
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/fire/core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/fire/core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/app/qlib_rd_loop/factor.py", line 43, in main
model_loop.run(step_n=step_n)
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/utils/workflow.py", line 123, in run
self.loop_prev_out[name] = func(self.loop_prev_out)
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/components/workflow/rd_loop.py", line 67, in coding
exp = self.coder.develop(prev_out["direct_exp_gen"]["exp_gen"])
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/components/coder/factor_coder/init.py", line 27, in develop
exp = super().develop(exp)
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/components/coder/CoSTEER/init.py", line 102, in develop
for evo_exp in self.evolve_agent.multistep_evolve(evo_exp, self.evaluator):
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/core/evolving_agent.py", line 75, in multistep_evolve
self.rag.generate_knowledge(self.evolving_trace)
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/components/coder/CoSTEER/knowledge_management.py", line 265, in generate_knowledge
self.knowledgebase.update_success_task(
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/components/coder/CoSTEER/knowledge_management.py", line 758, in update_success_task
self.graph.add_nodes(
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/components/knowledge_management/graph.py", line 165, in add_nodes
self.add_node(node)
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/components/knowledge_management/graph.py", line 142, in add_node
node.create_embedding()
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/components/knowledge_management/vector_base.py", line 48, in create_embedding
self.embedding = APIBackend().create_embedding(input_content=self.content)
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/oai/backend/base.py", line 289, in create_embedding
resp = self._try_create_chat_completion_or_embedding( # type: ignore[misc]
File "/home/roger/anaconda3/envs/rdagent/lib/python3.10/site-packages/rdagent/oai/backend/base.py", line 345, in _try_create_chat_completion_or_embedding
raise RuntimeError(error_message)
RuntimeError: Failed to create chat completion after 10 retries.
3.
Expected Behavior
Screenshot
Environment
rdagent collect_info
2025-04-14 13:01:24.557 | INFO | rdagent.oai.backend.litellm::27 - backend='rdagent.oai.backend.LiteLLMAPIBackend' chat_model='deepseek/deepseek-chat' embedding_model='text-embedding-3-small' log_llm_chat_content=True use_azure=False chat_use_azure=False embedding_use_azure=False chat_use_azure_token_provider=False embedding_use_azure_token_provider=False managed_identity_client_id=None max_retry=10 retry_wait_seconds=1 dump_chat_cache=False use_chat_cache=False dump_embedding_cache=False use_embedding_cache=False prompt_cache_path='/home/roger/rdagent/prompt_cache.db' max_past_message_include=10 use_auto_chat_cache_seed_gen=False init_chat_cache_seed=42 openai_api_key='' chat_openai_api_key=None chat_openai_base_url=None chat_azure_api_base='' chat_azure_api_version='' chat_max_tokens=None chat_temperature=0.5 chat_stream=True chat_seed=None chat_frequency_penalty=0.0 chat_presence_penalty=0.0 chat_token_limit=100000 default_system_prompt="You are an AI assistant who helps to answer user's questions." system_prompt_role='system' embedding_openai_api_key='' embedding_openai_base_url='' embedding_azure_api_base='' embedding_azure_api_version='' embedding_max_str_num=50 use_llama2=False llama2_ckpt_dir='Llama-2-7b-chat' llama2_tokenizer_path='Llama-2-7b-chat/tokenizer.model' llams2_max_batch_size=8 use_gcr_endpoint=False gcr_endpoint_type='llama2_70b' llama2_70b_endpoint='' llama2_70b_endpoint_key='' llama2_70b_endpoint_deployment='' llama3_70b_endpoint='' llama3_70b_endpoint_key='' llama3_70b_endpoint_deployment='' phi2_endpoint='' phi2_endpoint_key='' phi2_endpoint_deployment='' phi3_4k_endpoint='' phi3_4k_endpoint_key='' phi3_4k_endpoint_deployment='' phi3_128k_endpoint='' phi3_128k_endpoint_key='' phi3_128k_endpoint_deployment='' gcr_endpoint_temperature=0.7 gcr_endpoint_top_p=0.9 gcr_endpoint_do_sample=False gcr_endpoint_max_token=100 chat_use_azure_deepseek=False chat_azure_deepseek_endpoint='' chat_azure_deepseek_key='' chat_model_map='{}'
2025-04-14 13:01:25.486 | INFO | rdagent.app.utils.info:sys_info:22 - Name of current operating system: Linux
2025-04-14 13:01:25.488 | INFO | rdagent.app.utils.info:sys_info:22 - Processor architecture: x86_64
2025-04-14 13:01:25.489 | INFO | rdagent.app.utils.info:sys_info:22 - System, version, and hardware information: Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.35
2025-04-14 13:01:25.491 | INFO | rdagent.app.utils.info:sys_info:22 - Version number of the system: #1 SMP Tue Nov 5 00:21:55 UTC 2024
2025-04-14 13:01:25.492 | INFO | rdagent.app.utils.info:python_info:29 - Python version: 3.10.16 (main, Dec 11 2024, 16:24:50) [GCC 11.2.0]
2025-04-14 13:01:25.545 | INFO | rdagent.app.utils.info:docker_info:39 - Container ID: 3a085373da6a8fea2e17407502b385a81d65893c50aca7c43c933599a3cea556
2025-04-14 13:01:25.546 | INFO | rdagent.app.utils.info:docker_info:40 - Container Name: epic_gates
2025-04-14 13:01:25.548 | INFO | rdagent.app.utils.info:docker_info:41 - Container Status: created
2025-04-14 13:01:25.551 | INFO | rdagent.app.utils.info:docker_info:42 - Image ID used by the container: sha256:14f2ad155813bfa4df863a8fcbdee95a6e23d111ef0d652a2c953368bcb7a96d
2025-04-14 13:01:25.554 | INFO | rdagent.app.utils.info:docker_info:43 - Image tag used by the container: ['local_qlib:latest']
2025-04-14 13:01:25.555 | INFO | rdagent.app.utils.info:docker_info:44 - Container port mapping: {}
2025-04-14 13:01:25.557 | INFO | rdagent.app.utils.info:docker_info:45 - Container Label: {'com.nvidia.volumes.needed': 'nvidia_driver', 'org.opencontainers.image.ref.name': 'ubuntu', 'org.opencontainers.image.version': '22.04'}
2025-04-14 13:01:25.559 | INFO | rdagent.app.utils.info:docker_info:46 - Startup Commands: nvidia-smi
2025-04-14 13:01:25.562 | INFO | rdagent.app.utils.info:rdagent_info:54 - RD-Agent version: 0.4.0
2025-04-14 13:01:27.182 | INFO | rdagent.app.utils.info:rdagent_info:76 - Package version: ['pydantic-settings==2.8.1', 'python-Levenshtein==0.27.1', 'scikit-learn==1.6.1', 'filelock==3.18.0', 'loguru==0.7.3', 'fire==0.7.0', 'fuzzywuzzy==0.18.0', 'openai==1.73.0', 'litellm==1.66.0', 'azure.identity==1.21.0', 'pyarrow==19.0.1', 'rich==14.0.0', 'tqdm==4.67.1', 'numpy==2.2.4', 'pandas==2.2.3', 'pandarallel==1.6.5', 'matplotlib==3.10.1', 'langchain==0.3.23', 'langchain-community==0.3.21', 'tiktoken==0.9.0', 'pymupdf==1.25.5', 'pypdf==5.4.0', 'azure-ai-formrecognizer==3.3.3', 'tables==3.10.1', 'tree-sitter-python==0.23.6', 'tree-sitter==0.24.0', 'python-dotenv==1.1.0', 'docker==7.1.0', 'streamlit==1.44.1', 'plotly==6.0.1', 'st-theme==1.2.3', 'selenium==4.31.0', 'kaggle==1.7.4.2', 'nbformat==5.10.4', 'setuptools-scm==8.2.0', 'seaborn==0.13.2', 'azure.ai.inference==1.0.0b9', 'humanize==4.12.2', 'genson==1.3.0']
Additional Notes
and also many others met this same issue in the wechat group, so guess it's a bug, please fix it
ADDED:
2025-04-14 13:21:00.480 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:342 - litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
2025-04-14 13:21:00.483 | WARNING | rdagent.oai.backend.base:try_create_chat_completion_or_embedding:343 - Retrying 1th time...
2025-04-14 13:21:00.485 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:57 - Using emb model text-embedding-3-small
2025-04-14 13:21:00.486 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:58 - Creating embedding for: factor_name: Momentum_5D
factor_description: The 5-day price momentum factor calculated as the percentage change in closing price over the past 5 trading days.
factor_formulation: \frac{close_t - close{t-5}}{close{t-5}}
variables: {'close_t': 'Closing price at day t', 'close{t-5}': 'Closing price 5 days before day t'}
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
2025-04-14 13:21:01.504 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:342 - litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
2025-04-14 13:21:01.507 | WARNING | rdagent.oai.backend.base:try_create_chat_completion_or_embedding:343 - Retrying 2th time...
2025-04-14 13:21:01.509 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:57 - Using emb model text-embedding-3-small
2025-04-14 13:21:01.511 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:58 - Creating embedding for: factor_name: Momentum_5D
factor_description: The 5-day price momentum factor calculated as the percentage change in closing price over the past 5 trading days.
factor_formulation: \frac{close_t - close{t-5}}{close{t-5}}
variables: {'close_t': 'Closing price at day t', 'close{t-5}': 'Closing price 5 days before day t'}
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
2025-04-14 13:21:02.531 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:342 - litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
2025-04-14 13:21:02.533 | WARNING | rdagent.oai.backend.base:try_create_chat_completion_or_embedding:343 - Retrying 3th time...
2025-04-14 13:21:02.535 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:57 - Using emb model text-embedding-3-small
2025-04-14 13:21:02.537 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:58 - Creating embedding for: factor_name: Momentum_5D
factor_description: The 5-day price momentum factor calculated as the percentage change in closing price over the past 5 trading days.
factor_formulation: \frac{close_t - close{t-5}}{close{t-5}}
variables: {'close_t': 'Closing price at day t', 'close{t-5}': 'Closing price 5 days before day t'}
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
2025-04-14 13:21:03.560 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:342 - litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
2025-04-14 13:21:03.562 | WARNING | rdagent.oai.backend.base:try_create_chat_completion_or_embedding:343 - Retrying 4th time...
2025-04-14 13:21:03.563 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:57 - Using emb model text-embedding-3-small
2025-04-14 13:21:03.565 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:58 - Creating embedding for: factor_name: Momentum_5D
factor_description: The 5-day price momentum factor calculated as the percentage change in closing price over the past 5 trading days.
factor_formulation: \frac{close_t - close{t-5}}{close{t-5}}
variables: {'close_t': 'Closing price at day t', 'close{t-5}': 'Closing price 5 days before day t'}
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
2025-04-14 13:21:04.585 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:342 - litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
2025-04-14 13:21:04.587 | WARNING | rdagent.oai.backend.base:try_create_chat_completion_or_embedding:343 - Retrying 5th time...
2025-04-14 13:21:04.590 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:57 - Using emb model text-embedding-3-small
2025-04-14 13:21:04.592 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:58 - Creating embedding for: factor_name: Momentum_5D
factor_description: The 5-day price momentum factor calculated as the percentage change in closing price over the past 5 trading days.
factor_formulation: \frac{close_t - close{t-5}}{close{t-5}}
variables: {'close_t': 'Closing price at day t', 'close{t-5}': 'Closing price 5 days before day t'}
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
2025-04-14 13:21:05.609 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:342 - litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
2025-04-14 13:21:05.612 | WARNING | rdagent.oai.backend.base:try_create_chat_completion_or_embedding:343 - Retrying 6th time...
2025-04-14 13:21:05.613 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:57 - Using emb model text-embedding-3-small
2025-04-14 13:21:05.615 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:58 - Creating embedding for: factor_name: Momentum_5D
factor_description: The 5-day price momentum factor calculated as the percentage change in closing price over the past 5 trading days.
factor_formulation: \frac{close_t - close{t-5}}{close{t-5}}
variables: {'close_t': 'Closing price at day t', 'close{t-5}': 'Closing price 5 days before day t'}
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
2025-04-14 13:21:06.632 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:342 - litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
2025-04-14 13:21:06.633 | WARNING | rdagent.oai.backend.base:try_create_chat_completion_or_embedding:343 - Retrying 7th time...
2025-04-14 13:21:06.635 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:57 - Using emb model text-embedding-3-small
2025-04-14 13:21:06.637 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:58 - Creating embedding for: factor_name: Momentum_5D
factor_description: The 5-day price momentum factor calculated as the percentage change in closing price over the past 5 trading days.
factor_formulation: \frac{close_t - close{t-5}}{close{t-5}}
variables: {'close_t': 'Closing price at day t', 'close{t-5}': 'Closing price 5 days before day t'}
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
2025-04-14 13:21:07.653 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:342 - litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
2025-04-14 13:21:07.655 | WARNING | rdagent.oai.backend.base:try_create_chat_completion_or_embedding:343 - Retrying 8th time...
2025-04-14 13:21:07.657 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:57 - Using emb model text-embedding-3-small
2025-04-14 13:21:07.659 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:58 - Creating embedding for: factor_name: Momentum_5D
factor_description: The 5-day price momentum factor calculated as the percentage change in closing price over the past 5 trading days.
factor_formulation: \frac{close_t - close{t-5}}{close{t-5}}
variables: {'close_t': 'Closing price at day t', 'close{t-5}': 'Closing price 5 days before day t'}
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
2025-04-14 13:21:08.676 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:342 - litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
2025-04-14 13:21:08.679 | WARNING | rdagent.oai.backend.base:try_create_chat_completion_or_embedding:343 - Retrying 9th time...
2025-04-14 13:21:08.680 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:57 - Using emb model text-embedding-3-small
2025-04-14 13:21:08.682 | INFO | rdagent.oai.backend.litellm:create_embedding_inner_function:58 - Creating embedding for: factor_name: Momentum_5D
factor_description: The 5-day price momentum factor calculated as the percentage change in closing price over the past 5 trading days.
factor_formulation: \frac{close_t - close{t-5}}{close{t-5}}
variables: {'close_t': 'Closing price at day t', 'close{t-5}': 'Closing price 5 days before day t'}
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
2025-04-14 13:21:09.699 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:342 - litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
2025-04-14 13:21:09.703 | WARNING | rdagent.oai.backend.base:_try_create_chat_completion_or_embedding:343 - Retrying 10th time...
before the trace, there are these outputs, I use deepseek not openai, why gets an openai exception
The text was updated successfully, but these errors were encountered: