[BUG] ERROR 无效响应类型。响应类型: <class 'NoneType'> #600
Comments
Dragon617
changed the title
|
使用的是最新的版本吗 |
|
请问推送dockerhub 了吗？辛苦大大 |
|
I'm having the same issue right now, problem only arose after several thousands of requests. I am working on a research project, could it be i went over some limit, if so is there any solution? here is a sample of my logs: 2025-03-25 15:54:38 INFO: 172.17.0.1:51082 - "GET /api/douyin/web/fetch_video_comment_replies?item_id=7480024022340717862&comment_id=7482436070828065555&cursor=0&count=2 HTTP/1.1" 200 OK |
我也是相同的问题 |
I'm sorry I don't speak chinese so i just machine translated your comment. I am helping my friends girlfriend with her research project on douyin videos, they are both chinese nationals but bad at tech so they enlisted my help. so im collecting a lot of data, i've realized that the problem begins once a day after processing video, comment, and reply user data for about 25 videos. The problem goes away the next day, making me assume its a quota on the upstream ai that actually fetches the data from douyin, i have a feeling that maybe by generating new ttwid, verify_fp, s_v_web_id, x_bogus, and a_bogus. we might be able to feed that into the docker config and restart to reset quota but im not sure and the english documentation is somewhat lacking. if you would like to try and post results, that would be cool. Please contact me if you find out anything else |
|
I did some more research, and it would seem that I got blocked from the upstream douyin server at the 1010th call for a specific endpoint. |
|
I tested my method of replacing ttwid and s_v_web_id in dockerContainer/app/crawlers/douyin/web.config.yaml once requests are done and it fixed my issue, im not sure if this is a guaranteed solution, but it has worked for me at least now. |
|
我测试了三四个 cookie，请求单个 video 的解析，每个 cookie 都是大概 200 个请求就返回为空，所以应该是有请求上限。 |
I did further testing and it would seem that the technique I outline works to some extent but is inconsistent. Waiting it out works well. I consistently got ~1000 requests before running out as for the ttwid and s_v_web_id yes you can generate them their are endpoints in the API dedicated to this. However if you want to see a full implementation refer to this file: https://github.com/schiffinor/Yanjun/blob/master/DouyinScraper%2FFileFetcher.py Any function here that uses my get_with_retry function: https://github.com/schiffinor/Yanjun/blob/master/DouyinScraper%2FFileFetcher.py#L568 Demonstrates the full implementation but I specifically have a fetch_ttwid and fetch_s_v_web_id function. Have in mind I wrote this as a quick project to help my friends girlfriend with her masters' thesis. So the code is a little unorganized and documentation is ai generated lol, I wrote all code and it works if you'd like to reference or use it though. |
|
我在本地运行可以拿到数据，部署到腾讯云服务器上就遇到了和你一样的问题，而且更换了cookies也没有用🥲 |
发生错误的平台？
如：抖音
提交的输入值？
短视频链接：https://v.douyin.com/i5Tbu5eN/ 08/02 icn:/ [email protected]
是否有再次尝试？
如：是，发生错误后在2025年3月23日21:38:32后错误依旧存在。
错误代码
2025-03-23 21:35:21 INFO: 172.17.0.1:37520 - "GET /api/hybrid/video_data?url=https://v.douyin.com/XIIyVaTM0vQ&minimal=False HTTP/1.1" 400 Bad Request
2025-03-23 21:35:59 WARNING 第 1 次响应内容为空, 状态码: 200,
2025-03-23 21:35:59 URL:https://www.douyin.com/aweme/v1/web/aweme/detail/?device_platform=w
2025-03-23 21:35:59 ebapp&aid=6383&channel=channel_pc_web&pc_client_type=1&version_code=290
2025-03-23 21:35:59 100&version_name=29.1.0&cookie_enabled=true&screen_width=1920&screen_he
2025-03-23 21:35:59 ight=1080&browser_language=zh-CN&browser_platform=Win32&browser_name=Ch
2025-03-23 21:35:59 rome&browser_version=130.0.0.0&browser_online=true&engine_name=Blink&en
2025-03-23 21:35:59 gine_version=130.0.0.0&os_name=Windows&os_version=10&cpu_core_num=12&de
2025-03-23 21:35:59 vice_memory=8&platform=PC&downlink=10&effective_type=4g&from_user_page=
2025-03-23 21:35:59 1&locate_query=false&need_time_list=1&pc_libra_divert=Windows&publish_v
2025-03-23 21:35:59 ideo_strategy_type=2&round_trip_time=0&show_live_replay_strategy=1&time
2025-03-23 21:35:59 _list_query=0&whale_cut_token=&update_version_code=170400&msToken=&awem
2025-03-23 21:35:59 e_id=7476519315408178495&a_bogus=YvmhBmLhDi6shD6X56ALfY3q6l8VYmsg0SVkMD
2025-03-23 21:35:59 2fqBDO1L39HMP-9exoX2JvwwRjx4%2FhIeEjy4hbT3ohrQ2y0Hwf9W0L%2F25ksDSkKl5Q5
2025-03-23 21:35:59 xSSs1X9eghgJ04qmkt5SMx2RvB-rOXmqhZHKRbp09oHmhK4b1dzFgf3qJLz5f%3D%3D
2025-03-23 21:36:09 WARNING 第 2 次响应内容为空, 状态码: 200,
2025-03-23 21:36:09 URL:https://www.douyin.com/aweme/v1/web/aweme/detail/?device_platform=w
2025-03-23 21:36:09 ebapp&aid=6383&channel=channel_pc_web&pc_client_type=1&version_code=290
2025-03-23 21:36:09 100&version_name=29.1.0&cookie_enabled=true&screen_width=1920&screen_he
2025-03-23 21:36:09 ight=1080&browser_language=zh-CN&browser_platform=Win32&browser_name=Ch
2025-03-23 21:36:09 rome&browser_version=130.0.0.0&browser_online=true&engine_name=Blink&en
2025-03-23 21:36:09 gine_version=130.0.0.0&os_name=Windows&os_version=10&cpu_core_num=12&de
2025-03-23 21:36:09 vice_memory=8&platform=PC&downlink=10&effective_type=4g&from_user_page=
2025-03-23 21:36:09 1&locate_query=false&need_time_list=1&pc_libra_divert=Windows&publish_v
2025-03-23 21:36:09 ideo_strategy_type=2&round_trip_time=0&show_live_replay_strategy=1&time
2025-03-23 21:36:09 _list_query=0&whale_cut_token=&update_version_code=170400&msToken=&awem
2025-03-23 21:36:09 e_id=7476519315408178495&a_bogus=YvmhBmLhDi6shD6X56ALfY3q6l8VYmsg0SVkMD
2025-03-23 21:36:09 2fqBDO1L39HMP-9exoX2JvwwRjx4%2FhIeEjy4hbT3ohrQ2y0Hwf9W0L%2F25ksDSkKl5Q5
2025-03-23 21:36:09 xSSs1X9eghgJ04qmkt5SMx2RvB-rOXmqhZHKRbp09oHmhK4b1dzFgf3qJLz5f%3D%3D
2025-03-23 21:36:19 WARNING 第 3 次响应内容为空, 状态码: 200,
2025-03-23 21:36:19 URL:https://www.douyin.com/aweme/v1/web/aweme/detail/?device_platform=w
2025-03-23 21:36:19 ebapp&aid=6383&channel=channel_pc_web&pc_client_type=1&version_code=290
2025-03-23 21:36:19 100&version_name=29.1.0&cookie_enabled=true&screen_width=1920&screen_he
2025-03-23 21:36:19 ight=1080&browser_language=zh-CN&browser_platform=Win32&browser_name=Ch
2025-03-23 21:36:19 rome&browser_version=130.0.0.0&browser_online=true&engine_name=Blink&en
2025-03-23 21:36:19 gine_version=130.0.0.0&os_name=Windows&os_version=10&cpu_core_num=12&de
2025-03-23 21:36:19 vice_memory=8&platform=PC&downlink=10&effective_type=4g&from_user_page=
2025-03-23 21:36:19 1&locate_query=false&need_time_list=1&pc_libra_divert=Windows&publish_v
2025-03-23 21:36:19 ideo_strategy_type=2&round_trip_time=0&show_live_replay_strategy=1&time
2025-03-23 21:36:19 _list_query=0&whale_cut_token=&update_version_code=170400&msToken=&awem
2025-03-23 21:36:19 e_id=7476519315408178495&a_bogus=YvmhBmLhDi6shD6X56ALfY3q6l8VYmsg0SVkMD
2025-03-23 21:36:19 2fqBDO1L39HMP-9exoX2JvwwRjx4%2FhIeEjy4hbT3ohrQ2y0Hwf9W0L%2F25ksDSkKl5Q5
2025-03-23 21:36:19 xSSs1X9eghgJ04qmkt5SMx2RvB-rOXmqhZHKRbp09oHmhK4b1dzFgf3qJLz5f%3D%3D
2025-03-23 21:36:19 程序出现异常，请检查错误信息。
2025-03-23 21:36:19 ERROR 无效响应类型。响应类型: <class 'NoneType'>
2025-03-23 21:36:19 程序出现异常，请检查错误信息。
2025-03-23 21:36:19 INFO: 172.17.0.1:44478 - "GET /api/hybrid/video_data?url=https://v.douyin.com/i5Tbu5eN&minimal=False HTTP/1.1" 400 Bad Request
在AstrBot平台是这个提示
[21:35:59] [Core] [INFO] [core.event_bus:21]: [aiocqhttp] Dragon0617/1534676849: 2.87 复制打开抖音，看看【耀哥说AI的作 品】万物开花2.0视频教程 答应大家的万物开花2.0版... https://v.douyin.com/i5Tbu5eN/ 08/02 icn:/ [email protected]
Error fetching video data: 400, message='Bad Request', url='http://localhost/api/hybrid/video_data?url=https://v.douyin.com/i5Tbu5eN&minimal=False'
解析失败，请检查链接是否正确。
[21:36:20] [Core] [INFO] [respond.stage:110]: AstrBot -> Dragon0617/1534676849: 检测到抖音链接，但解析失败，请检查链接 是否正确。
The text was updated successfully, but these errors were encountered: