求问一个 python3 的 gevent。 monkey.patch_ssl() 之后运行代码会出错的问题

2017-09-19 10:32:30 +08:00
 nightstart

先上代码

import requests
import gevent
from gevent.pool import Pool
from gevent import monkey
# monkey.patch_all()
monkey.patch_socket()
monkey.patch_ssl()


contents = []
pool = Pool(20)

def req(url):
    res = requests.get(url)
    contents.append(res.status_code)
urls = [
    "http://www.baidu.com",
    "http://www.iplaysoft.com",
    "http://www.cupfox.com",
    "http://www.opbear.com",
    "http://www.bing.ren"
]

gtasks = [pool.spawn(req, url) for url in urls]
gevent.joinall(gtasks)
print(contents)

就这段代码用的是最简单的 gevent 协程请求 requests 如果只用 monkey.patch_socket()是完全没有问题的 但凡加上了 monkey.patch_ssl()就会报错 或者加上 monkey.patch_all()也会报错

Traceback (most recent call last):
File "/usr/local/python3.6/lib/python3.6/site-packages/gevent/greenlet.py", line 536, in run
result = self._run(*self.args, **self.kwargs)
File "test.py", line 14, in req
res = requests.get(url)
File "/usr/local/python3.6/lib/python3.6/site-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/python3.6/lib/python3.6/ssl.py", line 459, in options
super(SSLContext, SSLContext).options.set(self, value)
[Previous line repeated 316 more times]

 RecursionError: maximum recursion depth exceeded while calling a Python object`
 During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/python3.6/lib/python3.6/site-packages/gevent/hub.py", line 866, in switch
switch(value)
File "/usr/local/python3.6/lib/python3.6/site-packages/gevent/_tblib.py", line 425, in dump_traceback
return dumps(tb)

Tue Sep 19 10:28:11 2017 <built-in method switch of Greenlet object at 0x10c6c4f20> failed with RecursionError<br/>

Traceback (most recent call last):
File "test.py", line 25, in <module>
gevent.joinall(gtasks)
File "/usr/local/python3.6/lib/python3.6/site-packages/gevent/greenlet.py", line 649, in joinall
return wait(greenlets, timeout=timeout, count=count)
File "/usr/local/python3.6/lib/python3.6/site-packages/gevent/hub.py", line 1037, in wait
return list(iwait(objects, timeout, count))

gevent.hub.LoopExit: ('This operation would block forever',<Hub at 0x10c6c4af8 select default pending=0 ref=0 resolver=<gevent.resolver_thread.Resolver at 0x10c7905f8 pool=<ThreadPool at 0x10c790668 0/2/10>> threadpool=<ThreadPool at 0x10c790668 0/2/10>>)

今天的错误代码长这样。 因为错误代码很长很乱 我就删掉了中间的一些过程 把头尾都留下来了。 gevent 版本 1.2.2 requests 版本 2.18.4

有没有大大帮忙解释下出现错误的原因以及 monkey.patch_ssl()所做的事情

6290 次点击
所在节点    Python
10 条回复
nightstart
2017-09-19 10:49:13 +08:00
求不沉.... 这个问题困扰我好久了
NoAnyLove
2017-09-19 11:46:48 +08:00
这种问题,翻一下官方 repo 的 issue 就知道了。Python 3 下不知道什么问题造成的,解决方法是,把下面几行内容移动到最开头:

from gevent.pool import Pool
from gevent import monkey
# monkey.patch_all()
monkey.patch_socket()
monkey.patch_ssl()
nightstart
2017-09-19 14:25:17 +08:00
@NoAnyLove 可以问一下是在哪里看的吗。
qs
2017-09-19 14:34:55 +08:00
同样的问题 我本地跑会报这个错误 测试服务器跑和线上服务器跑没这个错误
官方 repo 的 issue 里说是 gevent 和 requests 的问题 gevent 版本升级到 1.21 也都是在 3.6 上出现的 可以尝试升级下 gevent
fy
2017-09-19 15:01:25 +08:00
老哥 python3 还用 gevent ? asyncio 不好吗
janxin
2017-09-19 15:04:19 +08:00
calease
2017-09-19 16:37:37 +08:00
gevent 文档里写的很清楚了一定要先 from gevent import monkey; monkey.patch_all()之后才能 import 别的 library。
http://www.gevent.org/intro.html#monkey-patching

When monkey patching, it is recommended to do so as early as possible in the lifetime of the process. If possible, monkey patching should be the first lines executed.
NoAnyLove
2017-09-19 22:15:10 +08:00
nightstart
2017-09-21 13:54:04 +08:00
@qs 我这边也发生了这个情况 我的自己上跑有问题 我小伙伴上的就没问题
nightstart
2017-09-21 13:55:58 +08:00
@fy gevent 简直小白必备 之前去看了下 asyncio 发现哇 gevent 还是好简单于是就用回来了

这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/391886

V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX