请教使用 gunicorn 部署 flask 时遇到的 redis 连接数飙升问题

2018-01-08 11:27:51 +08:00
 lywane
小弟写了个简单的 flask 接口,其中使用了 redis,连接 redis 的语句写在全局里了,如下:
cache = redis.StrictRedis('127.0.0.1', 6379)

然后使用 gunicorn 部署,部署命令是:
gunicorn --workers=9 --worker-class=gevent t_fl:app -b 0.0.0.0:5002

在高并发情况下(并发两千至五千)执行十万次请求,使用 info clients 命令查看 redis 的客户端连接数飙升到 3000 左右,严重拖慢了查询速度。请问有什么解决办法。
3158 次点击
所在节点    Python
4 条回复
myyou
2018-01-08 11:58:12 +08:00
使用 Redis 连接池 ConnectionPool 设置最大连接数量
lywane
2018-01-08 12:32:03 +08:00
@myyou
如下声明之后,
rdp = redis.ConnectionPool(host='127.0.0.1', port=6379,max_connections=10)
cache = redis.StrictRedis(connection_pool=rdp)

请求过来时会报错:
[2018-01-08 12:06:10,117] ERROR in app: Exception on / [POST]
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/opt/t_fl.py", line 54, in index
timeout, endtime = get_timeout_endtime(GID)
File "/opt/t_fl.py", line 20, in get_timeout_endtime
timeout_endtime = cache.get(gid)
File "/usr/lib/python2.7/site-packages/redis/client.py", line 976, in get
return self.execute_command('GET', name)
File "/usr/lib/python2.7/site-packages/redis/client.py", line 665, in execute_command
connection = pool.get_connection(command_name, **options)
File "/usr/lib/python2.7/site-packages/redis/connection.py", line 965, in get_connection
connection = self.make_connection()
File "/usr/lib/python2.7/site-packages/redis/connection.py", line 981, in make_connection
raise ConnectionError("Too many connections")
myyou
2018-01-08 14:54:08 +08:00
gunicorn 使用 gevent 时--worker-connections 默认 1000,你可以把这个值设置小一些
liuxingpu
2018-06-05 15:49:56 +08:00
我也碰到这个问题,请问楼主解决了没?

这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/420984

V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX