express 压缩接口响应的数据量

2021-03-04 16:44:18 +08:00
 6EyesFlyFish
express 接口假如响应的数据量达到 7Kb 的时候,在高并发压测的情况下,异常率很高,有没有什么办法压缩数据量?
响应数据如下:

{
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 22.95,
"content": "ok",
"item": [{
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 107.22,
"nick": "糖衣炮弹 54720",
"rank": "1",
"score": "180",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 37.99,
"nick": "糖衣炮弹 54717",
"rank": "2",
"score": "100",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://thirdwx.qlogo.cn/mmopen/vi_32/qDNuYJXiaUZJSlZq2XnmaBDNIIUJBlmPGNZpvbbv6NHibLzIGwzKDmzvibZTd3B0rm7I5BQ0Tcc0aLicU027L7IDibg/132",
"all_time": 60.41,
"nick": "一船星辉",
"rank": "3",
"score": "120",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 36.1,
"nick": "54723",
"rank": "4",
"score": "90",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 43.72,
"nick": "54725",
"rank": "5",
"score": "90",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 25.32,
"nick": "54719",
"rank": "6",
"score": "70",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 28.51,
"nick": "54713",
"rank": "7",
"score": "70",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 38.81,
"nick": "54714",
"rank": "8",
"score": "80",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 21.8,
"nick": "54718",
"rank": "9",
"score": "60",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 32.19,
"nick": "54726",
"rank": "10",
"score": "70",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 18.34,
"nick": "54716",
"rank": "11",
"score": "50",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 28.7,
"nick": "54724",
"rank": "12",
"score": "60",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 19.37,
"nick": "54727",
"rank": "13",
"score": "50",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 22.95,
"nick": "54715",
"rank": "14",
"score": "50",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 34.49,
"nick": "糖衣炮弹 oaAq05gIwIhVcT4ydGkqfqaodfdo11",
"rank": "15",
"score": "60",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 18.08,
"nick": "54711",
"rank": "16",
"score": "40",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 19.53,
"nick": "糖衣炮弹 54721",
"rank": "17",
"score": "40",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 9.78,
"nick": "糖衣炮弹 54722",
"rank": "18",
"score": "20",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 4.53,
"nick": "54710",
"rank": "19",
"score": "10",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 14.79,
"nick": "糖衣炮弹",
"rank": "20",
"score": "20",
"wrongcount": "0"
}, {
"HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
"all_time": 14.74,
"nick": "54712",
"rank": "21",
"score": "10",
"wrongcount": "0"
}],
"all_score": "50",
"nick": "54715",
"pNumber": "166556",
"rank": "14",
"status": 2000,
"wrongcount": "0"
}
1699 次点击
所在节点    Node.js
11 条回复
libook
2021-03-04 16:51:13 +08:00
异常的具体信息发一下,可能跟数据量大小没关系。
6EyesFlyFish
2021-03-04 17:14:01 +08:00
异常信息是连接重置
Response code:Non HTTP response code: java.net.SocketException
Response message:Non HTTP response message: Connection reset

我把数据量减少,再压测就没有异常了
6EyesFlyFish
2021-03-04 17:14:37 +08:00
libook
2021-03-04 17:18:07 +08:00
Node 上、服务器系统级别、反向代理服务器等等,是否异常日志呢?
以及服务器资源情况,负载、网络连接数、磁盘 IO 、内存使用量等等。
killergun
2021-03-04 17:25:01 +08:00
我感觉这个是不是设置下服务 response 可响应大小就行了
killergun
2021-03-04 17:25:39 +08:00
@killergun 还有就是设置响应超时时间
6EyesFlyFish
2021-03-04 17:41:07 +08:00
@libook 腾讯云服务器 8 核 16G,0 负载,8%内存使用,ulimit 设置了 100W,没有反向代理,异常日志没有,部分连接直接被重置了
6EyesFlyFish
2021-03-04 17:44:32 +08:00
@killergun express 可响应大小?在哪设置?我看了官网,似乎没有这方面的介绍
wakzz
2021-03-05 09:28:07 +08:00
建议楼主抓个包,把异常 TCP 抓到后贴出来,单单通过楼主目前提供的数据推测原因有点困难
6EyesFlyFish
2021-03-05 17:03:04 +08:00
@wakzz 搞定了,网络带宽问题,数据量太大了
wakzz
2021-03-06 14:24:29 +08:00
@6EyesFlyFish 这让我想起一次跟第三方联调时压测的网络错误问题,抓包后发现压测到某个阈值就有大量的网络超时,最后查明是对方公司的宽带不够,吐血

这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/758494

V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX