慎用 AWS 中国 S3 Glacier

2018-04-16 18:27:51 +08:00
 AntonChen

可能是个例,把遇到的情况发出来仅供参考

我厂在使用 AWS S3 Glacier 存储 Elasticsearch 快照,今天需要恢复大约 20 天的索引(按天建立的索引),总量约 1GB。

然而 Glacier 的解冻只能对每个对象进行解冻,不能使用通配符去解冻,当然可以使用 AWS cli 或 s3cmd 等工具来进行递归解冻。

问题就出在递归解冻上,每一个 Elasticsearch 索引快照会包含 100+ 的对象,所以每解冻一个快照索引都需要请求 100+ 次以上。

然而 AWS 中国 S3 Glacier 在请求 40+ 对象后就报错 503,尝试多次都是如此。

restore: 's3://BUCKET/index_20180401/1/__h'
restore: 's3://BUCKET/index_20180401/1/__i'
WARNING: Retrying failed request: /indices/index__20180401/1/__j?restore (503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later)
WARNING: Waiting 3 sec...
restore: 's3://BUCKET/index_20180401/1/__j'
WARNING: Retrying failed request: /indices/index__20180401/1/__k?restore (503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later)
WARNING: Waiting 3 sec...
WARNING: Retrying failed request: /indices/index__20180401/1/__k?restore (503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later)
WARNING: Waiting 6 sec...
WARNING: Retrying failed request: /indices/index__20180401/1/__k?restore (503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later)
WARNING: Waiting 9 sec...
WARNING: Retrying failed request: /indices/index__20180401/1/__k?restore (503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later)
WARNING: Waiting 12 sec...
WARNING: Retrying failed request: /indices/index__20180401/1/__k?restore (503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later)
WARNING: Waiting 15 sec...
ERROR: S3 error: 503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later

40+ 请求时间在 2 秒左右完成,大约 1 秒 20 个请求,如此大型的服务频率限制如此低?

目前还在联系 AWS 中国,询问具体原因,后续会附言更新

5178 次点击
所在节点    Amazon Web Services
3 条回复
fredcc
2018-04-16 18:55:42 +08:00
AntonChen
2018-04-16 19:36:55 +08:00
@fredcc 你是指使用 「批量取回」 么?
fredcc
2018-04-17 18:58:22 +08:00

这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/447277

V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX