windows 下跑 Docker Nextcloud 在同步大量小文件和上传大文件出现效能问题

2019-10-28 23:05:31 +08:00
 wyxls

#680 This is relative issue but there're some differences.

运行环境:
Physical OS: Windows server 2019 Datecenter (v1809, 17763.805)
Docker: Docker for windows 2.1.0.4 (Engine: 19.03.4, Compose: 1.24.1, Hyper-V with 4 vCPUs and 4G vRAM)
MySQL: MySQL for winx64 8.0.18 Community Server
Nginx: Nginx for windows 1.16.1
Nextcloud: 17.0.0 (nextcloud:latest)

Additional Containers:
Onlyoffice-document-server: https://hub.docker.com/r/onlyoffice/documentserver, tag:latest
redis: https://hub.docker.com/_/redis, tag:latest
docker-compose yml:
version: '3'

networks:
  nextcloud:

services:

  redis:
    image: redis
    container_name: redis
    hostname: redis
    restart: always
    networks:
      - nextcloud
    expose:
      - 6379

  nextcloud:
    image: wyxls/nextcloud:full
    container_name: nextcloud
    restart: always
    depends_on:
      - redis
    environment:
      - NEXTCLOUD_TABLE_PREFIX=oc_
    volumes:
      - C:/Docker/nextcloud:/var/www/html
      - D:/Docker/nextcloud/data:/var/www/html/data
    ports:
      - 10000:80
    networks:
      - nextcloud

  onlyoffice:
    container_name: onlyoffice
    image: onlyoffice/documentserver:latest
    stdin_open: true
    tty: true
    restart: always
    depends_on:
      - nextcloud
    volumes:
      - C:/Docker/onlyoffice/document_data:/var/www/onlyoffice/Data
      - C:/Docker/onlyoffice/document_log:/var/log/onlyoffice
      - C:/Docker/onlyoffice/document_fonts:/usr/share/fonts/truetype/custom
      - C:/Docker/onlyoffice/document_forgotten:/var/lib/onlyoffice/documentserver/App_Data/cache/files/forgotten
    ports:
      - 10005:443
    networks:
      - nextcloud
nginx.conf and nextcloud.conf:
#user nobody;
worker_processes auto;
error_log  logs/error.log;
pid        logs/nginx.pid;


events {
        multi_accept on;
        worker_connections 1024;
}


http {
      include       mime.types;
      #include       naxsi_core.rules;
      default_type  application/octet-stream;

      access_log     off;
      charset        utf-8;
      #aio           threads;
      #directio      1000m;
      sendfile       on;
      tcp_nopush     on;
      tcp_nodelay    on;
      server_tokens  off;
      types_hash_max_size 2048;

      keepalive_timeout 65s;
      keepalive_requests 100;
      client_body_timeout 60s;
      client_header_timeout 60s;
      send_timeout 60s;
      reset_timedout_connection on;
      client_header_buffer_size 4k;
      large_client_header_buffers 4 16k;
      client_max_body_size 32m;
	  client_body_buffer_size 32m ;

      open_file_cache max=100000 inactive=20s;
      open_file_cache_valid    30s;
      open_file_cache_min_uses 2;
      open_file_cache_errors   on;

      fastcgi_connect_timeout 600s;
      fastcgi_send_timeout 600s;
      fastcgi_read_timeout 600s;

      gzip on;
      gzip_vary on;
      gzip_min_length 1k;
      gzip_buffers 8 32k;
      gzip_comp_level 4;
      gzip_types text/plain text/css text/xml text/x-component application/json application/javascript application/rss+xml application/xhtml+xml application/atom+xml image/svg+xml image/x-ms-bmp image/x-icon;
      gzip_disable "MSIE [1-6]\.(?!.*SV1)";

      include C:/nginx/appconf/*.conf;
}
server {
    listen      10002 ssl;
    server_name example localhost 127.0.0.1;
    root        C:/Docker/nextcloud;
    index       index.php;

    ssl_certificate       C:/SSL-Certificates/cer.cer;
    ssl_certificate_key   C:/SSL-Certificates/key.key;
    ssl_protocols SSLv2 SSLv3 TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
    ssl_prefer_server_ciphers on;

    proxy_hide_header Strict-Transport-Security;
    proxy_hide_header X-Content-Type-Options;
    proxy_hide_header X-Robots-Tag;
    proxy_hide_header X-Frame-Options;
    proxy_hide_header X-Download-Options;
    proxy_hide_header X-Permitted-Cross-Domain-Policies;
    proxy_hide_header Referrer-Policy;
    proxy_hide_header X-XSS-Protection;

    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
    add_header X-Content-Type-Options nosniff;
    add_header X-Robots-Tag "none";
    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-Download-Options "noopen";
    add_header X-Permitted-Cross-Domain-Policies "none";
    add_header Referrer-Policy "no-referrer";
    add_header X-XSS-Protection "1; mode=block";

    client_max_body_size 10G;
    fastcgi_buffers 64 4K;
    fastcgi_hide_header X-Powered-By;
    
    location / {
        proxy_pass http://127.0.0.1:10000/;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_buffer_size 64k;
        proxy_buffers 8 64k;
        proxy_busy_buffers_size 64k;
        proxy_temp_file_write_size 64k;
    }

	location = /.well-known/carddav {
        return 301 $scheme://$http_host/remote.php/dav;
    }

	location = /.well-known/caldav {
        return 301 $scheme://$http_host/remote.php/dav;
    }

    location = /robots.txt {
        allow all;
        log_not_found off;
        access_log off;
    }

    location ~ ^\/(?:build|tests|config|lib|3rdparty|templates|data)\/ {
        deny all;
    }

    location ~ ^\/(?:\.|autotest|occ|issue|indie|db_|console) {
        deny all;
    }

}
MySQL my.ini
[mysqld]
port=3306
basedir=C:\mysql-winx64
datadir=C:\mysql-winx64\Data
max_connect_errors=10
character-set-server=utf8mb4
collation-server = utf8mb4_general_ci
default-storage-engine=INNODB
default_authentication_plugin=mysql_native_password

key_buffer_size = 128M
max_allowed_packet = 32M
bulk_insert_buffer_size = 32M
thread_stack = 256K
thread_cache_size = 16
table_open_cache = 1024
innodb_buffer_pool_size = 512M
innodb_log_buffer_size = 32M

[mysql]
default-character-set=utf8

[client]
port=3306

在用 Nextcloud 的 windows 客户端同步大量小文件( office 的小文档,图片之类的),显示“检查远程变更 '文件夹名'”,每检查几个文件夹后就卡住

类似的情况发生在 web 端上传大文件的时候,上传跑满带宽每持续一段时间后就下降到 0,然后再恢复下载

上面两种情况在卡住持续 60s 左右就会超时失败,同时会有以下提示

Nginx 错误日志

upstream timed out (10060: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond) while reading response header from upstream, client: x.x.x.x, server: example.com, request: "PROPFIND /remote.php/dav/files/wyxls/filename.xlsx HTTP/1.1", upstream: " http://127.0.0.1:10000/remote.php/dav/files/wyxls/filename.xlsx", host: "example.com:10002"

Nextcloud 客户端

Error transferring https://example.com:10002/remote.php/dav/files/filename.xlsx - server replied: 504 Gateway time-out

与此同时,Nextcloud 的网页端疑似不能响应,但 Nextcloud 容器并没有崩溃


后续我已经修改过很多相关的参数:

增大 MySQL 的 buffer 的相关参数
增大 Nginx 跟反向代理 buffer 的相关参数
Nextcloud .user.ini 添加并增大上传下载和内存限制参数
……

Google 了很多内容,但能尝试的都尝试了,都没有解决问题

最神奇的是,在 Nextcloud 容器通过 compose 刚刚启动后的一段时间内是一切正常的,客户端 webdav 同步文件效率极高,大文件上传超级顺畅

软路由上的 Nextcloud 环境,除了 PHP 是 php-fpm、没用 apache、redis、onlyoffice 之外,其他硬件性能都是现在的 windows x64 主机更强


我现在真的抓狂了,个人不是程序员,Nextcloud 搭建也是边看边学边做,临时抱佛脚型,实在不知道哪里出问题了

5129 次点击
所在节点    程序员
21 条回复
wyxls
2020-05-03 10:34:23 +08:00
@NSDont
NextCloud 毕竟基于 PHP,通过上传或同步客户端处理大量文件到数据库和自身程序里是很吃力的(除非专门进行调优),官方也是知道自己的情况,所以服务端提供了一系列 occ 的指令操作,其中有一个 files:scan 是专门处理文件扫描的,你可以去服务端手册看一下

```
sudo -u www-data php occ files:scan --help
Usage:
files:scan [-p|--path="..."] [-q|--quiet] [-v|vv|vvv --verbose] [--all]
[user_id1] ... [user_idN]
Arguments:
user_id will rescan all files of the given user(s)
Options:
--path limit rescan to the user/path given
--all will rescan all files of all known users
--quiet suppress any output
--verbose files and directories being processed are shown
additionally during scanning
--unscanned scan only previously unscanned files
```

用其他手段传输好后直接扫描添加到数据库和 NextCloud 中,效率会高很多,最好的是物理手段,移动硬盘之类的。另外针对大量小文件,我建议在本地打包后放到 NextCloud 对应的路径后进行解压,再使用 occ files:scan 扫描添加

这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/613827

V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX