基于 docker swarm 搭建跨主机的 mongodb 副本集遇到的坑,请大牛赐教

2019-11-18 14:26:01 +08:00
 just4id
问题如下:
创建了两个 vm,主机名分别是: 'RedHat'和'gen8-arch-linux',用 docker stack 命令在两个 docker node 上起了两个 mongodb 的容器,并已加入到 docker overlay 的网络。但进入到其中任何一个 mongodb 的容器,初始化 replica set 时,连通不了另一个容器里的 mongodb 服务,故组 rs 不成功。经本人查看两个容器的 overlay 网络 IP,网络应该是通的,详情如下:

管理结点:
[jacky@gen8-arch-linux ~]$ cat mongo.yml
version: '3.4'

services:

  primary:
    image: mongo:3.4
    hostname: "{{.Service.Name}}"
    command: mongod --port 27017 --bind_ip=0.0.0.0 --replSet mongo --dbpath /data/db
    networks:
      - es
    volumes:
      - /home/jacky/data/mongo-data:/data/db
    deploy:
      restart_policy:
        condition: on-failure
      replicas: 1
      placement:
        constraints:
          - node.hostname==gen8-arch-linux
      endpoint_mode: dnsrr

  secondary:
    image: mongo:3.4
    hostname: "{{.Service.Name}}"
    command: mongod --port 27017 --bind_ip=0.0.0.0 --replSet mongo --dbpath /data/db
    networks:
      - es
    volumes:
      - /home/oracle/data/mongo-data:/data/db
    deploy:
      restart_policy:
        condition: on-failure
      replicas: 1
      placement:
        constraints:
          - node.hostname==RedHat
      endpoint_mode: dnsrr

networks:
  es:
    external: true
[jacky@gen8-arch-linux ~]$ docker stack deploy -c mongo.yml mongo
Creating service mongo_secondary
Creating service mongo_primary
[jacky@gen8-arch-linux ~]$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
vt82wfc13t831n2wcltm3d3lz     RedHat              Ready               Active                                  18.06.0-ce
xazp9k3n4axfb2n8emdrxl8ba *   gen8-arch-linux     Ready               Active              Leader              19.03.2-ce
[jacky@gen8-arch-linux ~]$ docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
mjfaxe0h7bms        mongo_primary       replicated          1/1                 mongo:3.4
bcs50kfp6hd2        mongo_secondary     replicated          1/1                 mongo:3.4
[jacky@gen8-arch-linux ~]$ docker network ls | grep es
se7u7jxai61l        es                  overlay             swarm
[jacky@gen8-arch-linux ~]$ docker ps -a | grep mongo
97d200f88726        mongo:3.4           "docker-entrypoint.s…"   8 minutes ago       Up 8 minutes        27017/tcp                mongo_primary.1.699xh09shmdlyu7dpd43yzsy8
[jacky@gen8-arch-linux ~]$ docker inspect 97 | grep v4
                        "IPv4Address": "10.0.0.4"

工作结点:
[oracle@RedHat ~]$ docker ps -a | grep 3.4
227dac4e143b        mongo:3.4                           "docker-entrypoint.s…"   12 minutes ago      Up 11 minutes             27017/tcp           mongo_secondary.1.mk3jiksavbmh5qch1yx4p2mht
[oracle@RedHat ~]$ docker inspect 227 | grep v4
                        "IPv4Address": "10.0.0.2"

最后在管理结点上的 mongodb 容器组 replica set:
[jacky@gen8-arch-linux ~]$ docker exec -it 97 /bin/bash
root@mongo_primary:/# mongo
MongoDB shell version v3.4.23
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.23
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
        http://docs.mongodb.org/
Questions? Try the support group
        http://groups.google.com/group/mongodb-user
Server has startup warnings:
2019-11-18T05:59:18.088+0000 I STORAGE  [initandlisten]
2019-11-18T05:59:18.088+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2019-11-18T05:59:18.088+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten]
2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten]
> exit
bye
root@mongo_primary:/# mongo
MongoDB shell version v3.4.23
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.23
Server has startup warnings:
2019-11-18T05:59:18.088+0000 I STORAGE  [initandlisten]
2019-11-18T05:59:18.088+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2019-11-18T05:59:18.088+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten]
2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten]
> config={_id:"mongo",members:[{_id:1,host:"mongo_primary:27017"},{_id:2,host:"mongo_secondary:27017"}]}
{
        "_id" : "mongo",
        "members" : [
                {
                        "_id" : 1,
                        "host" : "mongo_primary:27017"
                },
                {
                        "_id" : 2,
                        "host" : "mongo_secondary:27017"
                }
        ]
}
> rs.initiate(config)
{
        "ok" : 0,
        "errmsg" : "replSetInitiate quorum check failed because not all proposed set members responded affirmatively: mongo_secondary:27017 failed with Couldn't get a connection within the time limit",
        "code" : 74,
        "codeName" : "NodeNotFound"
}
> config={_id:"mongo",members:[{_id:1,host:"10.0.0.4:27017"},{_id:2,host:"10.0.0.2:27017"}]}
{
        "_id" : "mongo",
        "members" : [
                {
                        "_id" : 1,
                        "host" : "10.0.0.4:27017"
                },
                {
                        "_id" : 2,
                        "host" : "10.0.0.2:27017"
                }
        ]
}
> rs.initiate(config)
{
        "ok" : 0,
        "errmsg" : "replSetInitiate quorum check failed because not all proposed set members responded affirmatively: 10.0.0.2:27017 failed with Couldn't get a connection within the time limit",
        "code" : 74,
        "codeName" : "NodeNotFound"
}

总结:
在管理结点上的 mongodb 容器上用主机名或 IP 地址访问另一个工作结点上的 mongodb 容器的服务,不通。

请大牛们赐教,小白多谢了!
5859 次点击
所在节点    Docker
2 条回复
hopingtop
2019-11-29 16:28:58 +08:00
@just4id
overlay 如果容器之间相互 ping 能通
你试试 telnet 27017 这个端口
是不是差一个端口映射? -p 27017:27017
just4id
2019-11-29 21:20:54 +08:00
@hopingtop 谢谢回复,问题已解决,是防火墙的问题,关闭防火墙或开放端口就能解决问题。

这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/620662

V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX