WireGuard的管理利器Netmaker安装配置

Read

一、基础环境

$ curl cip.cc IP : 35.206.235.142 数据三 : 中国台湾省彰化县 | 谷歌 $ uname -a Linux tw.t4x.org 6.8.0-1017-gcp #19-Ubuntu SMP Tue Oct 15 19:02:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux $ curl cip.cc IP : 35.207.210.15 数据三 : 印度马哈拉施特拉孟买 | 谷歌 $ uname -a Linux mumbai 5.10.0-33-cloud-amd64 #1 SMP Debian 5.10.226-1 (2024-10-03) x86_64 GNU/Linux $ curl cip.cc IP : 35.213.179.122 数据三 : 新加坡 | 谷歌 $ uname -a Linux singapore 6.1.0-27-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.115-1 (2024-11-01) x86_64 GNU/Linux

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

$ curl cip.cc
IP      : 35.206.235.142
数据三  : 中国台湾省彰化县 | 谷歌
$ uname -a
Linux tw.t4x.org 6.8.0-1017-gcp #19-Ubuntu SMP Tue Oct 15 19:02:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

$ curl cip.cc
IP      : 35.207.210.15
数据三  : 印度马哈拉施特拉孟买 | 谷歌
$ uname  -a
Linux mumbai 5.10.0-33-cloud-amd64 #1 SMP Debian 5.10.226-1 (2024-10-03) x86_64 GNU/Linux

$ curl cip.cc
IP      : 35.213.179.122
数据三  : 新加坡 | 谷歌
$ uname -a
Linux singapore 6.1.0-27-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.115-1 (2024-11-01) x86_64 GNU/Linux

特别说明 : Netmaker的服务端有时候是用香港的Server,有时候用台湾的Server,安装方法不一样,一个是采用官方的快速安装,另外一个是采用docker-compose部署,但是基本操作方法是一样的.[自定义网段不同官方默认的100.64.0.0/16、docker-compose自定义网段是100.63.0.0/16] SourceByrd's Weblog-https://note.t4x.org/basic/netmaker-manager-wireguard/

二、防火墙配置

2.0 基础配置

apt install firewalld -y
systemctl start firewalld
firewall-cmd --new-zone=personal
firewall-cmd --permanent --new-zone=personal
firewall-cmd --zone=personal --add-interface=netmaker
firewall-cmd --permanent --zone=personal --add-interface=netmaker
firewall-cmd --zone=personal --add-port=22/tcp
firewall-cmd --set-default=personal

apt install firewalld -y

systemctl start firewalld

firewall-cmd --new-zone=personal

firewall-cmd --permanent --new-zone=personal

firewall-cmd --zone=personal --add-interface=netmaker

firewall-cmd --permanent --zone=personal --add-interface=netmaker

firewall-cmd --zone=personal --add-port=22/tcp

firewall-cmd --set-default=personal

2.1 必须配置

firewall-cmd --zone=personal --add-masquerade
firewall-cmd --zone=personal --add-port=51820-51899/udp
firewall-cmd --zone=personal --add-rich-rule="rule family="ipv4" source address="10.170.0.0/24" accept"
firewall-cmd --zone=personal --add-rich-rule="rule family="ipv4" source address="10.170.0.0/24" masquerade"
firewall-cmd --zone=personal --add-interface=ens4

firewall-cmd --zone=personal --add-masquerade

firewall-cmd --zone=personal --add-port=51820-51899/udp

firewall-cmd --zone=personal --add-rich-rule="rule family="ipv4" source address="10.170.0.0/24" accept"

firewall-cmd --zone=personal --add-rich-rule="rule family="ipv4" source address="10.170.0.0/24" masquerade"

firewall-cmd --zone=personal --add-interface=ens4

2.2 非必须配置

firewall-cmd --zone=personal --add-port=53/udp
firewall-cmd --zone=personal --add-port=53/tcp
firewall-cmd --zone=personal --add-port=443/tcp
firewall-cmd --zone=personal --add-port=80/tcp 
firewall-cmd --zone=personal --add-rich-rule="rule family="ipv4" source address="100.63.0.0/16" accept"
firewall-cmd --zone=personal --add-rich-rule="rule family="ipv4" source address="100.63.0.0/16" masquerade"
firewall-cmd --zone=personal --add-forward

firewall-cmd --zone=personal --add-port=53/udp

firewall-cmd --zone=personal --add-port=53/tcp

firewall-cmd --zone=personal --add-port=443/tcp

firewall-cmd --zone=personal --add-port=80/tcp

firewall-cmd --zone=personal --add-rich-rule="rule family="ipv4" source address="100.63.0.0/16" accept"

firewall-cmd --zone=personal --add-rich-rule="rule family="ipv4" source address="100.63.0.0/16" masquerade"

firewall-cmd --zone=personal --add-forward

SourceByrd's Weblog-https://note.t4x.org/basic/netmaker-manager-wireguard/

三、服务端部署tw

3.1 基础环境

cat >> /etc/sysctl.conf <<BYRD # WireGuard configure start net.ipv4.ip_forward = 1 net.ipv4.conf.all.rp_filter=2 net.ipv4.conf.all.proxy_arp = 1 # WireGuard configure end BYRD sysctl -p cat /proc/sys/net/ipv4/ip_forward cat /proc/sys/net/ipv4/conf/all/proxy_arp # sysctl -w net.ipv4.ip_forward=1 # sysctl -w net.ipv4.conf.all.rp_filter=2 # echo "1" > /proc/sys/net/ipv4/conf/all/proxy_arp # cat /proc/sys/net/ipv4/conf/all/proxy_arp

0
1
2
3
4
5
6
7
8
9
10
11
12
13

cat >> /etc/sysctl.conf <<BYRD
# WireGuard configure start
net.ipv4.ip_forward = 1
net.ipv4.conf.all.rp_filter=2
net.ipv4.conf.all.proxy_arp = 1
# WireGuard configure end
BYRD
sysctl -p
cat /proc/sys/net/ipv4/ip_forward
cat /proc/sys/net/ipv4/conf/all/proxy_arp
# sysctl -w net.ipv4.ip_forward=1
# sysctl -w net.ipv4.conf.all.rp_filter=2
# echo "1" >  /proc/sys/net/ipv4/conf/all/proxy_arp
# cat /proc/sys/net/ipv4/conf/all/proxy_arp

3.2 Centos

sudo dnf -y install dnf-plugins-core sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo sudo dnf install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

0
1
2

sudo dnf -y install dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

3.3 ubuntu、debian

sudo apt-get update -y sudo apt-get install ca-certificates curl -y sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc # Add the repository to Apt sources: echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update

0
1
2
3
4
5
6
7
8
9
10
11

sudo apt-get update -y
sudo apt-get install ca-certificates curl -y
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

VIP内容

VIP登录后查看

Login

3.4.4 nginx部署

$ wget -O - https://openresty.org/package/pubkey.gpg | sudo gpg --dearmor -o /usr/share/keyrings/openresty.gpg $ echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/openresty.gpg] http://openresty.org/package/ubuntu $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/openresty.list > /dev/null $ sudo apt-get update $ sudo apt-get -y install openresty $ egrep -v "^$|#" /usr/local/openresty/nginx/conf/nginx.conf worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; server { listen 80; server_name localhost; location / { root html; index index.html index.htm; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } server { listen 443 ssl; server_name dashboard.network.t4x.org; ssl_certificate certificate/fullchain.cer; ssl_certificate_key certificate/cert.key; ssl_session_cache shared:SSL:1m; ssl_session_timeout 5m; ssl_ciphers HIGH:!aNULL:!MD5; ssl_prefer_server_ciphers on; location / { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://127.0.0.1:8082; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; } } server { listen 443 ssl; server_name api.network.t4x.org; ssl_certificate certificate/fullchain.cer; ssl_certificate_key certificate/cert.key; ssl_session_cache shared:SSL:1m; ssl_session_timeout 5m; ssl_ciphers HIGH:!aNULL:!MD5; ssl_prefer_server_ciphers on; location / { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://127.0.0.1:8081; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; } } server { listen 443 ssl; server_name grpc.network.t4x.org; ssl_certificate certificate/fullchain.cer; ssl_certificate_key certificate/cert.key; ssl_session_cache shared:SSL:1m; ssl_session_timeout 5m; ssl_ciphers HIGH:!aNULL:!MD5; ssl_prefer_server_ciphers on; location / { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://127.0.0.1:8083; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; } } }

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83

$ wget -O - https://openresty.org/package/pubkey.gpg | sudo gpg --dearmor -o /usr/share/keyrings/openresty.gpg
$ echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/openresty.gpg] http://openresty.org/package/ubuntu $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/openresty.list > /dev/null
$ sudo apt-get update
$ sudo apt-get -y install openresty
$ egrep -v "^$|#" /usr/local/openresty/nginx/conf/nginx.conf
worker_processes  1;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  localhost;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
    server {
        listen       443 ssl;
        server_name  dashboard.network.t4x.org;
        ssl_certificate      certificate/fullchain.cer;
        ssl_certificate_key  certificate/cert.key;
        ssl_session_cache    shared:SSL:1m;
        ssl_session_timeout  5m;
        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;
        location / {
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass http://127.0.0.1:8082;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "Upgrade";
        }
    }
    server {
        listen       443 ssl;
        server_name  api.network.t4x.org;
        ssl_certificate      certificate/fullchain.cer;
        ssl_certificate_key  certificate/cert.key;
        ssl_session_cache    shared:SSL:1m;
        ssl_session_timeout  5m;
        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;
        location / {
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass http://127.0.0.1:8081;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "Upgrade";
        }
    }
    server {
        listen       443 ssl;
        server_name  grpc.network.t4x.org;
        ssl_certificate      certificate/fullchain.cer;
        ssl_certificate_key  certificate/cert.key;
        ssl_session_cache    shared:SSL:1m;
        ssl_session_timeout  5m;
        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;
        location / {
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass http://127.0.0.1:8083;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "Upgrade";
        }
    }
}

SourceByrd's Weblog-https://note.t4x.org/basic/netmaker-manager-wireguard/

四、节点接入mumbai

wget -O netclient https://fileserver.netmaker.io/releases/download/v0.26.0/netclient-linux-amd64 && chmod +x ./netclient && mv netclient /usr/bin/netclient && sudo netclient install netclient join -t eyJzZXJ2ZXIiOiJhcGkucGVyc29uYWwubmV0d29yay52Zy50NHgub3JnIiwidmFsdWUiOiJDQTNUUU5VWFRBR0JVVUVBRDUzNTNHTkhCUENFNlpLTSJ9

0
1

wget -O netclient https://fileserver.netmaker.io/releases/download/v0.26.0/netclient-linux-amd64 && chmod +x ./netclient && mv netclient /usr/bin/netclient && sudo netclient install
netclient join -t eyJzZXJ2ZXIiOiJhcGkucGVyc29uYWwubmV0d29yay52Zy50NHgub3JnIiwidmFsdWUiOiJDQTNUUU5VWFRBR0JVVUVBRDUzNTNHTkhCUENFNlpLTSJ9

SourceByrd's Weblog-https://note.t4x.org/basic/netmaker-manager-wireguard/

五、连通验证

5.1 windows客户端[河北]

C:\Windows\System32>netclient join -t eyJzZXJ2ZXIiOiJhcGkucGVyc29uYWwubmV0d29yay52Zy50NHgub3JnIiwidmFsdWUiOiJSRzNZUTVaQVhMRlA0UkdZREk2Uk5QRklON1pSUFJPVCJ9 setting host fields {"time":"2024-11-23T14:19:35.8615368+08:00","level":"ERROR","source":"daemon.go 424}","msg":"unable to connect to broker","server":"wss://broker.nm.35-206-235-142.nip.io","error":"connect timeout"} WARNING: Joining any network on another server will disconnect netclient from the networks of the current server -> nm.35-206-235-142.nip.io registered with server C:\Windows\System32>tracert 100.63.0.1 通过最多 30 个跃点跟踪到 100.63.0.1 的路由 1 51 ms 50 ms 51 ms 100.63.0.1 跟踪完成。 C:\Windows\System32>ping 100.63.0.1 #台湾服务器正在 Ping 100.63.0.1 具有 32 字节的数据: 来自 100.63.0.1 的回复: 字节=32 时间=1143ms TTL=64 来自 100.63.0.1 的回复: 字节=32 时间=54ms TTL=64 来自 100.63.0.1 的回复: 字节=32 时间=52ms TTL=64 来自 100.63.0.1 的回复: 字节=32 时间=51ms TTL=64 C:\Windows\System32>ping 100.63.0.3 #新加坡服务器正在 Ping 100.63.0.3 具有 32 字节的数据: 来自 100.63.0.3 的回复: 字节=32 时间=203ms TTL=64 来自 100.63.0.3 的回复: 字节=32 时间=205ms TTL=64 来自 100.63.0.3 的回复: 字节=32 时间=203ms TTL=64 来自 100.63.0.3 的回复: 字节=32 时间=203ms TTL=64 C:\Windows\System32>wg interface: netmaker public key: fzOHq2WPjzYIBw7pX/rv4OE7GqDE07QOOKMvOOF4u00= private key: (hidden) listening port: 51821 peer: nS5+/qajqCVlsn0euvnKTBks7zLV5MDmbV9t0b0lJiQ= endpoint: 35.213.179.122:51821 allowed ips: 100.63.0.3/32 latest handshake: 3 seconds ago transfer: 124 B received, 476 B sent persistent keepalive: every 20 seconds peer: WIB3JASnLHpxbDXzTAKlNeNbkJ7l0wTqArofcJMQ2hQ= endpoint: 35.206.235.142:51821 allowed ips: 100.63.0.1/32 latest handshake: 9 seconds ago transfer: 508 B received, 712 B sent persistent keepalive: every 20 seconds

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

C:\Windows\System32>netclient join -t eyJzZXJ2ZXIiOiJhcGkucGVyc29uYWwubmV0d29yay52Zy50NHgub3JnIiwidmFsdWUiOiJSRzNZUTVaQVhMRlA0UkdZREk2Uk5QRklON1pSUFJPVCJ9
setting host fields
{"time":"2024-11-23T14:19:35.8615368+08:00","level":"ERROR","source":"daemon.go 424}","msg":"unable to connect to broker","server":"wss://broker.nm.35-206-235-142.nip.io","error":"connect timeout"}
WARNING: Joining any network on another server will disconnect netclient from the networks of the current server -> nm.35-206-235-142.nip.io
registered with server
C:\Windows\System32>tracert 100.63.0.1
通过最多 30 个跃点跟踪到 100.63.0.1 的路由
  1    51 ms    50 ms    51 ms  100.63.0.1
跟踪完成。
C:\Windows\System32>ping 100.63.0.1    #台湾服务器
正在 Ping 100.63.0.1 具有 32 字节的数据:
来自 100.63.0.1 的回复: 字节=32 时间=1143ms TTL=64
来自 100.63.0.1 的回复: 字节=32 时间=54ms TTL=64
来自 100.63.0.1 的回复: 字节=32 时间=52ms TTL=64
来自 100.63.0.1 的回复: 字节=32 时间=51ms TTL=64
C:\Windows\System32>ping 100.63.0.3    #新加坡服务器
正在 Ping 100.63.0.3 具有 32 字节的数据:
来自 100.63.0.3 的回复: 字节=32 时间=203ms TTL=64
来自 100.63.0.3 的回复: 字节=32 时间=205ms TTL=64
来自 100.63.0.3 的回复: 字节=32 时间=203ms TTL=64
来自 100.63.0.3 的回复: 字节=32 时间=203ms TTL=64
C:\Windows\System32>wg
interface: netmaker
  public key: fzOHq2WPjzYIBw7pX/rv4OE7GqDE07QOOKMvOOF4u00=
  private key: (hidden)
  listening port: 51821

peer: nS5+/qajqCVlsn0euvnKTBks7zLV5MDmbV9t0b0lJiQ=
  endpoint: 35.213.179.122:51821
  allowed ips: 100.63.0.3/32
  latest handshake: 3 seconds ago
  transfer: 124 B received, 476 B sent
  persistent keepalive: every 20 seconds

peer: WIB3JASnLHpxbDXzTAKlNeNbkJ7l0wTqArofcJMQ2hQ=
  endpoint: 35.206.235.142:51821
  allowed ips: 100.63.0.1/32
  latest handshake: 9 seconds ago
  transfer: 508 B received, 712 B sent
  persistent keepalive: every 20 seconds

5.2 ubuntu客户端[新加坡]

$ netclient join -t eyJzZXJ2ZXIiOiJhcGkucGVyc29uYWwubmV0d29yay52Zy50NHgub3JnIiwidmFsdWUiOiJSRzNZUTVaQVhMRlA0UkdZREk2Uk5QRklON1pSUFJPVCJ9 setting host fields WARNING: Joining any network on another server will disconnect netclient from the networks of the current server -> nm.35-206-235-142.nip.io registered with server $ wg interface: netmaker public key: nS5+/qajqCVlsn0euvnKTBks7zLV5MDmbV9t0b0lJiQ= private key: (hidden) listening port: 51821 peer: WIB3JASnLHpxbDXzTAKlNeNbkJ7l0wTqArofcJMQ2hQ= endpoint: 10.140.0.3:51821 allowed ips: 100.63.0.1/32 latest handshake: 29 seconds ago transfer: 956 B received, 1020 B sent persistent keepalive: every 20 seconds peer: fzOHq2WPjzYIBw7pX/rv4OE7GqDE07QOOKMvOOF4u00= endpoint: x.x.x.x:22628 allowed ips: 100.63.0.2/32 latest handshake: 35 seconds ago transfer: 212 B received, 272 B sent persistent keepalive: every 20 seconds

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

$ netclient join -t eyJzZXJ2ZXIiOiJhcGkucGVyc29uYWwubmV0d29yay52Zy50NHgub3JnIiwidmFsdWUiOiJSRzNZUTVaQVhMRlA0UkdZREk2Uk5QRklON1pSUFJPVCJ9
setting host fields
WARNING: Joining any network on another server will disconnect netclient from the networks of the current server -> nm.35-206-235-142.nip.io
registered with server
$ wg
interface: netmaker
  public key: nS5+/qajqCVlsn0euvnKTBks7zLV5MDmbV9t0b0lJiQ=
  private key: (hidden)
  listening port: 51821

peer: WIB3JASnLHpxbDXzTAKlNeNbkJ7l0wTqArofcJMQ2hQ=
  endpoint: 10.140.0.3:51821
  allowed ips: 100.63.0.1/32
  latest handshake: 29 seconds ago
  transfer: 956 B received, 1020 B sent
  persistent keepalive: every 20 seconds

peer: fzOHq2WPjzYIBw7pX/rv4OE7GqDE07QOOKMvOOF4u00=
  endpoint: x.x.x.x:22628
  allowed ips: 100.63.0.2/32
  latest handshake: 35 seconds ago
  transfer: 212 B received, 272 B sent
  persistent keepalive: every 20 seconds

5.3 netmaker服务端[台湾]

$ wg interface: netmaker public key: WIB3JASnLHpxbDXzTAKlNeNbkJ7l0wTqArofcJMQ2hQ= private key: (hidden) listening port: 51821 peer: nS5+/qajqCVlsn0euvnKTBks7zLV5MDmbV9t0b0lJiQ= endpoint: 10.148.0.2:51821 allowed ips: 100.63.0.3/32 latest handshake: 1 minute, 39 seconds ago transfer: 820 B received, 1.03 KiB sent persistent keepalive: every 20 seconds peer: fzOHq2WPjzYIBw7pX/rv4OE7GqDE07QOOKMvOOF4u00= endpoint: x.x.x.x:24751 allowed ips: 100.63.0.2/32 latest handshake: 1 minute, 50 seconds ago transfer: 724 B received, 656 B sent persistent keepalive: every 20 seconds

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

$ wg
interface: netmaker
  public key: WIB3JASnLHpxbDXzTAKlNeNbkJ7l0wTqArofcJMQ2hQ=
  private key: (hidden)
  listening port: 51821

peer: nS5+/qajqCVlsn0euvnKTBks7zLV5MDmbV9t0b0lJiQ=
  endpoint: 10.148.0.2:51821
  allowed ips: 100.63.0.3/32
  latest handshake: 1 minute, 39 seconds ago
  transfer: 820 B received, 1.03 KiB sent
  persistent keepalive: every 20 seconds

peer: fzOHq2WPjzYIBw7pX/rv4OE7GqDE07QOOKMvOOF4u00=
  endpoint: x.x.x.x:24751
  allowed ips: 100.63.0.2/32
  latest handshake: 1 minute, 50 seconds ago
  transfer: 724 B received, 656 B sent
  persistent keepalive: every 20 seconds

SourceByrd's Weblog-https://note.t4x.org/basic/netmaker-manager-wireguard/

六、官方快速安装方案

sudo ufw allow 443/tcp sudo ufw allow 80/tcp sudo ufw allow 51821/udp sudo ufw allow 51821/tcp sudo ufw allow 53 sudo ufw allow 8085/tcp sudo ufw allow 1883/tcp sudo ufw allow 8883/tcp sudo ufw allow 8083/tcp sudo ufw allow 18083/tcp sudo ufw allow 22/tcp sudo ufw enable wget -qO /root/nm-quick.sh https://raw.githubusercontent.com/gravitl/netmaker/master/scripts/nm-quick.sh && sudo chmod +x /root/nm-quick.sh && sudo /root/nm-quick.sh . . . ----------------------------------------------------- 1) Auto Generated (nm.35-206-235-142.nip.io) 2) Custom Domain (e.x: netmaker.example.com) #? 1 using nm.35-206-235-142.nip.io for base domain . . . ----------------------------------------------------- The following subdomains will be used: dashboard.nm.35-206-235-142.nip.io api.nm.35-206-235-142.nip.io broker.nm.35-206-235-142.nip.io ----------------------------------------------------- . . . Email Address for Domain Registration: admin@qq.com . . . ----------------------------------------------------------------- SETUP ARGUMENTS ----------------------------------------------------------------- domain: nm.35-206-235-142.nip.io email: admin@qq.com public ip: 35.206.235.142 ----------------------------------------------------------------- Confirm Settings for Installation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Does everything look right? [y/n]: y ----------------------------------------------------------------- Beginning installation... ----------------------------------------------------------------- . . . Pulling config files... Saving the config to /root/netmaker.env Starting containers... . . . Testing Caddy setup (please be patient, this may take 1-2 minutes) Downloading nmctl... using server api.nm.35-206-235-142.nip.io using master key p6iXAxIMEyUvVHrvtcRv943vk8fayg . . . . . . . . . . . . . . . Creating netmaker network (100.64.0.0/16) { "addressrange": "100.64.0.0/16", "addressrange6": "", "netid": "netmaker", "nodeslastmodified": 1732333478, "networklastmodified": 1732333478, "defaultinterface": "nm-netmaker", "defaultlistenport": 51821, "nodelimit": 999999999, "defaultpostdown": "", "defaultkeepalive": 20, "allowmanualsignup": "no", "isipv4": "yes", "isipv6": "no", "defaultudpholepunch": "no", "defaultmtu": 1280, "defaultacl": "yes" } Obtaining enrollment key... . . . . . . . . . [netclient] 2024-11-23 03:44:48 setting OS [netclient] 2024-11-23 03:44:48 setting version [netclient] 2024-11-23 03:44:48 setting netclient hostid [netclient] 2024-11-23 03:44:48 setting name [netclient] 2024-11-23 03:44:48 setting macAddress [netclient] 2024-11-23 03:44:48 setting wireguard keys [netclient] 2024-11-23 03:44:48 setting wireguard interface [netclient] 2024-11-23 03:44:48 setting listenport [netclient] 2024-11-23 03:44:48 setting MTU [netclient] 2024-11-23 03:44:48 setting traffic keys Register token: eyJzZXJ2ZXIiOiJhcGkubm0uMzUtMjA2LTIzNS0xNDIubmlwLmlvIiwidmFsdWUiOiIzNVRUUDdHV0JJS1JVV0tCSU9aNFlIRkgyQjZYWUNRTCJ9 setting host fields registered with server nm.35-206-235-142.nip.io waiting for netclient to become available register complete. New node ID: ae37cf11-d853-4081-868f-e75a8a0348c5 making host a default Host ID: 7d529a13-7f85-4ef9-b418-67698b42f1e8 { "id": "7d529a13-7f85-4ef9-b418-67698b42f1e8", "verbosity": 0, "firewallinuse": "", "version": "v0.26.0", "name": "tw", "os": "linux", "debug": false, "isstaticport": false, "isstatic": false, "listenport": 51821, "wg_public_listen_port": 0, "mtu": 1420, "interfaces": [ { "name": "ens4", "addressString": "10.140.0.3/32" }, { "name": "ens4", "addressString": "fe80::4001:aff:fe8c:3/64" }, { "name": "veth91681b9", "addressString": "fe80::1e:9fff:fe45:7766/64" }, { "name": "veth744f7dd", "addressString": "fe80::e818:80ff:fee8:1d25/64" }, { "name": "veth38266bd", "addressString": "fe80::d421:e1ff:fe99:cef2/64" }, { "name": "veth88cdfc5", "addressString": "fe80::7085:71ff:fea9:2c4b/64" }, { "name": "veth9fcc8b9", "addressString": "fe80::e4d4:b2ff:fe90:6c5/64" } ], "defaultinterface": "ens4", "endpointip": "35.206.235.142", "endpointipv6": "", "publickey": "PcGvmnP8w66LXjsp26jGVMxolYbxXHL6wGUOmuXGj3A=", "macaddress": "42:01:0a:8c:00:03", "nodes": [ "ae37cf11-d853-4081-868f-e75a8a0348c5" ], "isdefault": true, "nat_type": "behind_nat", "persistentkeepalive": 20, "autoupdate": false } { "id": "ae37cf11-d853-4081-868f-e75a8a0348c5", "hostid": "7d529a13-7f85-4ef9-b418-67698b42f1e8", "address": "100.64.0.1/16", "address6": "", "localaddress": "", "allowedips": null, "lastmodified": 1732333519, "expdatetime": 4890599111, "lastcheckin": 1732333512, "lastpeerupdate": -62135596800, "network": "netmaker", "networkrange": "100.64.0.0/16", "networkrange6": "", "isrelayed": false, "isrelay": false, "relayedby": "", "relaynodes": null, "isegressgateway": false, "isingressgateway": true, "egressgatewayranges": null, "egressgatewaynatenabled": false, "dnson": true, "ingressdns": "", "ingresspersistentkeepalive": 20, "ingressmtu": 1420, "server": "nm.35-206-235-142.nip.io", "connected": true, "pendingdelete": false, "metadata": "This host can be used for remote access", "defaultacl": "yes", "is_fail_over": false, "fail_over_peers": {}, "failed_over_by": "00000000-0000-0000-0000-000000000000", "isinternetgateway": false, "inet_node_req": { "inet_node_client_ids": null }, "internetgw_node_id": "", "additional_rag_ips": [], "tags": { "netmaker.remote-access-gws": {} }, "is_static": false, "is_user_node": false, "static_node": { "clientid": "", "privatekey": "", "publickey": "", "network": "", "dns": "", "address": "", "address6": "", "extraallowedips": null, "allowed_ips": null, "ingressgatewayid": "", "ingressgatewayendpoint": "", "lastmodified": 0, "enabled": false, "ownerid": "", "deniednodeacls": null, "remote_access_client_id": "", "postup": "", "postdown": "", "tags": null } } root@hk:~# cat netmaker.env NM_EMAIL=admin@qq.com NM_DOMAIN=nm.35-215-135-34.nip.io FRONTEND_URL= UI_IMAGE_TAG=v0.26.0 METRICS_EXPORTER=off PROMETHEUS=off SERVER_IMAGE_TAG=v0.26.0 SERVER_HOST=35.215.135.34 MASTER_KEY=bhU0fiy5aAwXEnTykIlcVhLb5qjLl2 MQ_USERNAME=netmaker MQ_PASSWORD=j6ZmMFxmijTb84SKKgokB10YiD9YTL LICENSE_KEY= NETMAKER_TENANT_ID= INSTALL_TYPE=ce NODE_ID=netmaker-server-1 DNS_MODE=on NETCLIENT_AUTO_UPDATE=enabled API_PORT=8081 CORS_ALLOWED_ORIGIN=* DISPLAY_KEYS=on DATABASE=sqlite SERVER_BROKER_ENDPOINT=ws://mq:1883 VERBOSITY=1 DEBUG_MODE=off REST_BACKEND=on DISABLE_REMOTE_IP_CHECK=off TELEMETRY=on ALLOWED_EMAIL_DOMAINS=* AUTH_PROVIDER= CLIENT_ID= CLIENT_SECRET= AZURE_TENANT= OIDC_ISSUER= EXPORTER_API_PORT=8085 JWT_VALIDITY_DURATION=43200 RAC_AUTO_DISABLE=false CACHING_ENABLED=true ENDPOINT_DETECTION=true SMTP_HOST=smtp.gmail.com SMTP_PORT=587 EMAIL_SENDER_ADDR= EMAIL_SENDER_USER= EMAIL_SENDER_PASSWORD= root@hk:~# cat wait.sh #!/bin/ash encrypt_password() { echo "${MQ_USERNAME}:${MQ_PASSWORD}" > /mosquitto/password.txt mosquitto_passwd -U /mosquitto/password.txt } main(){ encrypt_password echo "Starting MQ..." # Run the main container command. /docker-entrypoint.sh /usr/sbin/mosquitto -c /mosquitto/config/mosquitto.conf } main "${@}" root@hk:~# cat mosquitto.conf per_listener_settings false listener 8883 protocol websockets allow_anonymous false listener 1883 protocol websockets allow_anonymous false password_file /mosquitto/password.txt root@hk:~# cat docker-compose.yml version: "3.4" services: netmaker: container_name: netmaker image: gravitl/netmaker:$SERVER_IMAGE_TAG env_file: ./netmaker.env restart: always volumes: - dnsconfig:/root/config/dnsconfig - sqldata:/root/data environment: # config-dependant vars - STUN_LIST=stun1.netmaker.io:3478,stun2.netmaker.io:3478,stun1.l.google.com:19302,stun2.l.google.com:19302 # The domain/host IP indicating the mq broker address - BROKER_ENDPOINT=wss://broker.${NM_DOMAIN} # For EMQX broker use `BROKER_ENDPOINT=wss://broker.${NM_DOMAIN}/mqtt` # For EMQX broker (uncomment the two lines below) #- BROKER_TYPE=emqx #- EMQX_REST_ENDPOINT=http://mq:18083 # The base domain of netmaker - SERVER_NAME=${NM_DOMAIN} - SERVER_API_CONN_STRING=api.${NM_DOMAIN}:443 # Address of the CoreDNS server. Defaults to SERVER_HOST - COREDNS_ADDR=${SERVER_HOST} # Overrides SERVER_HOST if set. Useful for making HTTP available via different interfaces/networks. - SERVER_HTTP_HOST=api.${NM_DOMAIN} netmaker-ui: container_name: netmaker-ui image: gravitl/netmaker-ui:$UI_IMAGE_TAG env_file: ./netmaker.env environment: # config-dependant vars # URL where UI will send API requests. Change based on SERVER_HOST, SERVER_HTTP_HOST, and API_PORT BACKEND_URL: "https://api.${NM_DOMAIN}" depends_on: - netmaker links: - "netmaker:api" restart: always caddy: image: caddy:2.8.4 container_name: caddy env_file: ./netmaker.env restart: unless-stopped extra_hosts: - "host.docker.internal:host-gateway" volumes: - ./Caddyfile:/etc/caddy/Caddyfile - caddy_data:/data - caddy_conf:/config ports: - "80:80" - "443:443" coredns: #network_mode: host container_name: coredns image: coredns/coredns:1.10.1 command: -conf /root/dnsconfig/Corefile env_file: ./netmaker.env restart: always depends_on: - netmaker volumes: - dnsconfig:/root/dnsconfig mq: container_name: mq image: eclipse-mosquitto:2.0.15-openssl env_file: ./netmaker.env depends_on: - netmaker restart: unless-stopped command: [ "/mosquitto/config/wait.sh" ] volumes: - ./mosquitto.conf:/mosquitto/config/mosquitto.conf - ./wait.sh:/mosquitto/config/wait.sh - mosquitto_logs:/mosquitto/log - mosquitto_data:/mosquitto/data volumes: caddy_data: { } # runtime data for caddy caddy_conf: { } # configuration file for Caddy sqldata: { } dnsconfig: { } # storage for coredns mosquitto_logs: { } # storage for mqtt logs mosquitto_data: { } # storage for mqtt data root@tw:~# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 10.140.0.1 0.0.0.0 UG 100 0 0 ens4 10.140.0.1 0.0.0.0 255.255.255.255 UH 100 0 0 ens4 100.64.0.0 0.0.0.0 255.255.0.0 U 0 0 0 netmaker 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 172.18.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-90d6161ad499 root@tw:~# netstat -tunlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:51821 0.0.0.0:* LISTEN 6113/netclient tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 5569/docker-proxy tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 5546/docker-proxy tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 861/sshd: /usr/sbin tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 453/systemd-resolve tcp6 0 0 :::51821 :::* LISTEN 6113/netclient tcp6 0 0 :::443 :::* LISTEN 5575/docker-proxy tcp6 0 0 :::80 :::* LISTEN 5560/docker-proxy tcp6 0 0 :::22 :::* LISTEN 861/sshd: /usr/sbin udp 0 0 127.0.0.53:53 0.0.0.0:* 453/systemd-resolve udp 0 0 10.140.0.3:68 0.0.0.0:* 450/systemd-network udp 0 0 127.0.0.1:323 0.0.0.0:* 1544/chronyd udp 0 0 0.0.0.0:51821 0.0.0.0:* - udp6 0 0 ::1:323 :::* 1544/chronyd udp6 0 0 :::51821 :::* -

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416

sudo ufw allow 443/tcp
sudo ufw allow 80/tcp
sudo ufw allow 51821/udp
sudo ufw allow 51821/tcp
sudo ufw allow 53
sudo ufw allow 8085/tcp
sudo ufw allow 1883/tcp
sudo ufw allow 8883/tcp
sudo ufw allow 8083/tcp
sudo ufw allow 18083/tcp
sudo ufw allow 22/tcp
sudo ufw enable
wget -qO /root/nm-quick.sh https://raw.githubusercontent.com/gravitl/netmaker/master/scripts/nm-quick.sh && sudo chmod +x /root/nm-quick.sh && sudo /root/nm-quick.sh
. . .
-----------------------------------------------------
1) Auto Generated (nm.35-206-235-142.nip.io)
2) Custom Domain (e.x: netmaker.example.com)
#? 1
using nm.35-206-235-142.nip.io for base domain
. . .
-----------------------------------------------------
The following subdomains will be used:
          dashboard.nm.35-206-235-142.nip.io
                api.nm.35-206-235-142.nip.io
             broker.nm.35-206-235-142.nip.io
-----------------------------------------------------
. . .
Email Address for Domain Registration: admin@qq.com
. . .
-----------------------------------------------------------------
                SETUP ARGUMENTS
-----------------------------------------------------------------
        domain: nm.35-206-235-142.nip.io
         email: admin@qq.com
     public ip: 35.206.235.142
-----------------------------------------------------------------
Confirm Settings for Installation
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Does everything look right? [y/n]: y
-----------------------------------------------------------------
Beginning installation...
-----------------------------------------------------------------
. . .
Pulling config files...
Saving the config to /root/netmaker.env
Starting containers...
. . .
Testing Caddy setup (please be patient, this may take 1-2 minutes)

Downloading nmctl...
using server api.nm.35-206-235-142.nip.io
using master key p6iXAxIMEyUvVHrvtcRv943vk8fayg
. . .
. . .
. . .
. . .
. . .
Creating netmaker network (100.64.0.0/16)
{
  "addressrange": "100.64.0.0/16",
  "addressrange6": "",
  "netid": "netmaker",
  "nodeslastmodified": 1732333478,
  "networklastmodified": 1732333478,
  "defaultinterface": "nm-netmaker",
  "defaultlistenport": 51821,
  "nodelimit": 999999999,
  "defaultpostdown": "",
  "defaultkeepalive": 20,
  "allowmanualsignup": "no",
  "isipv4": "yes",
  "isipv6": "no",
  "defaultudpholepunch": "no",
  "defaultmtu": 1280,
  "defaultacl": "yes"
}
Obtaining enrollment key...
. . .
. . .
. . .
[netclient] 2024-11-23 03:44:48 setting OS
[netclient] 2024-11-23 03:44:48 setting version
[netclient] 2024-11-23 03:44:48 setting netclient hostid
[netclient] 2024-11-23 03:44:48 setting name
[netclient] 2024-11-23 03:44:48 setting macAddress
[netclient] 2024-11-23 03:44:48 setting wireguard keys
[netclient] 2024-11-23 03:44:48 setting wireguard interface
[netclient] 2024-11-23 03:44:48 setting listenport
[netclient] 2024-11-23 03:44:48 setting MTU
[netclient] 2024-11-23 03:44:48 setting traffic keys
Register token: eyJzZXJ2ZXIiOiJhcGkubm0uMzUtMjA2LTIzNS0xNDIubmlwLmlvIiwidmFsdWUiOiIzNVRUUDdHV0JJS1JVV0tCSU9aNFlIRkgyQjZYWUNRTCJ9
setting host fields
registered with server nm.35-206-235-142.nip.io
waiting for netclient to become available
register complete. New node ID: ae37cf11-d853-4081-868f-e75a8a0348c5
making host a default
Host ID: 7d529a13-7f85-4ef9-b418-67698b42f1e8
{
  "id": "7d529a13-7f85-4ef9-b418-67698b42f1e8",
  "verbosity": 0,
  "firewallinuse": "",
  "version": "v0.26.0",
  "name": "tw",
  "os": "linux",
  "debug": false,
  "isstaticport": false,
  "isstatic": false,
  "listenport": 51821,
  "wg_public_listen_port": 0,
  "mtu": 1420,
  "interfaces": [
    {
      "name": "ens4",
      "addressString": "10.140.0.3/32"
    },
    {
      "name": "ens4",
      "addressString": "fe80::4001:aff:fe8c:3/64"
    },
    {
      "name": "veth91681b9",
      "addressString": "fe80::1e:9fff:fe45:7766/64"
    },
    {
      "name": "veth744f7dd",
      "addressString": "fe80::e818:80ff:fee8:1d25/64"
    },
    {
      "name": "veth38266bd",
      "addressString": "fe80::d421:e1ff:fe99:cef2/64"
    },
    {
      "name": "veth88cdfc5",
      "addressString": "fe80::7085:71ff:fea9:2c4b/64"
    },
    {
      "name": "veth9fcc8b9",
      "addressString": "fe80::e4d4:b2ff:fe90:6c5/64"
    }
  ],
  "defaultinterface": "ens4",
  "endpointip": "35.206.235.142",
  "endpointipv6": "",
  "publickey": "PcGvmnP8w66LXjsp26jGVMxolYbxXHL6wGUOmuXGj3A=",
  "macaddress": "42:01:0a:8c:00:03",
  "nodes": [
    "ae37cf11-d853-4081-868f-e75a8a0348c5"
  ],
  "isdefault": true,
  "nat_type": "behind_nat",
  "persistentkeepalive": 20,
  "autoupdate": false
}
{
  "id": "ae37cf11-d853-4081-868f-e75a8a0348c5",
  "hostid": "7d529a13-7f85-4ef9-b418-67698b42f1e8",
  "address": "100.64.0.1/16",
  "address6": "",
  "localaddress": "",
  "allowedips": null,
  "lastmodified": 1732333519,
  "expdatetime": 4890599111,
  "lastcheckin": 1732333512,
  "lastpeerupdate": -62135596800,
  "network": "netmaker",
  "networkrange": "100.64.0.0/16",
  "networkrange6": "",
  "isrelayed": false,
  "isrelay": false,
  "relayedby": "",
  "relaynodes": null,
  "isegressgateway": false,
  "isingressgateway": true,
  "egressgatewayranges": null,
  "egressgatewaynatenabled": false,
  "dnson": true,
  "ingressdns": "",
  "ingresspersistentkeepalive": 20,
  "ingressmtu": 1420,
  "server": "nm.35-206-235-142.nip.io",
  "connected": true,
  "pendingdelete": false,
  "metadata": "This host can be used for remote access",
  "defaultacl": "yes",
  "is_fail_over": false,
  "fail_over_peers": {},
  "failed_over_by": "00000000-0000-0000-0000-000000000000",
  "isinternetgateway": false,
  "inet_node_req": {
    "inet_node_client_ids": null
  },
  "internetgw_node_id": "",
  "additional_rag_ips": [],
  "tags": {
    "netmaker.remote-access-gws": {}
  },
  "is_static": false,
  "is_user_node": false,
  "static_node": {
    "clientid": "",
    "privatekey": "",
    "publickey": "",
    "network": "",
    "dns": "",
    "address": "",
    "address6": "",
    "extraallowedips": null,
    "allowed_ips": null,
    "ingressgatewayid": "",
    "ingressgatewayendpoint": "",
    "lastmodified": 0,
    "enabled": false,
    "ownerid": "",
    "deniednodeacls": null,
    "remote_access_client_id": "",
    "postup": "",
    "postdown": "",
    "tags": null
  }
}

root@hk:~# cat netmaker.env
NM_EMAIL=admin@qq.com
NM_DOMAIN=nm.35-215-135-34.nip.io
FRONTEND_URL=
UI_IMAGE_TAG=v0.26.0
METRICS_EXPORTER=off
PROMETHEUS=off
SERVER_IMAGE_TAG=v0.26.0
SERVER_HOST=35.215.135.34
MASTER_KEY=bhU0fiy5aAwXEnTykIlcVhLb5qjLl2
MQ_USERNAME=netmaker
MQ_PASSWORD=j6ZmMFxmijTb84SKKgokB10YiD9YTL
LICENSE_KEY=
NETMAKER_TENANT_ID=
INSTALL_TYPE=ce
NODE_ID=netmaker-server-1
DNS_MODE=on
NETCLIENT_AUTO_UPDATE=enabled
API_PORT=8081
CORS_ALLOWED_ORIGIN=*
DISPLAY_KEYS=on
DATABASE=sqlite
SERVER_BROKER_ENDPOINT=ws://mq:1883
VERBOSITY=1
DEBUG_MODE=off
REST_BACKEND=on
DISABLE_REMOTE_IP_CHECK=off
TELEMETRY=on
ALLOWED_EMAIL_DOMAINS=*
AUTH_PROVIDER=
CLIENT_ID=
CLIENT_SECRET=
AZURE_TENANT=
OIDC_ISSUER=
EXPORTER_API_PORT=8085
JWT_VALIDITY_DURATION=43200
RAC_AUTO_DISABLE=false
CACHING_ENABLED=true
ENDPOINT_DETECTION=true
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
EMAIL_SENDER_ADDR=
EMAIL_SENDER_USER=
EMAIL_SENDER_PASSWORD=

root@hk:~# cat wait.sh
#!/bin/ash

encrypt_password() {
  echo "${MQ_USERNAME}:${MQ_PASSWORD}" > /mosquitto/password.txt
  mosquitto_passwd -U /mosquitto/password.txt
}

main(){

encrypt_password
echo "Starting MQ..."
# Run the main container command.
/docker-entrypoint.sh
/usr/sbin/mosquitto -c /mosquitto/config/mosquitto.conf

}

main "${@}"

root@hk:~# cat mosquitto.conf
per_listener_settings false
listener 8883
protocol websockets
allow_anonymous false

listener 1883
protocol websockets
allow_anonymous false

password_file /mosquitto/password.txt

root@hk:~# cat docker-compose.yml
version: "3.4"

services:

  netmaker:
    container_name: netmaker
    image: gravitl/netmaker:$SERVER_IMAGE_TAG
    env_file: ./netmaker.env
    restart: always
    volumes:
      - dnsconfig:/root/config/dnsconfig
      - sqldata:/root/data
    environment:
      # config-dependant vars
      - STUN_LIST=stun1.netmaker.io:3478,stun2.netmaker.io:3478,stun1.l.google.com:19302,stun2.l.google.com:19302
      # The domain/host IP indicating the mq broker address
      - BROKER_ENDPOINT=wss://broker.${NM_DOMAIN} # For EMQX broker use `BROKER_ENDPOINT=wss://broker.${NM_DOMAIN}/mqtt`
      # For EMQX broker (uncomment the two lines below)
      #- BROKER_TYPE=emqx
      #- EMQX_REST_ENDPOINT=http://mq:18083
      # The base domain of netmaker
      - SERVER_NAME=${NM_DOMAIN}
      - SERVER_API_CONN_STRING=api.${NM_DOMAIN}:443
      # Address of the CoreDNS server. Defaults to SERVER_HOST
      - COREDNS_ADDR=${SERVER_HOST}
      # Overrides SERVER_HOST if set. Useful for making HTTP available via different interfaces/networks.
      - SERVER_HTTP_HOST=api.${NM_DOMAIN}

  netmaker-ui:
    container_name: netmaker-ui
    image: gravitl/netmaker-ui:$UI_IMAGE_TAG
    env_file: ./netmaker.env
    environment:
      # config-dependant vars
      # URL where UI will send API requests. Change based on SERVER_HOST, SERVER_HTTP_HOST, and API_PORT
      BACKEND_URL: "https://api.${NM_DOMAIN}"
    depends_on:
      - netmaker
    links:
      - "netmaker:api"
    restart: always

  caddy:
    image: caddy:2.8.4
    container_name: caddy
    env_file: ./netmaker.env
    restart: unless-stopped
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - caddy_data:/data
      - caddy_conf:/config
    ports:
      - "80:80"
      - "443:443"

  coredns:
    #network_mode: host
    container_name: coredns
    image: coredns/coredns:1.10.1
    command: -conf /root/dnsconfig/Corefile
    env_file: ./netmaker.env
    restart: always
    depends_on:
      - netmaker
    volumes:
      - dnsconfig:/root/dnsconfig
  mq:
    container_name: mq
    image: eclipse-mosquitto:2.0.15-openssl
    env_file: ./netmaker.env
    depends_on:
      - netmaker
    restart: unless-stopped
    command: [ "/mosquitto/config/wait.sh" ]
    volumes:
      - ./mosquitto.conf:/mosquitto/config/mosquitto.conf
      - ./wait.sh:/mosquitto/config/wait.sh
      - mosquitto_logs:/mosquitto/log
      - mosquitto_data:/mosquitto/data
volumes:
  caddy_data: { } # runtime data for caddy
  caddy_conf: { } # configuration file for Caddy
  sqldata: { }
  dnsconfig: { } # storage for coredns
  mosquitto_logs: { } # storage for mqtt logs
  mosquitto_data: { } # storage for mqtt data

root@tw:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.140.0.1      0.0.0.0         UG    100    0        0 ens4
10.140.0.1      0.0.0.0         255.255.255.255 UH    100    0        0 ens4
100.64.0.0      0.0.0.0         255.255.0.0     U     0      0        0 netmaker
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-90d6161ad499
root@tw:~# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:51821           0.0.0.0:*               LISTEN      6113/netclient
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      5569/docker-proxy
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      5546/docker-proxy
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      861/sshd: /usr/sbin
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      453/systemd-resolve
tcp6       0      0 :::51821                :::*                    LISTEN      6113/netclient
tcp6       0      0 :::443                  :::*                    LISTEN      5575/docker-proxy
tcp6       0      0 :::80                   :::*                    LISTEN      5560/docker-proxy
tcp6       0      0 :::22                   :::*                    LISTEN      861/sshd: /usr/sbin
udp        0      0 127.0.0.53:53           0.0.0.0:*                           453/systemd-resolve
udp        0      0 10.140.0.3:68           0.0.0.0:*                           450/systemd-network
udp        0      0 127.0.0.1:323           0.0.0.0:*                           1544/chronyd
udp        0      0 0.0.0.0:51821           0.0.0.0:*                           -
udp6       0      0 ::1:323                 :::*                                1544/chronyd
udp6       0      0 :::51821                :::*                                -

SourceByrd's Weblog-https://note.t4x.org/basic/netmaker-manager-wireguard/