Multus 测试

一个pod，容器，要两块网卡，我以前也觉得很变态，吃饱没事撑着。当你真的实际场景，真的有这样的需求

对着官方文档，先来一遍，体验一下

• https://github.com/k8snetworkplumbingwg/multus-cni/blob/master/docs/quickstart.md
• 利用multus-cni和macvlan实现pod多网卡
• http://cifangzi.club/2021/06/k8s_installation/

我需要实现的就是下图的数据网络

我是使用kubekey来进行安装，用2个vm来做试验。使用calico的网络

# kubectl get nodes
NAME       STATUS   ROLES                         AGE     VERSION
master01   Ready    control-plane,master,worker   3h54m   v1.22.1
work01     Ready    worker                        3h54m   v1.22.1

下载multus代码

git clone https://github.com/k8snetworkplumbingwg/multus-cni.git && cd multus-cni
cat ./deployments/multus-daemonset.yml | kubectl apply -f -

没搞明白，为啥官方文档使用multus-daemonset-thick-plugin.yml，外面的文档都是：multus-daemonset.yml

如果顺利

# kubectl get pods -n kube-system | grep -i multus
kube-multus-ds-tdcf4                           1/1     Running   0          133m
kube-multus-ds-xlf4m                           1/1     Running   0          133m

macvlan-conf.yaml

master :eth0, 这个地方设置，应该是重点，如果机器有另外一块网卡，我应该可以考虑指向它。

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: macvlan-conf
spec:
  config: '{
      "cniVersion": "0.3.0",
      "type": "macvlan",
      "master": "eth0",
      "mode": "bridge",
      "ipam": {
        "type": "host-local",
        "subnet": "192.168.4.0/24",
        "rangeStart": "192.168.4.200",
        "rangeEnd": "192.168.4.216",
        "routes": [
          { "dst": "0.0.0.0/0" }
        ],
        "gateway": "192.168.4.1"
      }
    }'

根据自己需求，修改一下macvlan的网段

kubectl create -f macvlan-conf.yaml
kubectl get network-attachment-definitions
kubectl describe network-attachment-definitions macvlan-conf

验证

samplepod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: samplepod
  annotations:
    k8s.v1.cni.cncf.io/networks: macvlan-conf
spec:
  containers:
  - name: samplepod
    command: ["/bin/ash", "-c", "trap : TERM INT; sleep infinity & wait"]
    image: alpine

kubectl create -f samplepod.yaml
kubectl exec -it samplepod -- ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
4: eth0@if19: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1440 qdisc noqueue state UP 
    link/ether c6:c1:90:ac:c4:e4 brd ff:ff:ff:ff:ff:ff
    inet 10.233.106.10/32 brd 10.233.106.10 scope global eth0
       valid_lft forever preferred_lft forever
5: net1@tunl0: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP 
    link/ether 82:84:7e:da:6f:b8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.4.200/24 brd 192.168.4.255 scope global net1
       valid_lft forever preferred_lft forever

查看一下pod

# kubectl describe pod samplepod
Name:         samplepod
Namespace:    default
Priority:     0
Node:         master01/192.168.20.21
Start Time:   Wed, 24 Nov 2021 16:48:04 +0800
Labels:       <none>
Annotations:  cni.projectcalico.org/containerID: 75777c27377709cfba50d9f8ef59209869c26d0707f0ad38343351eae904d16a
              cni.projectcalico.org/podIP: 10.233.106.10/32
              cni.projectcalico.org/podIPs: 10.233.106.10/32
              k8s.v1.cni.cncf.io/network-status:
                [{
                    "name": "k8s-pod-network",
                    "ips": [
                        "10.233.106.10"
                    ],
                    "default": true,
                    "dns": {}
                },{
                    "name": "default/macvlan-conf",
                    "interface": "net1",
                    "ips": [
                        "192.168.4.200"
                    ],
                    "mac": "82:84:7e:da:6f:b8",
                    "dns": {}
                }]
              k8s.v1.cni.cncf.io/networks: macvlan-conf
              k8s.v1.cni.cncf.io/networks-status:
                [{
                    "name": "k8s-pod-network",
                    "ips": [
                        "10.233.106.10"
                    ],
                    "default": true,
                    "dns": {}
                },{
                    "name": "default/macvlan-conf",
                    "interface": "net1",
                    "ips": [
                        "192.168.4.200"
                    ],
                    "mac": "82:84:7e:da:6f:b8",
                    "dns": {}
                }]
Status:       Running
IP:           10.233.106.10
IPs:
  IP:  10.233.106.10