Category: Archiwum

Deploy kubernetes cluster in minutes with k3s

by Dawid Rycerz | Mar 23, 2020 | Archiwum

K3S is a lightweight and certified Kubernetes distribution, perfect for run development environments, CI/CD and IoT. It works very well with the ARM architecture. It’s packaged to a single binary that makes deployment and setup easy.

The default installation comes with:

embedded SQLite
containerd
flannel
coredns
traefik (ingress controller)
local path provisioner
service load balancer

All those features are more than enough for most use-cases. Of course, installation is fully configurable, so if you want etcd, docker, Nginx ingress and so on – go to official documentation page https://rancher.com/docs/k3s/latest/en/ .

Default installation

curl -sfL https://get.k3s.io | sh -

and that’s it… After a few seconds, you should see that the master node is ready with k3s kubectl get node.

You will find config.yaml for kubernetes in /etc/rancher/k3s/k3s.yaml

Advanced installation with nginx-ingress, metallb and cert-manager

Deploy k3s without traefik and servicelb

curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" INSTALL_K3S_EXEC=" --no-deploy=servicelb --no-deploy=traefik" sh -s -

Get config from /etc/rancher/k3s/k3s.yaml and put into ~/.kube/config on your machine

Install helm

curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

Add helm repo and update

helm repo add stable https://kubernetes-charts.storage.googleapis.com/
helm repo update

Install metallb. Range of available ip addresses for loadbalancer are set in configInline.address-pools[0].addresses[0].

helm install --namespace=kube-system 
  --set configInline.address-pools[0].name=default 
  --set configInline.address-pools[0].protocol=layer2 
  --set configInline.address-pools[0].addresses[0]=192.168.1.10-192.168.1.50 
  metallb stable/metallb

Install nginx-ingress

helm install --namespace=kube-system 
    --set rbac.create=true 
    nginx-ingress stable/nginx-ingress

Install and configure certmanager for lets-encrypt

kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.14/deploy/manifests/00-crds.yaml
helm install --namespace=kube-system cert-manager stable/cert-manager
cat &lt;&lt;EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: ACME_EMAIL
    privateKeySecretRef:
      name: letsencrypt
    solvers:
      - dns01:
          cloudflare:
            email: CLOUDFLARE_EMAIL
            apiKeySecretRef:
              name: cloudflare-apikey-secret
              key: CLOUDFLARE_APIKEY
        selector:
          dnsNames:
            - 'DOMAIN_NAME'
EOF

Where:

ACME_EMAIL – some email in yor domain (will be used to limit number of requests to acme)
CLOUDFLARE_EMAIL – email of your cloudflare account
CLOUDFLARE_APIKEY – api key to your cloudflare acccount
DOMAIN_NAME – domain that will be used in lets-encrypt certs (eg.: *.rycerz.co)

Important disclaimer

Don’t deploy everything in kube-system if it’s not test/development cluster. Be safe.

Debug Docker Network

by Dawid Rycerz | Feb 11, 2020 | Archiwum

What to do if docker image have installed no network utilities ("dig", "tcpdump", "nc", etc.) or even "shell"? We could use "namespaces" to enter one docker container isolated environment with another instead of trying to install those tools into running docker container.

Usecase

Let’s imagine following use-case: a "server" ("portainer") that is docker container without shell and a client ("busybox") which sends a http requests. Both "server" and "client" are in isolated network "backend".

Lab setup

At first create a isolated backend network, start "portainer" image as our server:

$ docker network create backend
$ docker run -d --net backend --name=portainer -v /var/run/docker.sock:/var/run/docker.sock portainer/portainer-ce

Next create our client container. Keep session working for later.

$ docker run -d --rm --net backend busybox

Capture packets:

Now we’ll try to capture packets on "portainer" container.

In one console spawn nicolaka/netshoot container in "portainer" network namespace and start tcpdump.

$ docker run -it --rm --net container:portainer nicolaka/netshoot bash
# tcpdump

On the busybox session run wget:

/ # wget http://portainer:9000

Console with "tcpdump" should show request from busybox client.

20:37:34.779034 ARP, Request who-has 6c9561758f20 tell 9dbe9ea2dbf9.backend, length 28
20:37:34.779044 ARP, Reply 6c9561758f20 is-at xx:xx:xx:xx:xx:xx (oui Unknown), length 28
20:37:34.779097 IP 9dbe9ea2dbf9.backend.46548 > 6c9561758f20.9000: Flags [S], seq 4285761690, win 64240, options [mss 1460,sackOK,TS val 4087033204 ecr 0,nop,wscale 7], length 0
20:37:34.779141 IP 6c9561758f20.9000 > 9dbe9ea2dbf9.backend.46548: Flags [S.], seq 325737431, ack 4285761691, win 65160, options [mss 1460,sackOK,TS val 505550796 ecr 4087033204,nop,wscale 7], length 0
20:37:34.779205 IP 9dbe9ea2dbf9.backend.46548 > 6c9561758f20.9000: Flags [.], ack 1, win 502, options [nop,nop,TS val 4087033205 ecr 505550796], length 0
20:37:34.779311 IP 9dbe9ea2dbf9.backend.46548 > 6c9561758f20.9000: Flags [P.], seq 1:78, ack 1, win 502, options [nop,nop,TS val 4087033205 ecr 505550796], length 77
20:37:34.779327 IP 6c9561758f20.9000 > 9dbe9ea2dbf9.backend.46548: Flags [.], ack 78, win 509, options [nop,nop,TS val 505550796 ecr 4087033205], length 0
20:37:34.780354 IP 6c9561758f20.9000 > 9dbe9ea2dbf9.backend.46548: Flags [P.], seq 1:306, ack 78, win 509, options [nop,nop,TS val 505550797 ecr 4087033205], length 305
20:37:34.780417 IP 9dbe9ea2dbf9.backend.46548 > 6c9561758f20.9000: Flags [.], ack 306, win 501, options [nop,nop,TS val 4087033206 ecr 505550797], length 0
20:37:34.780512 IP 6c9561758f20.9000 > 9dbe9ea2dbf9.backend.46548: Flags [.], seq 306:7546, ack 78, win 509, options [nop,nop,TS val 505550797 ecr 4087033206], length 7240
20:37:34.780555 IP 9dbe9ea2dbf9.backend.46548 > 6c9561758f20.9000: Flags [.], ack 7546, win 479, options [nop,nop,TS val 4087033206 ecr 505550797], length 0
20:37:34.780573 IP 6c9561758f20.9000 > 9dbe9ea2dbf9.backend.46548: Flags [.], seq 7546:14786, ack 78, win 509, options [nop,nop,TS val 505550797 ecr 4087033206], length 7240
20:37:34.780608 IP 9dbe9ea2dbf9.backend.46548 > 6c9561758f20.9000: Flags [.], ack 14786, win 446, options [nop,nop,TS val 4087033206 ecr 505550797], length 0
20:37:34.780631 IP 6c9561758f20.9000 > 9dbe9ea2dbf9.backend.46548: Flags [P.], seq 14786:23486, ack 78, win 509, options [nop,nop,TS val 505550797 ecr 4087033206], length 8700
20:37:34.780665 IP 9dbe9ea2dbf9.backend.46548 > 6c9561758f20.9000: Flags [.], ack 23486, win 407, options [nop,nop,TS val 4087033206 ecr 505550797], length 0
20:37:34.780810 IP 6c9561758f20.9000 > 9dbe9ea2dbf9.backend.46548: Flags [F.], seq 23486, ack 78, win 509, options [nop,nop,TS val 505550797 ecr 4087033206], length 0
20:37:34.781350 IP 9dbe9ea2dbf9.backend.46548 > 6c9561758f20.9000: Flags [F.], seq 78, ack 23487, win 501, options [nop,nop,TS val 4087033207 ecr 505550797], length 0
20:37:34.781375 IP 6c9561758f20.9000 > 9dbe9ea2dbf9.backend.46548: Flags [.], ack 79, win 509, options [nop,nop,TS val 505550798 ecr 4087033207], length 0
20:37:39.835649 ARP, Request who-has 9dbe9ea2dbf9.backend tell 6c9561758f20, length 28
20:37:39.835708 ARP, Reply 9dbe9ea2dbf9.backend is-at xx:xx:xx:xx:xx:xx (oui Unknown), length 28

Run commands in namespace from a host

For more powerful debugging we can run commands in container namespace from host.

On the docker host just run to spawn "wireshark" inside portainer namespace:

$ export CLIENT_PID=`docker inspect  --format '{{ .State.Pid }}' portainer`
$ sudo nsenter -t $CLIENT_PID -n wireshark

Configure Wireguard VPN

by Dawid Rycerz | Feb 10, 2020 | Archiwum

Wireguard is fast, simple (around 4k lines of code) and secure VPN. From my perspective as a user, a configuration is as simple as in SSH.

Installation

Add repository and install package (for other systems go to official docs)

add-apt-repository ppa:wireguard/wireguard
apt-get update
apt-get install -y wireguard

Ensure that you enabled forwarding in sysctl.

echo "net.ipv4.ip_forward = 1" &gt; /etc/sysctl.d/wg.conf
echo "net.ipv6.conf.all.forwarding = 1" &gt;&gt; /etc/sysctl.d/wg.conf
sysctl --system

Configuration

Create server and client keys

wg genkey | tee server.private.key | wg pubkey &gt; server.public.key
wg genkey | tee client.private.key | wg pubkey &gt; client.public.key

touch /etc/wireguard/wg0.conf and put config for VPN interface:

 [Interface]
 Address=/24
 PrivateKey = 
 ListenPort = 51820
 PostUp = iptables -t nat -A POSTROUTING -o  -j MASQUERADE; ip6tables -t nat -A POSTROUTING -o  -j MASQUERADE
 PostDown = iptables -t nat -D POSTROUTING -o  -j MASQUERADE; ip6tables -t nat -D POSTROUTING -o  -j MASQUERADE

 [Peer]
 PublicKey = 
 AllowedIPs = /32

Example:

 [Interface]
 Address=192.168.101.1/24
 PrivateKey = mHjrLYUTKbrGqJViVOHfQX9dN0Sn49gJNoof68nbJHA=
 ListenPort = 51820
 PostUp = iptables -t nat -A POSTROUTING -o enp0s3 -j MASQUERADE; ip6tables -t nat -A POSTROUTING -o enp0s3 -j MASQUERADE
 PostDown = iptables -t nat -D POSTROUTING -o enp0s3 -j MASQUERADE; ip6tables -t nat -D POSTROUTING -o enp0s3 -j MASQUERADE

 [Peer]
 PublicKey = XKT1Ctj5b+gjXc1gMtOdxNEpc9UUM2TsXaFdAyABd3w=
 AllowedIPs = 192.168.101.2/32

Run VPN server with wg-quick up

Create config for client

[Interface]
Address = /24
PrivateKey = 
ListenPort = 21841
DNS = ,

[Peer]
PublicKey = 
Endpoint = :51820
AllowedIPs = 0.0.0.0/0

Example:

[Interface]
Address = 192.168.101.2/32
PrivateKey = 0AQI65ehzszpXf9f2FWEABX90PX+gv5DJH3/mkZ/eW8=
ListenPort = 21841
DNS = 1.1.1.1,1.1.0.0

[Peer]
PublicKey = ccDLW5zKussL3ejxMqWpx1uZMfN09bkGAirCWXZWp0s=
Endpoint = 192.168.1.5:51820
AllowedIPs = 0.0.0.0/0

Install client software https://www.wireguard.com/install/ and paste client config