Kubernetes Cluster On Windows Server Worker Nodes
Kubernetes (k8s) is an open-source, cloud-native, container orchestration and management platform. It’s the go-to way to automate the deployment, scaling, and maintenance of containerised applications across different nodes. From service discovery to auto-restarts, and from resource allocation tracking to compute utilisation and scaling; a well-configured k8s cluster can manage a lot on its own.
Windows Server 2019 is the only Windows-based operating system that supports Kubernetes installation. Unfortunately, for Kubernetes to run on Windows, it still requires a Linux master.
Since the support is relatively new, pay special attention to the steps below.
What is a Kubernetes Cluster?
A Kubernetes cluster consists of a Master and at least one to several worker node(s). The Master is the virtual machine (VM) that administers all activities on your cluster. A node is a VM that serves as a worker machine in your k8s cluster to host running applications. We strongly recommend you only use VMs aka Cloud Servers to run Kubernetes, not system containers aka VPS, as these can cause issues with k8s.
HOSTAFRICA Kubernetes Cluster Diagram – Ubuntu Master and Windows Server workers
A node is comprised of the Kubelet, a container runtime, and the kube-proxy. The k8s installation’s three core modules: Kubelet, kubeadm, and kubectl are agents that control the node and communicate with the Kubernetes Master. Once they have been installed and other configurations done, you will be able to create your first k8s cluster. You can manage this cluster from the command line on your kubemaster node.
Every Kubernetes instance runs on top of a container runtime, which is software responsible for managing container operations. Containers in this case are not virtualised servers but rather a solution that packages code and dependencies to run a single application (service) in an isolated (containerised) environment, essentially disassociating applications from the host machine. The most popular and recommended one is Docker, and it’s the one we will use for the purpose of this guide. However, if you want to install a different underlying container runtime, you can harness the power of the Container Runtime Interface and use basically any runtime you want.
Kubernetes groups containers into pods, its most basic operational unit, which are basically just groups of containers running on the same node. Pods are connected over a network and share storage resources.
In order to connect your nodes or VMs and make them private, make sure to choose a hosting company who provides a Virtual Local Area Network (VLAN) with their VMs. We offer a VLAN add-on to our Cloud Servers for R200 per month.
• Multiple Windows Server 2019 VMs (Windows Cloud Servers) to house the worker nodes.
• A Linux VM (for this article we will use an Ubuntu VM for the Master node)
• Docker or any other container runtime.
• User with sudo or root privileges on every server.
Got CentOS VMs? Learn How to install Kubernetes and deploy a cluster with Docker on CentOS 7
Got Ubuntu VMs? Learn How to install Kubernetes and deploy a cluster with Docker on Ubuntu 18
How to install Kubernetes on Windows Server 2019 (worker node only)
Step 1. Install Docker on all VMs
Use our guide How to install Docker on Ubuntu 18 for your Ubuntu Master VM.
Then follow our guide How to install Docker on Windows Server 2019 for all your Windows worker VMs.
Step 2. Start Docker Service
If you haven’t already, make sure that the Docker service is installed. Open a PowerShell and enter the following command:
Start-Service docker
Step 3. Pull and tag an Image used by Kubernetes
Once docker is up and running, we need to pull and tag an image used by Kubernetes. Use the following commands:
docker image pull mcr.microsoft.com/windows/nanoserver:1809
docker image tag mcr.microsoft.com/windows/nanoserver:1809 microsoft/nanoserver:latest
Step 4. Download and extract Kubernetes binaries
Next up, we need to fetch, extract, and save the Kubernetes binaries. Use the following commands:
mkdir C:\k cd C:\k $ProgressPreference=’SilentlyContinue’ iwr -outf kubernetes-node-windows-amd64.tar.gz "https://dl.k8s.io/v1.15.1/kubernetes-node-windows-amd64.tar.gz" tar -xkf kubernetes-node-windows-amd64.tar.gz -C C:\k mv C:\k\kubernetes\node\bin\*.exe C:\k
Step 5. Set up binaries and install
Use the following snippet of code which will install and move the required files and binaries.
$NetworkMode="overlay"
$ClusterCIDR="10.244.0.0/16"
$KubeDnsServiceIP="10.96.0.10"
$ServiceCIDR="10.96.0.0/12"
$InterfaceName="Ethernet"
$LogDir="C:\k`logs"
$BaseDir = "c:\k"
$NetworkMode = $NetworkMode.ToLower()
$NetworkName = "vxlan0"
$GithubSDNRepository = 'Microsoft/SDN'
$helper = "c:\k\helper.psm1"
if (!(Test-Path $helper))
{
Start-BitsTransfer "https://raw.githubusercontent.com/$GithubSDNRepository/master/Kubernetes/windows/helper.psm1" -Destination c:\k\helper.psm1
}
ipmo $helper
$install = "c:\k\install.ps1"
if (!(Test-Path $install))
{
Start-BitsTransfer "https://raw.githubusercontent.com/$GithubSDNRepository/master/Kubernetes/windows/install.ps1" -Destination c:\k\install.ps1
}
powershell $install -NetworkMode "$NetworkMode" -clusterCIDR "$ClusterCIDR" -KubeDnsServiceIP "$KubeDnsServiceIP" -serviceCIDR "$ServiceCIDR" -InterfaceName "'$InterfaceName'" -LogDir "$LogDir"
You should see the following output:
How to add a Windows based Kubernetes Node to a Linux Cluster
Since the Kubernetes control plane still has components that are Linux-specific, it’s not possible for us to build an all-Windows Kubernetes cluster – yet. However, we will share the steps to add a Windows-based Kubernetes worker node to an existing Kubernetes cluster with an Ubuntu 18.09 master. Follow these steps:
Part 1. Set up the Ubuntu 18.09 Kubernetes Master node
Use our guide How to install Kubernetes and deploy a cluster with Docker on Ubuntu 18 and follow all the steps related to setting up the Master node. The worker nodes will be installed in the steps below.
Part 2. Configure and add Windows Kubernetes Node to the Cluster
Step 1. Copy Config File from Kubernetes Master
On your Ubuntu 18.09 VM, enter the following command and copy the output value
cat $HOME/.kube/config
Now, move over to the Windows VM and enter the following command to create a config file in the C:\k folder:
notepad C:\k\config
Paste the contents in the notepad, save and exit.
Note: Please make sure that you are not saving the file with a .txt extension. If necessary, change the extension from PowerShell using this command:
ren config.txt config
Step 2. Register node
Now that we have installed most of the Kubernetes binaries, we can proceed with registering the node. Run the following three commands:
powershell $BaseDir\start-kubelet.ps1 -RegisterOnly -NetworkMode $NetworkMode
ipmo C:\k\hns.psm1
RegisterNode
Step 3. Start Kubernetes services and join cluster
Copy the snippet below to start Kubernetes services on the Windows VM and add the Windows VM to the Ubuntu 18.09 cluster:
Note: You have to replace “102.130.119.93” with the IP of your machine. You can get that using the ipconfig command)
$KubeletSvc="kubelet"
$KubeProxySvc="kube-proxy"
$FlanneldSvc="flanneld2"
$Hostname=$(hostname).ToLower()
iwr -outf nssm.zip https://nssm.cc/ci/nssm-2.24-101-g897c7ad.zip
Expand-Archive nssm.zip
mv C:\k\nssm\nssm-2.24-101-g897c7ad\win64\*.exe C:\k
# register & start flanneld
.\nssm.exe install $FlanneldSvc C:\flannel\flanneld.exe
.\nssm.exe set $FlanneldSvc AppParameters --kubeconfig-file=c:\k\config --iface=102.130.119.93 --ip-masq=1 --kube-subnet-mgr=1
.\nssm.exe set $FlanneldSvc AppEnvironmentExtra NODE_NAME=$Hostname
.\nssm.exe set $FlanneldSvc AppDirectory C:\flannel
.\nssm.exe start $FlanneldSvc
# register & start kubelet
.\nssm.exe install $KubeletSvc C:\k\kubelet.exe
.\nssm.exe set $KubeletSvc AppParameters --hostname-override=$Hostname --v=6 --pod-infra-container-image=mcr.microsoft.com/k8s/core/pause:1.0.0 --resolv-conf="" --enable-debugging-handlers --cluster-dns=$KubeDnsServiceIP --cluster-domain=cluster.local --kubeconfig=c:\k\config --hairpin-mode=promiscuous-bridge --image-pull-progress-deadline=20m --cgroups-per-qos=false --log-dir=$LogDir --logtostderr=false --enforce-node-allocatable="" --network-plugin=cni --cni-bin-dir=c:\k\cni --cni-conf-dir=c:\k\cni\config
.\nssm.exe set $KubeletSvc AppDirectory C:\k
.\nssm.exe start $KubeletSvc
# register & start kube-proxy
.\nssm.exe install $KubeProxySvc C:\k\kube-proxy.exe
.\nssm.exe set $KubeProxySvc AppDirectory c:\k
GetSourceVip -ipAddress 102.130.119.93 -NetworkName $NetworkName
$sourceVipJSON = Get-Content sourceVip.json | ConvertFrom-Json
$sourceVip = $sourceVipJSON.ip4.ip.Split("/")[0]
.\nssm.exe set $KubeProxySvc AppParameters --v=4 --proxy-mode=kernelspace --feature-gates="WinOverlay=true" --hostname-override=$Hostname --kubeconfig=c:\k\config --network-name=vxlan0 --source-vip=$sourceVip --enable-dsr=false --cluster-cidr=$ClusterCIDR --log-dir=$LogDir --logtostderr=false
.\nssm.exe set $KubeProxySvc DependOnService $KubeletSvc
.\nssm.exe start $KubeProxySvc
To verify that you have successfully started kubelet and kubeproxy services on your Windows node, use the following command:
Get-Service kube*
You should see an output like this:
Now when you run kubectl get nodes on your Ubuntu 18.09 master node, you should see a new Windows node added:
To set the role for your worker node, use the following command:
sudo kubectl label node w-node1 node-role.kubernetes.io/worker=worker
Now you’re all set up.
Happy Hosting!
