{"id":806,"date":"2023-08-10T10:40:17","date_gmt":"2023-08-10T10:40:17","guid":{"rendered":"http:\/\/192.168.0.142\/?p=806"},"modified":"2023-08-11T10:24:59","modified_gmt":"2023-08-11T10:24:59","slug":"kubernetes-the-hard-way-using-ansible-and-proxmox","status":"publish","type":"post","link":"http:\/\/192.168.0.142\/kubernetes-the-hard-way-using-ansible-and-proxmox\/","title":{"rendered":"Kubernetes The Hard Way using Terraform, Ansible and Proxmox"},"content":{"rendered":"\n<h3 class=\"wp-block-heading\">Introduction<\/h3>\n\n\n\n<p>The scripts are available in <a href=\"https:\/\/github.com\/rinoymjoseph\/k8s-hardway\">https:\/\/github.com\/rinoymjoseph\/k8s-hardway<\/a> repo.<\/p>\n\n\n\n<p>Kubernetes The Hard Way is a popular git hub <a href=\"https:\/\/github.com\/kelseyhightower\/kubernetes-the-hard-way\" data-type=\"link\" data-id=\"https:\/\/github.com\/kelseyhightower\/kubernetes-the-hard-way\">repo<\/a> which take the long route to ensure you understand each task required to bootstrap a Kubernetes cluster.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">1. Prerequisites<\/h3>\n\n\n\n<ul>\n<li>Proxmox server.<\/li>\n\n\n\n<li>An Ubuntu 22.04 template provisioned on Proxmox.<\/li>\n\n\n\n<li>Terraform. Please refer<strong> <a href=\"http:\/\/192.168.0.142\/installation-of-terraform-on-ubuntu-22-04\/\">Terraform installation on Ubuntu 22.04<\/a><\/strong> for installation of Terraform.<\/li>\n\n\n\n<li>A way to provision static ip addresses mapped to mac addresses (I used a home router).<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">2. System Diagram<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">Single Controller and Three Worker Nodes<\/h4>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" loading=\"lazy\" width=\"1024\" height=\"364\" src=\"http:\/\/192.168.0.142\/wp-content\/uploads\/2023\/08\/k8s-the-hard-way-system-diagram-1024x364.png\" alt=\"Kubernetes The Hard Way System Diagram\" class=\"wp-image-817\" srcset=\"http:\/\/192.168.0.142\/wp-content\/uploads\/2023\/08\/k8s-the-hard-way-system-diagram-1024x364.png 1024w, http:\/\/192.168.0.142\/wp-content\/uploads\/2023\/08\/k8s-the-hard-way-system-diagram-300x107.png 300w, http:\/\/192.168.0.142\/wp-content\/uploads\/2023\/08\/k8s-the-hard-way-system-diagram-768x273.png 768w, http:\/\/192.168.0.142\/wp-content\/uploads\/2023\/08\/k8s-the-hard-way-system-diagram.png 1120w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">3. Configuration<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">Infrastructure Configuration<\/h4>\n\n\n\n<p>A terraform script creates the machines required for the setup. The script is available in terraform\/five-nodes path. Script is using Telmate\/proxmox provider to create virtual machines in Proxmox server from an Ubuntu 22.04 machine template. So make sure that an Ubuntu 22.04 server template is available in Proxmox server.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">System configuration for individual nodes<\/h4>\n\n\n\n<ul>\n<li>Bootstrap, CPU &#8211; 1 core, RAM &#8211; 2 GB<\/li>\n\n\n\n<li>k8s-controller-1,  CPU &#8211; 2 core, RAM &#8211; 4 GB<\/li>\n\n\n\n<li>k8s-worker-1, CPU &#8211; 2 core, RAM &#8211; 4 GB<\/li>\n\n\n\n<li>k8s-worker-2, CPU &#8211; 2 core, RAM &#8211; 4 GB<\/li>\n\n\n\n<li>k8s-worker-3, CPU &#8211; 2 core, RAM &#8211; 4 GB<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Proxmox settings<\/h4>\n\n\n\n<p>Change proxmox settings in below code block of k8s-infra.tf <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\">provider \"proxmox\" {\n  # Configuration options\n  pm_api_url    =   \"https:\/\/192.168.0.111:8006\/api2\/json\"\n  pm_user       =   \"root@pam\"\n  pm_password   =   \"\"\n}<\/code><\/pre>\n\n\n\n<p>Change the parameter values for each node in the tf file such as name, vmid, macaddr etc. In this setup a static IP is provisioned when VMs are created. The mac addresses specified for each VM is mapped to an IP address in the router.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\"># Provision bootstrap\nresource \"proxmox_vm_qemu\" \"k8s-bootstrap\" {\n  name          = \"k8s-bootstrap\"\n  vmid          = 149\n  target_node   = \"proxmox\"\n  clone         = \"Ubuntu-Server-22.04\"\n  full_clone    = false\n  memory        = 2048\n  cores         = 1\n  network   {\n    model       = \"virtio\"\n    macaddr     = \"86:AC:41:71:70:7D\"\n    bridge      = \"vmbr0\"\n    firewall    = true\n  }\n}\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Ansible hosts.ini configuration<\/h4>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\">localhost ansible_host=127.0.0.1 ansible_user=rinoy ansible_ssh_pass=\"welcome123#\" ansible_sudo_pass=\"welcome123#\"\n\n[bootstrap]\nk8s-bootstrap ansible_host=192.168.0.149 ansible_user=rinoy ansible_ssh_pass=\"welcome123#\" ansible_sudo_pass=\"welcome123#\"\n\n[load_balancer]\nload_balancer ansible_host=192.168.0.151 ansible_user=rinoy ansible_ssh_pass=\"welcome123#\"\n\n[controllers]\nk8s-controller-1 ansible_host=192.168.0.151 ansible_user=rinoy ansible_ssh_pass=\"welcome123#\" ansible_sudo_pass=\"welcome123#\"\n\n[workers]\nk8s-worker-1 ansible_host=192.168.0.161 ansible_user=rinoy ansible_ssh_pass=\"welcome123#\" ansible_sudo_pass=\"welcome123#\"\nk8s-worker-2 ansible_host=192.168.0.162 ansible_user=rinoy ansible_ssh_pass=\"welcome123#\" ansible_sudo_pass=\"welcome123#\"\nk8s-worker-3 ansible_host=192.168.0.163 ansible_user=rinoy ansible_ssh_pass=\"welcome123#\" ansible_sudo_pass=\"welcome123#\"\n\n[cluster:children]\ncontrollers\nworkers\n\n[nodes:children]\nbootstrap\ncontrollers\nworkers<\/code><\/pre>\n\n\n\n<p>The package configuration is available in ansible_scripts\/inventory\/group_vars\/all.yml file<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"yaml\" class=\"language-yaml\">k8s_version: v1.23.0\ncfssl_version: 1.6.3\netcd_version: v3.5.1\ncni_plugins_version: v1.0.1\ncrio_version: 1.23\ncrio_os: xUbuntu_22.04\n<\/code><\/pre>\n\n\n\n<p>Certificate settings is also available in all.yml file<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"yaml\" class=\"language-yaml\"># Cerfificate settings\nTLS_C: \"HY\"\nTLS_L: \"Hyrule\"\nTLS_OU: \"Link\"\nTLS_ST: \"Akkala\"\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">4. Provisioning<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">Terraform script execution<\/h4>\n\n\n\n<p>Login to Terraform machine (as per the system diagram) and clone the <a href=\"https:\/\/github.com\/rinoymjoseph\/k8s-hardway.git\" data-type=\"link\" data-id=\"https:\/\/github.com\/rinoymjoseph\/k8s-hardway.git\">repo<\/a>.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\">link@hyrule:~$ git clone https:\/\/github.com\/rinoymjoseph\/k8s-hardway.git\nCloning into 'k8s-hardway'...\nremote: Enumerating objects: 550, done.\nremote: Counting objects: 100% (550\/550), done.\nremote: Compressing objects: 100% (296\/296), done.\nremote: Total 550 (delta 184), reused 492 (delta 129), pack-reused 0\nReceiving objects: 100% (550\/550), 159.36 KiB | 2.12 MiB\/s, done.\nResolving deltas: 100% (184\/184), done.\nlink@hyrule:~$ cd k8s-hardway\/\nlink@hyrule:~\/k8s-hardway$ ls\nansible_scripts  ansible_scripts.zip  docs  k8s-infra.md  LICENSE  README.md  terraform  tests  tests.md  two-nodes<\/code><\/pre>\n\n\n\n<p>Zip ansible_scripts folder using below command<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code class=\"\">zip -r ansible_scripts.zip ansible_scripts\/<\/code><\/pre>\n\n\n\n<p>cd to <strong>terraform\/five-nodes<\/strong> folder<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\">terraform init<\/code><\/pre>\n\n\n\n<p><em>Terminal output for reference<\/em><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\">link@hyrule:~\/k8s-hardway\/terraform\/five-nodes$ terraform init\n\nInitializing the backend...\n\nInitializing provider plugins...\n- Finding telmate\/proxmox versions matching \"2.9.14\"...\n- Finding latest version of hashicorp\/null...\n- Installing telmate\/proxmox v2.9.14...\n- Installed telmate\/proxmox v2.9.14 (self-signed, key ID A9EBBE091B35AFCE)\n- Installing hashicorp\/null v3.2.1...\n- Installed hashicorp\/null v3.2.1 (signed by HashiCorp)\n\nPartner and community providers are signed by their developers.\nIf you'd like to know more about provider signing, you can read about it here:\nhttps:\/\/www.terraform.io\/docs\/cli\/plugins\/signing.html\n\nTerraform has created a lock file .terraform.lock.hcl to record the provider\nselections it made above. Include this file in your version control repository\nso that Terraform can guarantee to make the same selections by default when\nyou run \"terraform init\" in the future.\n\nTerraform has been successfully initialized!\n\nYou may now begin working with Terraform. Try running \"terraform plan\" to see\nany changes that are required for your infrastructure. All Terraform commands\nshould now work.\n\nIf you ever set or change modules or backend configuration for Terraform,\nrerun this command to reinitialize your working directory. If you forget, other\ncommands will detect it and remind you to do so if necessary.<\/code><\/pre>\n\n\n\n<p>Validate the tf file<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\">terraform validate<\/code><\/pre>\n\n\n\n<p><em>Terminal output for reference<\/em><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\">link@hyrule:~\/k8s-hardway\/terraform\/five-nodes$ terraform validate\nSuccess! The configuration is valid.<\/code><\/pre>\n\n\n\n<p>Execute the script<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\">terraform apply --auto-approve<\/code><\/pre>\n\n\n\n<p>In proxmox ui machines appear and starts automatically<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img decoding=\"async\" loading=\"lazy\" width=\"253\" height=\"173\" src=\"http:\/\/192.168.0.142\/wp-content\/uploads\/2023\/08\/k8s-the-hard-way-system-machines.png\" alt=\"\" class=\"wp-image-865\"\/><\/figure><\/div>\n\n\n<h3 class=\"wp-block-heading\">5. Workflow<\/h3>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" loading=\"lazy\" width=\"761\" height=\"697\" src=\"http:\/\/192.168.0.142\/wp-content\/uploads\/2023\/08\/kubernetes-the-hard-way-workflow.png\" alt=\"Kubernetes The Hard Way Workflow\" class=\"wp-image-883\" srcset=\"http:\/\/192.168.0.142\/wp-content\/uploads\/2023\/08\/kubernetes-the-hard-way-workflow.png 761w, http:\/\/192.168.0.142\/wp-content\/uploads\/2023\/08\/kubernetes-the-hard-way-workflow-300x275.png 300w\" sizes=\"(max-width: 761px) 100vw, 761px\" \/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>Introduction The scripts are available in https:\/\/github.com\/rinoymjoseph\/k8s-hardway repo. Kubernetes The Hard Way is a popular git hub repo which take the long route to ensure you understand each task required to bootstrap a Kubernetes cluster. 1. Prerequisites 2. System Diagram Single Controller and Three Worker Nodes 3. Configuration Infrastructure Configuration A terraform script creates the &#8230; <a title=\"Kubernetes The Hard Way using Terraform, Ansible and Proxmox\" class=\"read-more\" href=\"http:\/\/192.168.0.142\/kubernetes-the-hard-way-using-ansible-and-proxmox\/\" aria-label=\"More on Kubernetes The Hard Way using Terraform, Ansible and Proxmox\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":885,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[67],"tags":[68,10,9],"_links":{"self":[{"href":"http:\/\/192.168.0.142\/wp-json\/wp\/v2\/posts\/806"}],"collection":[{"href":"http:\/\/192.168.0.142\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/192.168.0.142\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/192.168.0.142\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/192.168.0.142\/wp-json\/wp\/v2\/comments?post=806"}],"version-history":[{"count":53,"href":"http:\/\/192.168.0.142\/wp-json\/wp\/v2\/posts\/806\/revisions"}],"predecessor-version":[{"id":889,"href":"http:\/\/192.168.0.142\/wp-json\/wp\/v2\/posts\/806\/revisions\/889"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/192.168.0.142\/wp-json\/wp\/v2\/media\/885"}],"wp:attachment":[{"href":"http:\/\/192.168.0.142\/wp-json\/wp\/v2\/media?parent=806"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/192.168.0.142\/wp-json\/wp\/v2\/categories?post=806"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/192.168.0.142\/wp-json\/wp\/v2\/tags?post=806"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}