How to Create a Kubernetes Cluster with AWS CLI

Elastic Kubernetes Service (EKS) is a managed Kubernetes service that is hosted on AWS.

The main reason for using EKS is to remove the burden of managing pods, nodes, etc. Running Kubernetes in AWS currently requires a great deal of technical expertise and often falls outside the wheelhouse of many organizations. With EKS, the required infrastructure is managed by Amazon's "in-house" team, leaving users with a fully managed Kubernetes engine that can be used either via an API or standard kubectl tooling.

EKS will support all Kubernetes features, including namespaces, security settings, resource quotas & tolerations, deployment strategies, autoscalers, and more. EKS will allow you to run your own control plane, but also integrates with AWS IAM so you can maintain your own access control to the API.

EKS was built on top of Amazon's existing "Kubernetes—as-a-Service" solution called Elastic Container Service for Kubernetes (EKS) is an AWS-managed service that simplifies the deployment, management, and operation of Kubernetes clusters in the AWS Cloud.

If you are running Kubernetes on AWS, you are responsible for managing the control plane (i.e. master nodes & worker nodes). You also have to make sure that api-server is highly available and fault-tolerant, etc.

EKS took the burden of managing the control plane away from you, by doing that, you are now able to focus on running your Kubernetes workloads. It is most commonly used for stateless applications like microservices since the control plane is managed by Amazon (EKS).

In this guide, we will leam how to create a Kubernetes cluster on AWS with EKS. You will learn how to create an administrative user for your Kubernetes cluster. You will also learn how to deploy an app to the cluster. Finally, you will test your cluster to ensure that everything is working properly.

Let's get started!

Prerequisites

• An AWS account.
• The article assumes that you are familiar with Kubernetes and AWS. If you are not, please take some time to go through the documentation on both before starting this guide.

Creating an Admin User with Permissions

Let's start with creating an admin user for your cluster.

1. Log in to your AWS console and go to IAM. Click Users > Add Users.

2. On the next screen, provide a user name like admin. Select Access key - Programmatic access. Click on Next: Permissions

3. On the next screen, select Attach existing policies directly. Click on AdministratorAccess. Click on Next: Tags.

The AdministratorAccess policy is a built-in policy with Amazon Elastic Container Service (ECS). It provides full access to all ECS resources and all actions in the ECS console. The main benefit of this policy is that we do not need to create or manage an additional user with extra privileges for accessing the AWS EKS service.

Your admin user can create EC2 instances, CloudFormation stacks, S3 buckets, etc. You should be very careful about who you give this kind of access to.

3. On the next screen, click on Next: Review

4. On the next screen, click on Create user.

5. On the next screen, you will get a green Success message. The Access key ID and
Secret access keys are also displayed in this screen. You will need these keys to configure your CLI tools later, so note down these keys to somewhere else.

Creating an EC2 Instance

Now that you have created the administrative user , let's create an EC2 instance to use as your Kubernetes master node.

1. Type EC2 into the search box. Click on the EC2 link. Click on Launch instance.

2. Select the Amazon Linux 2 AMI (HVM) for your EC2 instance. We will be using this Amazon Linux AMI to make it easy to install Kubernetes and other needed tools later, such as: kubectl!, docker, ect. 

3. On the next screen, click Next: Configure Instance Details.

3. On the next screen, enable the Auto-assign Public IP option. Since the server is inside a private subnet, it won't be accessible externally. You can give your servers public IP addresses by associating an Elastic IP address with the instance. By doing this, your EC2 and ELK is accessible. Click on Next: Storage.

3. On the next screen, click on Next: Add Tags > Next: Configure Security Group.

4. On the next screen, click on Review and Launch > Launch.

5. A key pair dialog will appear. Hit Create a new key pair. Give it a name, then download and store the .pem file in a secure location. Click Launch instance.

Configuring the Command Line Tools

Now that you have created an EC2 instance, you need to install the client for it. In AWS terms, a client is a command-line tool that allows you to manage cloud objects. In this section, you will learn how to configure the Command Line Interface (CLI) tools.

1. Navigate to your EC2 dashboard. You should see your new EC2 instance running. If not, your instance might take its first boot, wait 5 minutes and try again. Once your instance is running, click Connect.

2. On the next screen, click on Connect.

You will be taken to an interactive SSH session on your browser. SSH allows you to securely connect and operate on a remote server. The interactive SSH session will let us install the command line tools for EKS and Kubernetes directly onto your EC2 instance.

Once you log in the SSH session, the first thing you will need to do is to check for your aws-cli version. This is to make sure that you are using the latest version of AWS CLI. The AWS CLI is used to configure, manage and work with your cluster.

If your version is outdated, you may encounter some problems and errors during the cluster creation process. If your version is below 2.0, then you will need to upgrade it.

3. Run the following command to check your CLI version.

aws --version

As you can see in the output below, we are running version 1.18.147 of aws-cli, which is very out of date. Let's upgrade the CLI to the latest available version, which is v2+ at the time of writing this.

4. Run the command below to download the latest available version of AWS CLI to your EC2 instance. curl will download your file from the given url, -o will name it as you choose, and "awscli-exe-linux-x86_64.zip" is the file to be downloaded

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

5. Once it finishes downloading, run the command below to extract the contents of your downloaded file to the current directory.

unzip awscliv2.zip

6. Next, run the which aws command to set your link for the latest version of AWS CLI. This command will let you know where in your environment's PATH it can be found, so that you can run it from any directory.

which aws

As you can see in the output below, the outdated AWS CLI is located at /usr/bin/aws.

7. Now you need to configure your aws-cli by running an update command with some parameters. The first parameter ./aws/install will help us install AWS CLI to the current directory. The second parameter --bin-dir tells where in your environment's PATH the AWS CLI will be located, and the third parameter --install-dir is a path relative to bin-dir. This command will ensure that all of your paths are up to date.

sudo ./aws/install --bin-dir /usr/bin --install-dir /usr/bin/aws-cli --update

8. Rerun the aws --version command to make sure that you are using the latest version.

aws --version

You should see the currently installed AWS CLI version. As you can see in the output below, we are now using v2.4.7 of AWS CLI. This is the latest version and will not give you any problems while configuring the next steps.

9. Now that your environment is configured properly, it's time for you to configure which AWS account you want to communicate with through the AWS CLI. Run the following command to list your currently configured account environment variables with the alias you want to use along with it.

aws configure

This will show you all of your AWS account environment variables that are currently configured. You should see something like this in the output below. You need to set up some configuration parameters in order for the AWS CLI to communicate with your necessary accounts. Run the command below, which will take you through a configuration wizard to set up your AWS account.

• AWS Access Key ID [None]: Enter the AWS access key that you noted earlier.
• AWS Secret Access Key [None]: Enter the AWS Secret Access Key that you noted earlier.
• You also need to specify the default region name where your EKS cluster will be. You should choose an AWS region where your desired EKS cluster will be and which is closest to you. In this tutorial, we chose us-east-1 due to its geographical location close to us and ease of use for the next steps in the tutorial.
• Default output format [None]: Enter json as your default output format because it will be very useful for us to view the configuration files later.

Now that you have set up your AWS CLI tools. It's time to configure the Kubernetes CLI tool named kubectl in your environment so that you can interact with your EKS cluster.

Kubectl is the command-line interface for Kubernetes. With Kubectl, you can manage applications running on Kubernetes clusters. Kubectl is not installed by default on Linux and MacOS systems. You can install Kubectl on other systems by following the instructions on the Kubernetes website.

10. Run the command below to download the kubectl binary. A binary is a computer file with the extension ".bin", which is executable only on certain types of computers. It's an easy way for disparate types of computers to share files.  We use kubectl binary because the kubectl binary is platform agnostic. It will work on any system that can run a Unix-like operating system, including Linux and Mac OS.

curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.16.8/2020-04-16/bin/linux/amd64/kubectl

11. Run the chmod command below to make the kubectl binary executable. The chmod command is a Unix and Linux command used to change file or directory access permissions. The Linux chmod command uses the octal numeral system to specify the permissions for each user. Kubectl can now be used on your local machine.

chmod +x ./kubectl

12. Run the command below to create a kubectl directory in your $HOME/bin folder and copy the kubectl binary to it. The mkdir -p $HOME/bin command creates a bin sub-directory inside your home directory. The mkdir command is used to create new directories or folders. The -p option tells the mkdir command to automatically create any necessary parent directories for the new directory. The $HOME/bin is an environment variable that stores your home directory path. Every Linux user has the $HOME/bin directory in their file system. The && construct is called a logical AND operator. It is used to group commands together so that more than one command can be executed at once. The && construct is not necessary for this command to work, but it's there as a best practice.

The cp ./kubectl $HOME/bin/kubectl command copies the local kubectl binary file into your kubectl directory and renames the file to kubectl. Finally, the export command does what it says - it exports an environment variable into the shell's memory so that it can be used by any program run from this shell. In our case, we need to tell kubectl where our kubectl directory is so that it can find the kubectl binary.

mkdir -p $HOME/bin && cp ./kubectl $HOME/bin/kubectl && export PA