Tech Giant: Project

How to setup AWS cli and access for aws account.

1.Create a AWS user.

Add in access group, suppose you want to give full permission like AdmistratorAccess do like as below.

2. create access key for that user

3. Install aws cli on your local computer, go to below doc and install according to your OS.

https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

4. Configure aws cli Access key

aws configure

Done now you are connected with your AWS account via AWS cli.

Now Install Terraform

https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli

after install terraform verify it

terraform --help

now Create 1 folder go inside that folder and start terraform work.

Let suppose you have created 1 terraform script as below, store it inside that folder.

script name suppose - aws_ec2_jenkins_docker_install.tf

provider "aws" {
  region = "us-east-2"  # Update with your desired region
}

resource "aws_key_pair" "jenkins_keypair" {
  key_name   = "jenkins-keypair"
  public_key = file("/root/.ssh/id_rsa.pub")  # Replace with the path to your public key
}

resource "aws_instance" "jenkins_instance" {
  ami           = "ami-03a0c45ebc70f98ea"  # Replace with the desired AMI ID
  instance_type = "t2.small"      # Replace with the desired instance type
  key_name      = aws_key_pair.jenkins_keypair.key_name
  vpc_security_group_ids = [aws_security_group.jenkins_sg.id]

  user_data = <<-EOF
    #!/bin/bash
    sudo apt-get update
    sudo apt-get install -y docker.io
    sudo docker pull jenkins/jenkins:lts
 
    sudo docker run -d -p 8080:8080 -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts

    sleep 30  # Wait for Jenkins to start

    jenkins_password=$(sudo docker exec $(sudo docker ps -q --filter "ancestor=jenkins/jenkins:lts") cat /var/jenkins_home/secrets/initialAdminPassword)
    echo "Jenkins initial admin password: $jenkins_password"
  EOF
}

resource "aws_eip" "jenkins_eip" {
  instance = aws_instance.jenkins_instance.id
}

resource "aws_security_group" "jenkins_sg" {
  name        = "jenkins-sg"
  description = "Security group for Jenkins"
  vpc_id      = "vpc-0d67054cd23a8f716"  # Replace with the desired VPC ID

  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_eip_association" "jenkins_eip_association" {
  instance_id   = aws_instance.jenkins_instance.id
  allocation_id = aws_eip.jenkins_eip.id
}

output "jenkins_login_password" {
  value = aws_instance.jenkins_instance.user_data
}

output "jenkins_url" {
  value = "http://${aws_eip.jenkins_eip.public_ip}:8080" 
}
 
 
 
 

now from inside that folder

run below cmd

terraform init - it will download all plugin according to your provider

terraform plan - before execution of script you should run plan that will show you infra that will create if you execute that script.

terraform apply - once you apply it will create whole infra you defined inside the script.

now you can go to aws and check.

terraform destroy - it will delete all infra created by terraform apply.

After jenkins started installed below plugins on jenkins

Docker Pipeline

GitLab Plugin

Amazon ECR plugin

Configure Gitlab in jenkins plugin

Dashboard -> Manage Jenkins-> Configure System

create pipeline in Jenkins

Gitlab connection

Pipeline - Pipeline script from SCM

repo - https://gitlab.com/manjeetyadav19/sample_microservice_cicd_withjenkins_deployoneks.git

Branch specifier */main

Explain repo code.

we are building a simple nodejs app, that will print a message on web URL - Hello World-

index.js

const http = require('http');

const server = http.createServer((req, res) => {
  res.statusCode = 200;
  res.setHeader('Content-Type', 'text/plain');
  res.end('Hello, World!\n');
});

const port = process.env.PORT || 3000;
server.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

build it and try on local 1st if you want to try.

npm init

it will create a package.json and package.lock.json file

 npm start and node index.js

 go and check on the browser http://localhost:3000

After you build the code, Write a docker file to create above app as microservice.

Dockerfile

# Use the official Node.js image as the base
FROM node:14

# Set the working directory
WORKDIR /app

# Copy package.json and package-lock.json to the working directory
COPY package*.json ./

# Install dependencies
RUN npm install

# Copy the web app code to the working directory
COPY . .

# Expose the port that the app will run on
EXPOSE 3000

# Start the app
CMD [ "node", "index.js" ]


 and if you want you can test as docker container also on local

docker build -t sampleapp .

docker run -d -p 3000:3000 sampleapp

docker ps    - you will see running container sampleapp

go and check the browser you should able to see

same hello world on browser.

Now create a Jenkinsfile for the CI and push image to ECR repository.

Jenkinsfile

pipeline {
    agent any
    environment {
        ECR_REGISTRY = "896757523510.dkr.ecr.us-east-2.amazonaws.com" // Replace with your ECR registry URL
        ECR_REPO = "sampleapp" // Replace with your ECR repository name
        IMAGE_TAG = "latest" // Replace with your desired image tag

    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Build Docker Image') {
            steps {
                script {
                    def dockerImage = docker.build("${ECR_REGISTRY}/${ECR_REPO}:${IMAGE_TAG}", "-f Dockerfile .")
                }
            }
        }


        stage('Push Docker Image to ECR') {
            steps {
                script {
                    def dockerWithRegistry = { closure ->
                        docker.withRegistry("https://${ECR_REGISTRY}", 'ecr:us-east-2:aws-access-key', closure)
                    }

                    dockerWithRegistry {
                        sh "docker push ${ECR_REGISTRY}/${ECR_REPO}:${IMAGE_TAG}"
                    }
                }
            }
        }
    }
}

For above jenkins file pls define below value

Create ECR repository also, as above.

ECR_REGISTRY = "896757523510.dkr.ecr.us-east-2.amazonaws.com" // Replace with your ECR registry URL
ECR_REPO = "sampleapp" // Replace with your ECR repository name
IMAGE_TAG = "latest" // Replace with your desired image tag
ecr:us-east-2:aws-access-key'
aws-access-key is a aws credential, pls create in jenkins credential

Push all code to gilab repo main branch.

configure agent on Jenkins


Step 1. Create Virtual Machine / or connect docker host machine
First, create a virtual machine for the Jenkins agent.

Step 2. Install Java
# apt update && apt install openjdk-8-jdk

You can check if Java is installed.
java -version

Step 3. Add New Jenkins User
# useradd -m -d /var/lib/jenkins/ jenkins


Step 4. Configure Jenkins Master Credentials
The master must have private (jenkins_id_rsa) and public (jenkins_id_rsa.pub) ssh keys.

jenkins@test-jenkins-vm:~/.ssh$ ls | grep jenkins
jenkins_id_rsa
jenkins_id_rsa.pub

If necessary, you can generate it by the command:
# ssh-keygen -b 2048 -t rsa

$ chmod 600 jenkins_id_rsa*

Step 5. Copy the SSH Key from Master to Agent
Connect to the Jenkins Agent and create a directory .ssh into the Jenkins user home directory.
# mkdir .ssh

Now in the .ssh directory, create the file authorized_keys and copy the contents of jenkins_id_rsa.pub into it.


Step 6. Configure Jenkins Master Credentials
Manage Jenkins - Manage Credential - Add Credential

Now choose the authentication method.

    Kind: SSH Username with a private key
    Scope: Global
    Username: jenkins
    Private key: Enter directly and paste the ‘jenkins_id_rsa’ private key of Jenkins user from the master server.

Step 7. Add New Slave Nodes
On the Jenkins dashboard, click the ‘Manage Jenkins’ menu, and click ‘Manage Nodes’.
Click the ‘New Node’.

Type the node name ‘test-jenkins-slave’, choose the ‘permanent agent’, and click ‘OK’.

Step 8. Edit Node Information Details.

Now type node information details.

Description: test-jenkins-slave node agent server
Remote root directory: /var/lib/jenkins
Labels: test-jenkins-slave
Launch method: Launch slave agent via SSH,
Host: ‘10.0.0.5’ (it’s my test-jenkins-slave external IP)
Authentication: using ‘Jenkins’ credential.

 After configuring agent on jenkins

Create EKS cluster

go to jenkins agent and login as jenkins user

create below yaml file 
cluster.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: my-cluster
  region: us-east-2

nodeGroups:
  - name: my-nodegroup
    instanceType: t3.small
    desiredCapacity: 2

#eksctl create cluster -f cluster.yaml

Note:- After EKS cluster you have created, you need to go jenkins agent and configure aws cli for credential over there

#aws eks update-kubeconfig --region us-west-2 --name my-cluster

#kubectl get nodes



user below pipeline to deployment.

pipeline {
    agent any
    environment {
        ECR_REGISTRY = "896757523510.dkr.ecr.us-east-2.amazonaws.com" // Replace with your ECR registry URL
        ECR_REPO = "sampleapp" // Replace with your ECR repository name
        IMAGE_TAG = "latest" // Replace with your desired image tag
        KUBECONFIG_PATH = '~/.kube/config' // Path to the kubeconfig file inside Jenkins container
        NAMESPACE = "default" // Replace with your target namespace
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Build Docker Image') {
            steps {
                script {
                    def dockerImage = docker.build("${ECR_REGISTRY}/${ECR_REPO}:${IMAGE_TAG}", "-f Dockerfile .")
                }
            }
        }

        stage('Push Docker Image to ECR') {
            steps {
                script {
                    def dockerWithRegistry = { closure ->
                        docker.withRegistry("https://${ECR_REGISTRY}", 'ecr:us-east-2:aws-access-key', closure)
                    }

                    dockerWithRegistry {
                        sh "docker push ${ECR_REGISTRY}/${ECR_REPO}:${IMAGE_TAG}"
                    }
                }
            }
        }

        stage('Deploy to Kubernetes') {
            steps {
                script {
                    // Copy the kubeconfig file to the workspace
           
                    sh "cp ${KUBECONFIG_PATH} kubeconfig.yaml"
                    //sh "cp KUBECONFIG kubeconfig.yaml"

                    
                    // Apply the deployment to the Kubernetes cluster
                    sh "kubectl --kubeconfig=kubeconfig.yaml apply -f sampleapp-deployment.yaml -n ${NAMESPACE}"
                }
            }
        }
    }
}

Tech Giant

Friday, June 2, 2023

Project

configure agent on Jenkins

Create EKS cluster

Note:- After EKS cluster you have created, you need to go jenkins agent and configure aws cli for credential over there

No comments:

Post a Comment