flask-hello-world-devops-project
Build and deploy code a simple flask application using Jenkins and Kubernetes
In this project, we are going to build a simple CI/CD pipeline from scratch using tools like Flask, Docker, Git, Github, Jenkins and Kubernetes.
Prerequisites
- Python
- Flask
- Docker
- Git and Github
- Jenkins
- Kubernetes
- Linux machine
Steps in the CI/CD pipeline
- Create a "Hello world" Flask application
- Write basic test cases
- Dockerise the application
- Test the code locally by building docker image and running it
- Create a github repository and push code to it
- Start a Jenkins server on a host
- Write a Jenkins pipeline to build, test and push the docker image to Dockerhub.
- Set up Kubernetes on a host using Minikube
- Create a Kubernetes deployment and service for the application.
- Use Jenkins to deploy the application on Kubernetes
Project structure
- app.py - Flask application which will print "Hello world" when you run it
- test.py - Test cases for the application
- requirements.txt - Contains dependencies for the project
- Dockerfile - Contains commands to build and run the docker image
- Jenkinsfile - Contains the pipeline script which will help in building, testing and deploying the application
- deployment.yaml - Kubernetes deployment file for the application
- service.yaml - Kubernetes service file for the application
Create a project repository on Github
Login to your github account and create a new repository. Do make sure that you have given a unique name for the repository. It's good to add a README file, Python .gitignore template and choose a licence.
Click on "create repository". Now, the repository is created.
Clone the repository on your system
Go to your Github repository. Click on the "Code" section and note down the HTTPS url for the project.
Open terminal on your local machine(Desktop/laptop) and run the below commands.
git clone https://github.com/codophobia/flask-hello-world-devops-project.git # Replace the url with the url of your project
cd flask-hello-world-devops-project
Run ls command and you should be able to see a local copy of the github repository.
Set up virtual Python environment
Setting up a virtual Python environment will help in testing the code locally and also collecting all the dependencies.
python3 -m venv venv # Create the virtual env named venv
source venv/bin/activate # Activate the virtual envCreate a Flask application
Install the flask module.
pip install flaskOpen the repository that you have just cloned in your favourite text editor. Create a new file named "app.py" and add the below code.
from flask import Flask
import os
app = Flask(__name__)
@app.route("/")
def hello():
return "Hello world!!!"
if __name__ == "__main__":
port = int(os.environ.get("PORT", 5000))
app.run(debug=True, host='0.0.0.0', port=port)The above code when run will start a web server on port number 5000. You can test the code by running it.
python app.pyYou should see the output below after running the above command.
Open your browser and visit . You should see "Hello world" printed on the browser.
Write test cases using pytest
Install pytest module. We will use pytest for testing.
pip install pytestCreate a new file named "test.py" and add a basic test case.
from app import app
def test_hello():
response = app.test_client().get('/')
assert response.status_code == 200
assert response.data == b'Hello world!!!'Run the test file using pytest.
pytest test.pyRun code quality tests using flake
It's always recommended to write quality code with proper coding standards, proper code formatting and code with no syntax errors.
Flake8 can be used to check the quality of the code in Python.
Install the flake8 module.
pip install flake8Run flake8 command.
flake8 --exclude venv # Ignore files in venv for quality checkIf you do not see any output, it means that everything is alright with code quality.
Try removing the spaces between commas in the app.py and then run flake8 command again. You should see the following errors.
Add space again and you will not see the error now.
Dockerise the application
Install docker on your system. Follow https://docs.docker.com/get-docker/
Create a file named "Dockerfile" and add the below code.
FROM python:3.6
MAINTAINER Shivam Mitra "[email protected]" # Change the name and email address
COPY app.py test.py /app/
WORKDIR /app
RUN pip install flask pytest flake8 # This downloads all the dependencies
CMD ["python", "app.py"]
Build the docker image.
docker build -t flask-hello-world .Run the application using docker image.
docker run -it -p 5000:5000 flask-hello-worldRun test case
docker run -it flask-hello-world pytest test.pyRun flake8 tests
docker run -it flask-hello-world flake8You can verify if the application is running by opening the page in the browser.
Push the image to dockerhub. You will need an account on docker hub for this.
docker login # Login to docker hub
docker tag flask-hello-world shivammitra/flask-hello-world # Replace <shivammitra> with your docker hub username
docker push shivammitra/flask-hello-worldPush the code to github
Till now, we haven't pushed the code to our remote repository. Let's try some basic git commands to push the code.
git add .
git commit -m "Add flask hello world application, test cases and dockerfile"
git push origin mainIf you go to the github repository, you should see the changes.
Install Jenkins
In this example, we will be installing Jenkins on Ubuntu 20.04 Linux machine. If you have a different Linux distribution, follow steps mentioned at https://www.jenkins.io/doc/book/installing/linux/
Run the following commands on the server.
# Install jenkins
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo tee \
/usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
/etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt install openjdk-11-jre
sudo apt-get install jenkins
# Install docker and add jenkins user to docker group
# Installing docker is important as we will be using jenkins to run docker commands to build and run the application.
sudo apt install docker.io
sudo usermod -aG docker jenkins
sudo service jenkins restartOpen your browser and visit http://127.0.0.1:8080. If you have installed Jenkins on a Aws/Azure/GCP virtual machine, use the public address of the VM in place of localhost. If you are not able to access the cloud, make sure that port 8080 is added to the inbound port.
Copy the admin password from the path provided on the jenkins homepage and enter it. Click on "Install suggested plugins". It will take some time to install the plugins.
Create a new user.
You should now see the jenkins url. Click next and you should see the "Welcome to jenkins" page now.
Create a Jenkins pipeline
We will now create a Jenkins pipeline which will help in building, testing and deploying the application.
Click on "New Item" on the top left corner of the homepage.
Enter a name, select "Pipeline" and click next.
We now need to write a pipeline script in Groovy for building, testing and deploying code.
Enter the below code in the pipeline section and click on "Save".
pipeline {
agent any
environment {
DOCKER_HUB_REPO = "shivammitra/flask-hello-world"
CONTAINER_NAME = "flask-hello-world"
}
stages {
stage('Checkout') {
steps {
checkout([$class: 'GitSCM', branches: [[name: '*/main']], extensions: [], userRemoteConfigs: [[url: 'https://github.com/codophobia/flask-hello-world-devops-project']]])
}
}
stage('Build') {
steps {
echo 'Building..'
sh 'docker image build -t $DOCKER_HUB_REPO:latest .'
}
}
stage('Test') {
steps {
echo 'Testing..'
sh 'docker stop $CONTAINER_NAME || true'
sh 'docker rm $CONTAINER_NAME || true'
sh 'docker run --name $CONTAINER_NAME $DOCKER_HUB_REPO /bin/bash -c "pytest test.py && flake8"'
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
sh 'docker stop $CONTAINER_NAME || true'
sh 'docker rm $CONTAINER_NAME || true'
sh 'docker run -d -p 5000:5000 --name $CONTAINER_NAME $DOCKER_HUB_REPO'
}
}
}
}
Click on the "Build Now" button at the left of the page.
The pipelines should start running now and you should be able to see the status of the build on the page.
If the build is successful, you can visit http://127.0.0.1:5000 and you should see "Hello world" on the browser. If you have installed Jenkins on a Aws/Azure/GCP virtual machine, use the public address of the VM in place of localhost. If you are not able to access the cloud, make sure that port 5000 is added to the inbound port.
Pipeline script from SCM
We can also store the Jenkins pipeline code in our github repository and ask Jenkins to execute this file.
Go to the "flask-hello-world" pipeline page and click on "Configure".
Change definition from "Pipeline script" to "Pipeline script from SCM" and fill details on SCM and github url. Save the pipeline.
Now, create a new file named "Jenkins" in our local code repository and add the below pipeline code.
pipeline {
agent any
environment {
DOCKER_HUB_REPO = "shivammitra/flask-hello-world"
CONTAINER_NAME = "flask-hello-world"
}
stages {
/* We do not need a stage for checkout here since it is done by default when using the "Pipeline script from SCM" option. */
stage('Build') {
steps {
echo 'Building..'
sh 'docker image build -t $DOCKER_HUB_REPO:latest .'
}
}
stage('Test') {
steps {
echo 'Testing..'
sh 'docker stop $CONTAINER_NAME || true'
sh 'docker rm $CONTAINER_NAME || true'
sh 'docker run --name $CONTAINER_NAME $DOCKER_HUB_REPO /bin/bash -c "pytest test.py && flake8"'
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
sh 'docker stop $CONTAINER_NAME || true'
sh 'docker rm $CONTAINER_NAME || true'
sh 'docker run -d -p 5000:5000 --name $CONTAINER_NAME $DOCKER_HUB_REPO'
}
}
}
}
Push the code to github.
git add .
git commit -m "Add Jenkinsfile"
git push origin mainGo to "flask-hello-world" pipeline page and click on "Build Now"
Install Kubernetes
In this example, we will be installing kubernetes on Ubuntu 20.04 Linux machine using minikube. If you are on cloud, you can create a new instance and install kubernetes on that.
# https://minikube.sigs.k8s.io/docs/start/
# Install docker for managing containers
sudo apt-get install docker.io
# Install minikube
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube
# Add the current USER to docker group
sudo usermod -aG docker $USER && newgrp docker
# Start minikube cluster
minikube start
# Add an alias for kubectl command
alias kubectl="minikube kubectl --"Create a new file named "deployment.yaml" in your project and add the below code.
apiVersion: apps/v1
kind: Deployment
metadata:
name: flask-hello-deployment # name of the deployment
spec:
template: # pod defintion
metadata:
name: flask-hello # name of the pod
labels:
app: flask-hello
tier: frontend
spec:
containers:
- name: flask-hello
image: shivammitra/flask-hello-world:latest
replicas: 3
selector: # Mandatory, Select the pods which needs to be in the replicaset
matchLabels:
app: flask-hello
tier: frontend
Test the deployment manually by running the following command:
$ kubectl apply -f deployment.yaml
deployment.apps/flask-hello-deployment created
$ kubectl get deployments flask-hello-deployment
NAME READY UP-TO-DATE AVAILABLE AGE
flask-hello-deployment 3/3 3 3 45sCreate a new file named "service.yaml" and add the following code
apiVersion: v1
kind: Service
metadata:
name: flask-hello-service-nodeport # name of the service
spec:
type: NodePort # Used for accessing a port externally
ports:
- port: 5000 # Service port
targetPort: 5000 # Pod port, default: same as port
nodePort: 30008 # Node port which can be used externally, default: auto-assign any free port
selector: # Which pods to expose externally ?
app: flask-hello
tier: frontend
Test the service manually by running below commands.
$ kubectl apply -f service.yaml
service/flask-hello-service-nodeport created
$ kubectl get service flask-hello-service-nodeport
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
flask-hello-service-nodeport NodePort 10.110.46.59 <none> 5000:30008/TCP 36sRun below command to access the application on the browser.
minikube service flask-hello-service-nodeport
Finally, push the updated code to github.
git add .
git commit -m "Add kubernetes deployment and service yaml"
git push origin mainDeploy using jenkins on kubernetes
First, we will add our docker hub credential in jenkins. This is needed as we have to first push the docker image before deploying on kubernetes.
Open jenkins credentials page.
Click on 'global'.
Add the credentials for docker hub account.
We will now modify our Jenkinsfile in the project to push the image and then deploy the application on kubernetes.
pipeline {
agent any
environment {
DOCKER_HUB_REPO = "shivammitra/flask-hello-world"
CONTAINER_NAME = "flask-hello-world"
DOCKERHUB_CREDENTIALS=credentials('dockerhub-credentials')
}
stages {
/* We do not need a stage for checkout here since it is done by default when using "Pipeline script from SCM" option. */
stage('Build') {
steps {
echo 'Building..'
sh 'docker image build -t $DOCKER_HUB_REPO:latest .'
}
}
stage('Test') {
steps {
echo 'Testing..'
sh 'docker stop $CONTAINER_NAME || true'
sh 'docker rm $CONTAINER_NAME || true'
sh 'docker run --name $CONTAINER_NAME $DOCKER_HUB_REPO /bin/bash -c "pytest test.py && flake8"'
}
}
stage('Push') {
steps {
echo 'Pushing image..'
sh 'echo $DOCKERHUB_CREDENTIALS_PSW | docker login -u $DOCKERHUB_CREDENTIALS_USR --password-stdin'
sh 'docker push $DOCKER_HUB_REPO:latest'
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
sh 'minikube kubectl -- apply -f deployment.yaml'
sh 'minikube kubectl -- apply -f service.yaml'
}
}
}
}
Commit the changes to github.
git add .
git commit -m "Modify Jenkinsfile to deploy on kubernetes"
git push origin mainGo to "flask-hello-world" pipeline page and click on "Build Now"
Is the Kubernetes host different from the jenkins host ?
In case you have set up kubernetes on a different virtual machine, we will need to ssh to this machine from jenkins machine, copy the deployment and service files and then run kubernetes commands.
Create a ssh key pair on jenkins server.
$ cd ~/.ssh # We are on jenkins server
$ ssh-keygen -t rsa # select the default options
$ cat id_rsa.pub # Copy the public keyAdd the public key we created to authorized_keys on kubernetes server.
$ cd ~/.ssh # We are on kubernetes server
$ echo "<public key>" >> authorized_keysModify the 'Deploy' section of Jenkinsfile. Replace and with the username and ip address of kubernetes host respectively.
stage('Deploy') {
steps {
echo 'Deploying....'
sh 'scp -r -o StrictHostKeyChecking=no deployment.yaml service.yaml < username>@<ip address>:~/'
sh 'ssh <username><ip address> kubectl apply -f ~/deployment.yaml'
sh 'ssh <username><ip address> kubectl apply -f ~/service.yaml'
}
}
Commit the code. Build the pipeline again on Jenkins server.
Conclusion
In this tutorial, we have tried to build a very simple CI/CD pipeline using jenkins and kubernetes. Do note that this is only for learning purposes. If you are creating a CI/CD pipeline for production use, follow the official docs of jenkins and kubernetes for the best practices.
