Continuous Integration and Continuous Deployment (CI/CD) are fundamental to modern DevOps practices. Jenkins is one of the most popular open-source tools to implement CI/CD pipelines. In this guide, you’ll learn how to set up a Jenkins pipeline from scratch and automate the process of building, testing, and deploying your code.
Table of Contents
What Is Jenkins?
Jenkins is a free and open-source tool that helps automate parts of the software development process, like building, testing, and deploying code. It’s widely used to support continuous integration and continuous delivery (CI/CD), making it easier for teams to deliver updates faster and more reliably.
Prerequisites
Before you begin, make sure you have:
- Jenkins installed (locally or on a server)
- Java is installed on the server
- Git installed
- Access to a Git repository (e.g., GitHub)
- A basic understanding of CI/CD processes
Step 1: How to Install Jenkins in Ubuntu/Debian
Install Jenkins with the following steps
// update your Ubuntu system
sudo apt update
// install openjdk
sudo apt install openjdk-11-jdk -y
//Downloads the Jenkins GPG signing key, and adds the key to your system’s trusted software list
wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
// Adds Jenkins Debian package repository to your system’s list.
sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
//Update your Ubuntu system
sudo apt update
//Install jenkins
sudo apt install jenkins -yHow to install Jenkins on Amazon Linux 2
Belos steps are meant for Amazon Linux 2. If you’re using Amazon Linux 2023, it’s better to use “dnf” instead of “yum”. Although you can still run the “yum command” for compatibility, it’s just a shortcut to “dnf” and might not support all of its features. For more information, check the official AWS documentation.
//Update your Linux machine
[ec2-user ~]$ sudo yum update –y
//Add the Jenkins repo
[ec2-user ~]$ sudo wget -O /etc/yum.repos.d/jenkins.repo \
https://pkg.jenkins.io/redhat-stable/jenkins.repo
// Import a key file from Jenkins-CI to enable installation from the package
[ec2-user ~]$ sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key
// update your system
[ec2-user ~]$ sudo yum update
// install java
[ec2-user ~]$ sudo yum install java-17-amazon-corretto -y
//install Jenkins
[ec2-user ~]$ sudo yum install jenkins -y
Start and enable Jenkins: (These command are used on both Ubuntu and Amazon Linux)
sudo systemctl start jenkins //Start Jenkins as a service
sudo systemctl enable jenkins //Enable the Jenkins service to start at boot
sudo systemctl status jenkins //Check the status of the Jenkins serviceAccess Jenkins via: http://localhost:8080
Step 2: Unlock Jenkins and Install Jenkins Plugins
- When Jenkins starts for the first time, it will ask for a password.
- Get the password by running:
sudo cat /var/lib/jenkins/secrets/initialAdminPassword- Paste it into the browser and proceed.
- Choose “Install suggested plugins”.
Step 3: Create Admin User
After you install Jenkins plugins, it will prompt you to create an admin user. Fill in the required details and complete the setup.
Step 4: Create a New Pipeline Job
- From the Jenkins dashboard, click on “New Item”.
- Enter a name for the job (e.g., My-application-Pipeline).
- Choose “Pipeline” as the job type and click OK.
Step 5: Configure Your Pipeline
In the Pipeline Configuration, do the following:
- Scroll down to the Pipeline section.
- Choose the Pipeline script.
- Add your pipeline script (Jenkinsfile) directly here or use a Git repository.
pipeline {
agent any
stages {
stage('Clone') {
steps {
git 'https://github.com/your-repo-url.git'
}
}
stage('Build') {
steps {
sh './build.sh'
}
}
stage('Test') {
steps {
sh './run-tests.sh'
}
}
stage('Deploy') {
steps {
sh './deploy.sh'
}
}
}
}Step 6: Create a Jenkinsfile in Your Git Repo
Instead of writing the pipeline in Jenkins UI, you can place a Jenkinsfile in your repository root:
pipeline {
agent any
tools {
maven 'Maven 3.6.3'
}
environment {
DEPLOY_ENV = 'staging'
}
stages {
stage('Checkout') {
steps {
git 'https://github.com/your-repo.git'
}
}
stage('Build') {
steps {
sh 'mvn clean install'
}
}
stage('Test') {
steps {
sh 'mvn test'
}
}
stage('Deploy') {
steps {
echo "Deploying to $DEPLOY_ENV environment"
sh './deploy.sh'
}
}
}
}Please read one of our most valueable blog about the Jenkins integration with the Sonarqube for the code analysis
Step 7: Trigger Pipeline Automatically (Optional)
You can set up triggers so Jenkins runs the pipeline automatically on code changes.
GitHub Webhook:
- Go to GitHub repository → Settings → Webhooks
- Add webhook: http://localhost:8080/github-webhook/
- In Jenkins job config → enable “GitHub hook trigger for GITScm polling”
Step 8: View and Analyze Build Results
After running the pipeline:
- You can view logs for each stage.
- If the build fails, Jenkins will show the error messages.
- You can configure email/Slack notifications for build results.
Step 9: Add Test Reports or Artifacts
You can archive test reports or artifacts like so:
post {
always {
junit 'target/surefire-reports/*.xml'
archiveArtifacts artifacts: 'target/*.jar', fingerprint: true
}
}
Jenkins best practices
Use Pipeline as Code (Jenkinsfile)
Instead of setting up everything manually through the Jenkins UI, it’s best to use a Jenkinsfile stored in your source code repository. This approach keeps your entire CI/CD process under version control, promotes better collaboration and code review among team members, and ensures consistency across different environments. You can also start with a declarative pipeline because it’s more readable and beginner-friendly as compared to the scripted one.
Break Down Your Pipeline into Stages
It’s a smart idea to break your Jenkins pipeline into clear, logical stages like Build, Test, Quality Check, and Deploy. Doing this not only helps you quickly spot where something went wrong if there’s an issue. It also makes your pipeline easier to understand and manage. It will runs multiple tasks at the same time, like parallelizing long tests or deployments, which can save a lot of time.
Keep Jenkins Pipelines Fast and Efficient
To keep your Jenkins pipelines fast and efficient, it’s important to avoid long-running jobs because they can delay your entire development workflow. You can speed things up by caching dependencies like Maven or NPM so they don’t have to be downloaded every time. Running tests in parallel can save a lot of time, and using lightweight containers or build agents helps things run more smoothly and quickly.
Integrate Code Quality and Security Tools
To keep your code clean and secure, it’s a great idea to integrate tools like SonarQube, Checkmarx, or OWASP Dependency-Check into your CI/CD pipeline. These tools automatically scan your code for bugs and security issues before it ever reaches production. This not only helps you catch problems early but also makes your code easier to maintain and keeps your application safe. You can add a SonarQube scan as a stage in your pipeline and set it to fail the build if it doesn’t meet your quality standards.
Use Credentials Securely
When working with Jenkins, it’s important to keep your secrets, like API keys and passwords, safe. Instead of writing them directly in your Jenkinsfile, which can lead to accidental exposure, use the Jenkins Credentials Plugin. It securely stores your sensitive information and lets you access it safely within your pipeline using the “withCredentials” block. This not only protects your data but also keeps your infrastructure secure from potential leaks.
Use Shared Libraries for Common Logic
If you’re working on several projects that use the same pipeline steps, like deployment or sending notifications, it’s a smart idea to move that repeated logic into a shared library. This helps you avoid writing the same code over and over, makes future updates easier (since you only need to change it in one place), and ensures everything stays consistent across your projects. It’s like creating a reusable toolbox that all your pipelines can rely on.
Enable Notifications and Logging
Keep your team in the loop by automatically sending build updates through tools like Slack, email, or Microsoft Teams. This helps everyone stay informed, get faster feedback, work better together, and quickly jump in to fix any issues when a build fails.
Run Jenkins on Stable Infrastructure
If you’re managing your production Jenkins setup, it’s important to make sure both the main server (master) and its agents run smoothly. To keep everything running reliably, always back up your data, regularly update Jenkins and its plugins, and keep an eye on system performance and disk space. For even better performance and flexibility, you might want to run Jenkins on Kubernetes or use agents that can automatically scale based on your workload.
Tag Your Builds
Always tag your source code with the version of the build that gets deployed. This simple step is important because it makes it much easier to roll back to a previous version if something goes wrong, and it also helps you keep track of which version was released when.
Keep Your Pipeline Clean
Avoid cramming too much logic or complex business rules directly into your Jenkinsfile. Instead, keep the Jenkins pipeline focused on orchestrating tasks, and offload the heavy lifting, like detailed configurations or deployment steps, to external shell scripts or tools like Ansible and Terraform. This makes your pipeline cleaner, easier to manage, and more reusable.
Conclusion
Install Jenkins and creating a Jenkins pipeline helps you automate your entire CI/CD process, making it easier to deliver software quickly and consistently. With a Jenkinsfile, you can keep your pipeline code right next to your application code, which makes everything more organized and easier to manage.
Jenkins is very flexible, you can add plugins, stages, and other tools to fit your specific needs, whether you’re deploying a small website or managing a complex system of microservices.