Gitlab Continuous Integration & Delivery (Part-3)

image credit: gitlab.com

In the previous section (Part-2), we saw how to install and register Gitlab-runner.

To remind you, we are in the sisx section (Add Git remote URL to project) and seventh section (Setting up CI / CD with Spring as a service method)

Before we move on, it is good to get acquainted with a few concepts:

1. Pipelines are the top-level component of continuous integration, delivery, and deployment.

2. Jobs, which define what to do. For example, jobs that compile or test code.

3. Stages, which define when to run the jobs. For example, stages that run tests after stages that compile the code.

Setting up CI / CD with Spring as a service method

This section includes the following steps:

1. Add Git Remote URL to the project
2. SSH settings to enter the Host server without entering a password
3. Add SSH information to Variables in Gitlab
4. Make microservices executable in the pom file
5. Create Parent gitlab-ci.yml file
6. Transfer the Jar file created on the Client-server to the Host server
7. Create Child gitlab-ci.yml file
8. View the implementation of Pipelines in Gitlab

Add Git Remote URL to the project

1. In the GitLab panel, go to the main page and then click on the name of the project you have already created. Click on the clone button and copy the link from the Clone with HTTP section.

2. In the root path of the desired project in the IDE, open the terminal window and paste the link of the desired project as in the following command:

git remote add gitlab "YOUR_GIT_REPO_URL"

3. Now to push the desired project as a mirror with its complete history such as branch, commit, etc., run the following command:

git push gitlab --mirror

Then wait for this step to be done.

4. Now the desired project has been successfully added to the desired repository in Gitlab.

SSH settings to enter the Host server without entering a password

Now, to enter the server and transfer the file via SSH, we must make settings beforehand.

1. First, we generate SSH authentication keys on the server with the ssh-keygen command.
2. Then run the ssh-copy-id command on the client-server:

Note: ssh-copy-id Installs an SSH key as an authorized key on the server. Its purpose is to provide password-free access to any login. This facilitates automatic and password-free logging in with the SSH protocol.

Note: To be sure, log in to the host server with the ssh command, which must be done successfully without entering a password.

Add SSH information to Variables in Gitlab

1. In the gitlab panel, go to the project path on the left sidebar menu:

Settings →  CI / CD → Variables → (click on) Expand → (click on) Add variable

2. Then, in the key field, enter your desired name for the username, and in the Value field, enter the username of the host server, and in the Flags field, disable the Protect variable option, as shown below:

Then, Click on the Add variable option.

3. Click on the Add variable option again and enter the desired name for the server address and enter the IP address of the server address in the value field and disable the Protect variable option, as shown in the following image:

4. In the host server terminal, run the following command to copy the ssh private key and then copy the key:

cat/root/.ssh/id_rsa

5. Go to Variables in gitlab again and enter the desired name in the Key field and enter the private key that you copied in the previous step in the value field, as shown below. Finally, click on the Add variable button:

Make microservices executable in the pom file

Note: These steps must be repeated for each microservice.

1. Edit the pom.xml microservice file as follows:

<packaging>jar</packaging>
 
<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>1.4.0.RELEASE</version>
</parent>
 
<dependencies>
    ....
</dependencies>
 
<build>
<finalName> your microservice name </finalName>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
            <configuration>
                <executable>true</executable>
            </configuration>
        </plugin>
    </plugins>
</build>

I recommend reading this article before going to the gitlab parent-child pipeline step.

Create Parent gitlab-ci.yml file

1. In the gitlab panel, enter the desired project and make sure that you are in the root path of the project, then click on the + option as shown below and select the New File button.

2. As shown below, enter a name called .gitlab-ci.yml in the file name field as shown below:

Note: When we push in this parent gitlab-ci.yml file, all CI and CD commands will be executed through this file. In fact, this is the starting point of this file that we created in the root path of the project.

3. Now, copy the following commands and paste in the contents of the file above:

image: maven:latest

stages:
 - build-package-maven
 - trigger-microservices
build-package maven 1/2:
 stage: build-package-maven
 script:
   - mvn compile
 only:
   - qa
build-package maven 2/2:
 stage: build-package-maven
 script:
   - mvn clean package
   - cd /YOUR_PATH_FOR_MOVE_THE_JAR_FILES
   - cp /”YOUR_PATH_OF_JAR_FILES_ON_GITLAB_RUNNER_PATH/JAR_FILE.jar /YOUR_PATH_FOR_MOVE_THE_JAR_FILES
trigger microservices 1/2:
 stage: trigger-microservices
 trigger:
   include: "THE_NAME_OF_MICROSERVICE"/"THE_NAME_OF_CHILD_GILLAB-CI_YAML_FILE".yml
 only:
   changes:
     - "THE_NAME_OF_MICROSERVICE"/*
trigger microservices 2/2:
 stage: trigger-microservices
 trigger:
   include: "THE_NAME_OF_MICROSERVICE"/"THE_NAME_OF_CHILD_GILLAB-CI_YAML_FILE".yml
 only:
   changes:
     - "THE_NAME_OF_MICROSERVICE"/*

Note: You can validate the gitlab-ci.yml file before committing so that you do not encounter any errors while executing.

Enter the gitlab panel and in the current project, click on the CI / CD → Pipelines option through the left sidebar menu.

At the top right of the page, click on the CI Lint option, and in the new page that opens, enter the contents of your gitlab-ci.yml file, and click on the validate option, if there is an error in the structure of the yml file.

You will show otherwise you can commit the yml file with ease 🙅.

status: syntax ic correct

4. Finally, click on the Commit changes button.

Note: This implementation will not be successful because we have to complete this process.

Additional information about the commands defined in the parent gitlab-ci.yml file:

In the parent gitlab-ci.yml file, there are three stages that are defined as groups and their order is the same as the execution order.

“build-package-maven”:

1.The script associated with the mvn compile command is defined.
2. The script associated with the mvn package command is defined.
3. Then, copy the jar of the created files of all the microservices in the path we already created in the Linux operating system.
4. In the only: -qa section, we specify that these commands should only be executed in the branch dev, which means that when the developers request a merge request, the person acting as the maintainer will execute these commands on the test server after merging the code in the branch qa.

“trigger-microservices”:

Note: For all microservices that have the child gitlab-ci.yml file, we must repeat this step.

1. In this step, by defining the keyword trigger: include, we can run child gitlab-ci.yml for each microservice in the CI / CD execution step.

(In the next steps, how to make child gitlab-ci.yml will be explained)

2 . By definition: keyword only: changes means that if a change is made to the root path of the defined microservice, run it in the CI / CD process. In this project, we need to trigger all microservices in their root path.

Note: You can only include a specific path or file in the only: changes section. For example, we can say that if there is a change in the pom of the microservice file, only that file will be CI / CD processed.

Trigger: 
  Only:
    microservicename/pom.xml

Transfer the Jar file created on the Client-server to the Host server

Note: Before going to the child gitlab-ci creation step, you only need to transfer the jar files created in the previous step, which were created by executing the CD step on the client, to the server, to define them as a Linux service.

1. Repeat the following command on the client for all microservices:

scp /THE_PATH_OF_JAR_FILE/JAR_FILE.jar HOST_USERNAME@HOST_IP_ADDRESS:/THE_PATH_FOR_TRANSFER_THE_FILE

2. After transferring all the jar files in the Linux terminal, repeat the following command for each microservice:

sudo ln -s /path/to/JAR_FILE.jar /etc/init.d/NAME_OF_JAR_FILE

3. To make an executable file, enter the following command:

sudo chmod +x /etc/init.d/NAME_OF_JAR_FILE

4. (Optional) You can enter the following commands to check (start, stop, status) of the desired application, which is known as the later Linux service:

sudo service your-app start
sudo service your-app status
sudo service your-app stop

Create Child gitlab-ci.yml file

Note: Repeat this step for each microservice.

1. In the gitlab panel, click on the name of a microservice and then, as before, create a new file called gitlab-ci.yml and copy and paste the following commands into it:

stages:
 - deploy-microservice-one
deploy:
 stage: deploy-microservice-one
 before_script:
- eval $(ssh-agent -s) 
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config- scp /”THE_PATH_JAR_FILE”/”FILE_NAME.jar” $TEST_USER_SERVER@$TEST_SERVER:/”THE_PATH_ON_SERVER
script:
- ssh -o StrictHostKeyChecking=no -T $TEST_SERVER_USER@$SERVER_TEST "sudo service "NAME_OF_YOUR_MICROSERVICE" stop && sudo service "NAME_OF_YOUR_MICROSERVICE" start && sudo service "NAME_OF_YOUR_MICROSERVICE" status"
only:
   - qa

2. Additional information about the commands defined in child gitlab-ci.yml:

Note: Here we define only one stage called deploy aggregator, which is related to the Continuous deployment stage.

Description before_script:

1. First line: run ssh-agent (inside the built environment).
2. Line 2: Add the SSH key stored in the variable SSH_PRIVATE_KEY to the agent store.
3. Third and fourth lines: Create an SSH directory and give it access.
4. Sixth line: ‌ Transfer the file with the SCP command from the client to the server (Note: Replace the desired path where you created the jar of files in the client and in the second part create a path in the host server and replace the address with the example Upgrade like /opt/builds).

Script description:

This part is related to the ssh command to the desired server.

Note: The desired names for the username and server address that you have previously entered in Variables the desired project in gitlab.

In the continuation of this line, enter the names of the microservices that you introduced as a service in Linux.

3. Finally, click on Commit changes.

View the implementation of Pipelines in Gitlab

1. Go to the gitlab panel and select your project and then for example select the QA branch, after pushing changes like the image below you will see the progress of building the YAML file jobs:

2. Click on the Blue icon to enter the pipelines page, as shown below:

3. After complete execution, you will see all the pipelines of the green icon and you will encounter a red icon if there is an error, then by clicking on each step, you can see the result of the jobs.

4. Then, after running the microservices, you can view them in your browser.

Note: Add the ports of each microservice to the server firewall as before.

This part is over and we go to the next part.