Gitlab Continuous Integration & Delivery (Part-4)

In the previous section (Part-3), we saw how to set up CI / CD with Spring as a service method

To remind you, we are in the eight sections, setting up CI / CD with the Docker method.

Setting up CI / CD with Docker method

This section includes the following steps:

1. Setup private Docker Registry
2. Setup Gitlab Container Registry — insecure method
3. Create a Dockerfile for each microservice
4. Create Parent gitlab-ci.yml file
5. Create Child gitlab-ci.yml file

Setup private docker registry

First, we need to know what is the Registry?

In nutshell, Registry is a system for storing docker images. We can save the Docker images in the registry using the docker pull and push commands.

1. Start your registry by this command on the server:

docker run -d -p 5000:5000 --restart=always --name registry registry:2

Note: You should run this command on your server(which place that you want running your applications).

Note: The default port of the registry is 500.

2. Now, open to the docker daemon.json file in the following path:

nano /etc/docker/daemon.json

And then, add the insecure registry URL and put your server IP address for point to the registry:

{
   "insecure-registries" : ["THE_SERVER_IP_ADDRESS":5000"]
}

Save the file and exit.

3. Now, restart the docker:

service docker restart

Setup Gitlab Container Registry — insecure method

Now, go back to the gitlab system, and at this time we need to show the docker images in the Gitlab Container Registry. By default, this feature is disabled on the Gitlab CE and we need some configuration for enabling it.

1. First, we need to connect to the gitlab container with docker exec command:

sudo docker exec -it gitlab /bin/bash

2. Now, open the gitlab.rb file which is for configuring GitLab:

nano /etc/gitlab/gitlab.rb

3. Then change all the following lines as follows:

gitlab_rails['registry_api-url'] = "http://"YOUR_REGISTRY_SERVER_IP":5000"
registry_external_url 'http://"YOUR_REGISTRY_SERVER_IP"'

### Settings used by GitLab application
gitlab_rails['registry_enabled'] = true
gitlab_rails['registry_host'] = "YOUR_REGISTRY_SERVER_IP"
gitlab_rails['registry_port'] = "5000"
gitlab_rails['registry_path'] = "/var/opt/gitlab/gitlab-rails/shared/registry

Note: change the “YOUR_REGISTRY_SERVER_IP” string to your Registry server IP that we already install the docker registry on it.

4. Then, save the changes and exit.

5. Finally, run the following command to reconfigure the Gitlab:

gitlab-ctl reconfigure

6. Go to the Gitlab panel in the left sidebar menu, you can see the Packages & Registries item as shown:

Create a Dockerfile for each microservice

1. In the root path of each microservice, create a file called Dockerfile with the following contents:

FROM openjdk:8-alpine
VOLUME /tmp
ADD /target/”YOUR_MICROSERVICE_NAME”.jar ”YOUR_MICROSERVICE_NAME”.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/”YOUR_MICROSERVICE_NAME”.jar"]

Note: in this example, we using JDK 8, if you use a higher version just change it.

Create Parent gitlab-ci.yml file

I recommend reading this article before going to the gitlab parent-child pipeline step.

1. In the project root path, create a file called gitlab-ci.yml and enter the following contents into the file:

image: maven:latest
stages:
 - build-package-maven
 - trigger-microservices
build-package maven 1/2:
 stage: build-package-maven
 script:
   - mvn compile
 only:
   - qa
build-package maven 2/2:
 stage: build-package-maven
 script:
   - cd common-godar
   - mvn package
 only:
   - dev
trigger microservices 1/3:
 stage: trigger-microservices
 trigger:
   include: microservice-one/gitlab-microservice-one.yml
 only:
   changes:
     - microservice-one/*
trigger microservices 2/3:
 stage: trigger-microservices
 trigger:
   include: microservice-two/gitlab-microservice-two.yml
 only:
   changes:
     - microservice-two/*
trigger microservices 3/:
 stage: trigger-microservices
 trigger:
   include: microservice-three/gitlab-microservice-three.yml
 only:
   changes:
     - microservice-three/*

2. In this example, I define 3 microservice you should replace it with each number of your microservices.

Note: trigger-microservice 1/3, 2/3, 3/3 it means that we have only 3 microservice so we group them with the same stage name.

Create Child gitlab-ci.yml file

1. Create a file in the root path of each microservice such as the following naming pattern (the naming is optional), for example, gitlab-ci-eureka.yml.

stages:
 - package-maven
 - launch-microservice-one
package maven:
 stage: package-maven
 before_script:
   - 'which ssh-agent'
   - eval $(ssh-agent -s)
   - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
   - mkdir -p ~/.ssh
   - chmod 700 ~/.ssh
   - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
 script:
   - cd module-one
   - mvn package
   - cp $CI_PROJECT_DIR/module-one/target/module-one.jar $CI_STABLE_PROJECT_DIR
   - cp $CI_PROJECT_DIR/module-one/Dockerfile $CI_STABLE_PROJECT_DIR
   - scp /home/gitlab-runner/"CI_STABLE_PROJECT_DIR"/module-one.jar "SERVER_USERNAME"@”SERVER_IP_ADDRESS”:/"CI_SERVER_PROJECT_DIR"   - scp "CI_STABLE_PROJECT_DIR"/Dockerfile "SERVER_USERNAME"@”SERVER_IP_ADDRESS”:/CI_SERVER_PROJECT_DIR"
   - scp CI_STABLE_PROJECT_DIR/Dockerfile "USERNAME"@”SERVER_IP_ADDRESS”:/"CI_SERVER_PROJECT_DIR"
 artifacts:
   paths:
     - $CI_PROJECT_DIR/module-one/target/module-one.jar
 only:
   - qa
deploy:
 stage: launch-module-one
 image: docker:latest
 before_script:
   - 'which ssh-agent'
   - eval $(ssh-agent -s)
   - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
   - mkdir -p ~/.ssh
   - chmod 700 ~/.ssh
   - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
 script:
   - ssh -o StrictHostKeyChecking=no -T $SERVER_USERNAME@$SERVER_IP_ADDRESS "cd "CI_SERVER_PROJECT_DIR" && docker build -t $CI_REGISTRY_IMAGE . && echo $CI_BUILD_TOKEN | docker login -u "$CI_REGISTRY_USER" --password-stdin $CI_REGISTRY && docker push $CI_REGISTRY_IMAGE && docker run -d -p 8080:8080 -t $CI_REGISTRY_IMAGE "
 only:
   - qa

Notes:

1. $CI_PROJECT_DIR: this is the default path of the gitlab-runner after running the jobs all the result will save on this path, for example:

/home/gitlab-runner/...

2. $CI_STABLE_PROJECT_DIR: Needs to create a path for each microservice on the gitlab system for transferring the files on to this stable path

3. $CI SERVER PROJECT_DIR: this is a path that we need to define to the server for transferring the required files for docker (for example jar and docker files).

4. We have already created these variables in the Gitlab panel by defining the variables: SERVER_USERNAME, SERVER_TEST, SSH_PRIVATE_KEY.

Please refer to this link as a reminder.

5. $CI_REGISTRY_IMAGE, $CI_BUILD_TOKEN: is a default path of the project and token that already defined in the gitlab we don’t need to change it.

The steps on the launch-microservice-one stage:

1. ssh to the server
2. go to the path of the microservice folder
3. run the docker build
4. login to the registry and push the image on the registry container
5. running the application with docker run command (-d is for running the application in the background)

Finally, after committing the all changes you will be able to see them running the pipeline through the gitlab panel.

Gitlab Panel → Current project → select CI / CD → Pipelines

For example:

6. Wait for all jobs to be completed successfully.

7. Now go to the container registry path of the Gitlab and see your saved docker images.

Finally, these methods probably have their drawbacks, and I have tried to give an overview of how to work with Gitlab CI / CD.

I hope I have helped others along the way. And I will gladly answer any questions related to this topic.