Gitlab 서버이전 + Gitlab-runner 활용한 CI/CD

Gitlab 서버이전과 Gitlab-runner 이용한 지속정인 통합과 배포 활용(CI/CD)

 

새로운 서버로 데이터 이전을 해야 해서
구서버에 구동중인 gitlab 을 데이터와 모두 신서버로 이전하는 작업을 해보았습니다.
신서버는 최신버전의 gitlab 을 설치하였습니다. (설치문서 참고)

구서버 환경
Centos 6
Gitlab 12.1.6

신서버 환경
Centos 8
Gitlab 13.1.4

 

: 구서버 :

구서버의 gitlab 을 신서버로 데이터와 함께 이전하려면 구서버와 신서버의 버전이 같아야 합니다.
현재 구서버 버전은 12.1.6 이기 때문에 아래와 같이 작업합니다.
12.1.6 
    -> 12.10.10 
    -> 13.0.0
    -> 13.1.4

버전 확인은 아래 참고

docs.gitlab.com/ee/policy/maintenance.html

GitLab release and maintenance policy | GitLab

 

 

업그레이드는 yum 을 이용하여 합니다.
# sudo yum install gitlab-ce-12.10.10-ce.0.el6
# sudo yum install gitlab-ce-13.0.0-ce.0.el6
# sudo yum install gitlab-ce-13.1.4-ce.0.el6

구서버의 버전이 신서버와 버전이 동일하게 되었으면 구서버의 데이터를 백업합니다.

# sudo gitlab-rake gitlab:backup:create

위 명령어를 실행하면 아래 경로에 백업 파일 생성됩니다.
/var/opt/gitlab/backups/

 

: 신서버 복원 :

 

서비스 중지
# sudo gitlab-ctl stop unicorn
# sudo gitlab-ctl stop sidekiq

Verify
# sudo gitlab-ctl sta

. 신서버에서 구서버 백업파일 받기
# wget http://xxx.com/1595377720_2020_07_22_13.1.4_gitlab_backup.tar 

. 백업파일 이동
# cp 1595377720_2020_07_22_13.1.4_gitlab_backup.tar /var/opt/gitlab/backups/

. 복구 명령
# sudo gitlab-rake gitlab:backup:restore BACKUP=1595377720_2020_07_22_13.1.4

기존정보 다 지울거냐 물어봄
yes


여기서 이슈가 여러개 발생하였는데 아래 이슈별로 정리했습니다.
gitlab 이전시에 많은 참고 되셨으면 합니다.

 

 

 

이슈

복원시 아래와 같은 오류 발생

[root@servera131 src]# gitlab-rake gitlab:backup:restore BACKUP=1595377720_2020_07_22_13.1.4
Unpacking backup ... done
Before restoring the database, we will remove all existing
tables to avoid future upgrade problems. Be aware that if you have
custom tables in the GitLab database these tables and all data will be
removed.

Do you want to continue (yes/no)? yes
Removing all tables. Press `Ctrl-C` within 5 seconds to abort
2020-07-22 13:40:17 +0900 -- Cleaning the database ... 
rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:27:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:64:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:db:drop_tables
(See full trace by running task with --trace)



해결
postgresql 프로세스에서 구동하고 있어서 그렇다.
# ps aux | grep postgresql
# kill psid





이슈 

rake aborted!
Redis::CannotConnectError: Error connecting to Redis on /var/opt/gitlab/redis/redis.socket (Errno::ENOENT)
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/cache.rake:15:in `block (6 levels) in <top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/cache.rake:14:in `loop'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/cache.rake:14:in `block (5 levels) in <top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/cache.rake:12:in `each'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/cache.rake:12:in `block (4 levels) in <top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/redis/wrapper.rb:18:in `block in with'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/redis/wrapper.rb:18:in `with'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/cache.rake:8:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:81:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'

Caused by:
Errno::ENOENT: No such file or directory - connect(2) for /var/opt/gitlab/redis/redis.socket
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/cache.rake:15:in `block (6 levels) in <top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/cache.rake:14:in `loop'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/cache.rake:14:in `block (5 levels) in <top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/cache.rake:12:in `each'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/cache.rake:12:in `block (4 levels) in <top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/redis/wrapper.rb:18:in `block in with'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/redis/wrapper.rb:18:in `with'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/cache.rake:8:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:81:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => cache:clear => cache:clear:redis
(See full trace by running task with --trace)


해결
# gitlab-ctl start redis
timeout: down: redis: 1s, normally up, want up



이슈
관리자모드에서 CI/CD 500 error 발생

해결
구서버의 gitlab-secrets.json /etc/gitlab/  복사

# gitlab-ctl reconfigure
# gitlab-ctl start


이슈
2020-07-22 14:15:53 +0900 -- done
2020-07-22 14:15:53 +0900 -- Restoring repositories ...
 * na93008/ci-test ... Error: 14:failed to connect to all addresses
[Failed] restoring na93008/ci-test repository
 * redmine/test ... Error: 14:failed to connect to all addresses
[Failed] restoring redmine/test repository
 * na93008/sitemaker ... Error: 14:failed to connect to all addresses
[Failed] restoring na93008/sitemaker repository
 * na93008/cardeditor ... Error: 14:failed to connect to all addresses

해결
구서버 gitlab 저장디렉토리 (/var/opt/gitlab/git-data/repositories ) 압축해서
신서버 /var/opt/gitlab/git-data/repositories 압축푼후 gitlab 재시작




이슈
서버이전후 gitlab-runner 실행이안됨 계속 보류만 나옴
CI/CD 설정에서 ip 가 구서버 가르키고 있음

해결
설정 > CI/CD Runners 에서 등록되 있는 Runners activated for this project
삭제후 새로 생성 
(생성하는법은 gitlab-runner 프로젝트 등록 참고)