Self-built Gitlab (mailbox configuration, Split PostgreSQL, Redis) + Caprice

Source: Internet
Author: User
Tags postgresql prepare docker run docker swarm
Foreword
Recently, I have tossed some self-built gitlab, make a record here, for your reference. The entire build process is based on Docker Swarm (recently there are plans to port microservices to Kubernetes, but it has not been frustrated, and the old scheme will be used for the time being). The theme map has nothing to do with the theme, please ignore ...
1. Quick start
The general principle of configuration is to be available and then optimized. Simply starting gitlab requires only one command:

sudo docker run --detach --hostname gitlab.yuclk.com --publish 443: 443 --publish 80:80 --publish 22:22 --name gitlab --restart always --volume / mnt / nas / gitlab / config: / etc / gitlab --volume / mnt / nas / gitlab / logs: / var / log / gitlab --volume / mnt / nas / gitlab / data: / var / opt / gitlab gitlab / gitlab-ce: latest
Then in terms of functionality, it is only a step away from configuring the mailbox. Enter the container through docker exec -it, modify /etc/gitlab/gitlab.rb, and add the following configuration:

# SMTP
gitlab_rails ['smtp_enable'] = true
gitlab_rails ['smtp_address'] = 'smtp.exmail.qq.com'
gitlab_rails ['smtp_port'] = 465
gitlab_rails ['smtp_user_name'] = '[email protected]'
gitlab_rails ['smtp_password'] = '******'
gitlab_rails ['smtp_domain'] = 'smtp.qq.com'
gitlab_rails ['smtp_authentication'] = 'login'
gitlab_rails ['smtp_enable_starttls_auto'] = true
gitlab_rails ['smtp_tls'] = true
gitlab_rails ['smtp_openssl_verify_mode'] = 'peer'

# If your SMTP server does not like the default 'From: [email protected]' you
# can change the 'From' with this setting.
gitlab_rails ['gitlab_email_from'] = '[email protected]'
# gitlab_rails ['gitlab_email_reply_to'] = '[email protected]'
Then use the following command to make the configuration take effect:

gitlab-ctl reconfigure
gitlab-ctl restart
Finally, you can enter the gitlab console to test sending mail:

gitlab-rails console
Notify.test_email ('[email protected]', 'Message Subject', 'Message Body'). Deliver_now
In addition, there are two configuration methods, for example:

sudo docker run --env GITLAB_OMNIBUS_CONFIG = "external_url 'http://my.domain.com/'; gitlab_rails ['lfs_enabled'] = true;"
sudo docker run -e GITLAB_CDN_HOST = gitlab.youclk.com
The above is just a shorthand example. I personally do not recommend the latter two. Although setting the environment variable has a higher degree of freedom, but there are too many configurations, I still prefer to replace or mount the configuration file.

After the above configuration, the basic functions of the entire gitlab are available (CI / CD will be available in the future and then toss).

2. Integration into swarm
This step only needs to prepare two orchestration files, proxy:

version: '3.5'
services:
  proxy:
    image: vfarcic / docker-flow-proxy: 18.04.06-12
    ports:
      -80:80
    networks:
      -proxy
    environment:
      -LISTENER_ADDRESS = swarm-listener: 18.04.06-12
      -MODE = swarm
    secrets:
      -dfp_users_monitoring
    deploy:
      labels:
        -com.df.notify = true
        -com.df.port = 8080
        -com.df.serviceDomain = localhost
        -com.df.reqPathSearchReplace = / alive, / v1 / docker-flow-proxy / ping
      restart_policy:
        condition: any
        max_attempts: 3
      update_config:
        delay: 5s
        order: start-first

  swarm-listener:
    image: vfarcic / docker-flow-swarm-listener: 18.04.12-7
    networks:
      -proxy
    volumes:
      -/var/run/docker.sock:/var/run/docker.sock
    environment:
      -DF_NOTIFY_CREATE_SERVICE_URL = http: // proxy: 8080 / v1 / docker-flow-proxy / reconfigure
      -DF_NOTIFY_REMOVE_SERVICE_URL = http: // proxy: 8080 / v1 / docker-flow-proxy / remove
    deploy:
      placement:
        constraints: [node.role == manager]
      restart_policy:
        condition: any
        max_attempts: 3
      update_config:
        delay: 5s
        order: start-first

networks:
  proxy:
    external: true
secrets:
  dfp_users_monitoring:
    external: true
gitlab:

version: '3.5'
services:
  gitlab:
    image: gitlab / gitlab-ce
    hostname: gitlab.youclk.com
    networks:
      -proxy
      -youclk
    ports:
      -2289: 22
    volumes:
      -/ mnt / nas / gitlab / config: / etc / gitlab
      -/ mnt / nas / gitlab / logs: / var / log / gitlab
      -/ mnt / nas / gitlab / data: / var / opt / gitlab
    deploy:
      mode: replicated
      labels:
        -com.df.notify = true
        -com.df.port = 80
        -com.df.serviceDomain = gitlab.youclk.com
      restart_policy:
        condition: any
        max_attempts: 3
      update_config:
        delay: 5s
        order: start-first

networks:
  proxy:
    external: true
  youclk:
    external: true
Then it can be enabled in turn, for example:

# create infrastructure
echo "youclk: ****" | docker secret create dfp_users_monitoring-
docker network create --driver overlay proxy
docker network create --driver overlay youclk

# startup
docker stack deploy -c src / docker-compose-proxy.yml proxy
docker stack deploy -c src / docker-compose-gitlab.yml gitlab
The first step is optimized. If you do n’t want to be too frustrated, it can be over here. The service availability will not have much impact.

3. Separate cache and database
I do n’t know why gitlab does not provide a pure version of the image further, and the official recommendation is to install omnibus. Anyway, I do n’t like to integrate the database and cache into one image at all. It is expected to build a sub-image and remove nginx, postgreSQL, redis. After some painstaking testing, I still have to say that it is a pity that a lot of time is wasted but the goal is not successfully achieved. Finally, I can only disable postgreSQL and redis in the configuration file according to the official recommendation.

First prepare a db orchestration file:

version: '3.5'
services:
  redis:
    image: redis
    networks:
      -proxy
      -youclk
    ports:
      -6379: 6379
    deploy:
      restart_policy:
        condition: any
        max_attempts: 3
      update_config:
        delay: 5s
        order: start-first

  postgresql:
    image: postgres
    networks:
      -proxy
      -youclk
    ports:
      -5432: 5432
    volumes:
      -/ mnt / nas / db / postgresql: / var / lib / postgresql
      -$ PWD / src / postgresql.conf: /etc/postgresql/postgresql.conf
    deploy:
      labels:
        -com.df.notify = true
        -com.df.port = 5432
      restart_policy:
        condition: any
        max_attempts: 3
      update_config:
        delay: 5s
        o
rder: start-first
networks:
  proxy:
    external: true
  youclk:
    external: true
Note that postgreSQL disables remote connections by default. You need to modify /etc/postgresql/postgresql.conf. Anyway, for intranet use, allow all ips. Listen_addresses = ‘*’, the way to get the configuration file:

docker run -i --rm postgres cat /usr/share/postgresql/postgresql.conf.sample> my-postgres.conf
Then just like the example in my layout file, just mount it.

The default user name, password, and initial database of postgreSQL are all postgres, and the default configuration can be changed by setting environment variables:

environment:
  -POSTGRES_PASSWORD = mysecretpassword
  -POSTGRES_USER = myuser
  -POSTGRES_DB = mydb
The last step is to modify the gitlab configuration file:

# redis
redis ['enable'] = false

# Redis via TCP
gitlab_rails ['redis_host'] = 'redis'
gitlab_rails ['redis_port'] = 6379

# Disable the built-in Postgres
postgresql ['enable'] = false

# Fill in the connection details for database.yml
gitlab_rails ['db_adapter'] = 'postgresql'
gitlab_rails ['db_encoding'] = 'utf8'
gitlab_rails ['db_host'] = 'postgresql'
gitlab_rails ['db_port'] = 5432
gitlab_rails ['db_username'] = 'postgres'
gitlab_rails ['db_password'] = 'postgres'
gitlab_rails ['db_database'] = 'postgres'
This is the end of the second step of optimization. Start the command:

cp gitlab.rb /mnt/nas/gitlab/config/gitlab.rb
docker stack deploy -c src / docker-compose-gitlab.yml gitlab
4. Enable SSL
If your situation fits the scenario recommended by the official documentation, it is very simple:

external_url "https://gitlab.youclk.com"
nginx ['redirect_http_to_https'] = true

mkdir -p / etc / gitlab / ssl
chmod 700 / etc / gitlab / ssl
cp gitlab.youclk.com.key gitlab.youclk.com.crt / etc / gitlab / ssl /
But in general, certificates, load balancing, gateways, etc. in microservices belong to the peripheral infrastructure and will not be linked to applications. This case is simpler because no configuration is required at all.

5. Salvation of OCD
The previous section mentioned that enabling SSL in the current microservices environment does not require any configuration, so why should I write this section? Are you mentally disabled? Okay, it's a brain disability, and it's a brain disability with OCD. Let's put a picture first:
Can you understand what is obsessive-compulsive disorder? Maybe you will not use HTTP to clone the code in this life, but your brother ca n’t stand the prompt, and when he thinks about it, his scalp becomes numb, just like being crushed back and forth by ten thousand grass and mud horses.

This small problem makes me uncomfortable, can't fall asleep, and I have gained several pounds. I have to solve him. Then, I do n’t know if the state was already on the edge of Abi Hell. The first thing I thought of was to find the logic of this prompt from the source code and force it to change to HTTPS (failure); inject a section of js to modify the prompt ( There is a modified delay, which is still unbearable); modify the nginx configuration file (success).

The final solution is to first set external_url to https (this cannot be bypassed), and then because the load balance is only http, so first proxy to https, and then return to http (you can also not return), in order to offset the configuration of external_url, Finally, modify the request header:

proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Ssl on;
Isn't it more comfortable now? The above tests took almost a day, just for such a gadget, but no matter what, in the end, I still feel a little relieved.

But, do you think that is the end? No, when I reviewed the reference document again, I found such a hint:

# Other bundled components (Registry, Pages, etc) use a similar strategy for proxied SSL. Set the particular component's * _external_url with https: // and prefix the nginx [...] configuration with the component name. For example, for Registry use the following configuration:

registry_external_url 'https://registry.example.com'

registry_nginx ['listen_port'] = 80
registry_nginx ['listen_https'] = false
So, I tried to modify it like this:

external_url 'https://gitlab.youclk.com'
nginx ['listen_port'] = 80
nginx ['listen_https'] = false
The effect is exactly the same. At this time, it was like being crushed back and forth by tens of thousands of grass and mud horses. Why didn't I pull the reference document to the end? I had a big joke and bloody lesson all day long!

Conclusion
I was very sad after such a lot of tossing on Saturday, and I encountered the mac speaker broken, and I was in the end. Then I thought about how to adjust it, well, this thought, a bunch of negative energy came out.

I do n’t know how many people are often constrained by “self” like me. At the inner level, I ’m not a very homely person. I do n’t like to stay at home. I always talk about poetry and distance with many people, but only limited Yu Tan. On the one hand, the waste of a lot of time often brings me a sense of guilt, so I rarely have fun; on the other hand, for the asset situation where the monthly balance is negative, I occasionally have to be cheeky and ask the family to ask me I do n’t know what else I have to pursue?

I thought about it one night. I got up at five o'clock on Sunday morning and bought a moving ticket to go to Shanghai to relax. Okay, I forgot that the subway did not open so early, and I was reluctant to hit it (I feel miserable ~), I can only wait until Ordered in line to change the sign (I also had a quarrel with the cut in line, I just want to insert in front of me, and there is a silly X to help maintain, I also see what the hell ~) ... Although the process is tortuous, the result is not too Oops, after being shocked by the pile of buildings in Lujiazui, it is no longer so negative. Finally, I went to the top floor of the Shanghai Center Building, ordered a cup of coffee there, typed the day code, and returned at seven o'clock in the evening ~

My public account "Yu Ke", we grow together!
Self-built Gitlab (mailbox configuration, split PostgreSQL, Redis) + free thinking
Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.