GitLab

How to plot cumulative Gitlab group members using matplotlib

This is based on my previous post to find the group ID by group name.

import matplotlib.pyplot as plt
import matplotlib.dates as mdates
from datetime import datetime
import gitlab

def plot_cumulative_members(members):
    # Convert date strings to datetime objects and sort
    dates = sorted([datetime.strptime(member["access_granted_date"], '%Y-%m-%dT%H:%M:%S.%fZ') for member in members])

    # Calculate cumulative count
    cumulative_count = range(1, len(dates) + 1)

    # Plotting
    plt.figure(figsize=(10, 6))
    plt.plot(dates, cumulative_count, marker='o')
    plt.title('Cumulative Number of Users in GitLab Group Over Time')
    plt.xlabel('Date')
    plt.ylabel('Cumulative Number of Users')
    plt.grid(True)
    plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%Y-%m-%d'))
    plt.gca().xaxis.set_major_locator(mdates.YearLocator())
    plt.gcf().autofmt_xdate()  # Rotation
    plt.show()

group_id = get_gitlab_group_id("Protectors of the Footprint Realm", 'glpat-...')
members = get_group_members(group_id, 'glpat-...')
with plt.xkcd():
    plot_cumulative_members(members)

Posted by Uli Köhler in GitLab, Python

How to find Gitlab group ID using Gitlab API via python-gitlab

pip install python-gitlab
import gitlab

def get_gitlab_group_id(group_name, access_token):
    # Initialize a GitLab instance with your private token
    gl = gitlab.Gitlab('https://gitlab.com', private_token=access_token)

    # Search for groups by name
    groups = gl.groups.list(search=group_name)
    for group in groups:
        if group.name == group_name or group.path == group_name:
            return group.id

    raise ValueError("Group not found")

# Usage example
group_id = get_gitlab_group_id("Protectors of the Footprint Realm", 'glpat-yykIsrTg6RyKcFAvd2os')

group_id will be an integer, for example 6604163

Posted by Uli Köhler in GitLab, Python

How to generate random strings that look like Gitlab access tokens

import secrets
import string

def generate_gitlab_access_token(length=20):
    characters = string.ascii_letters + string.digits + '_'
    token = ''.join(secrets.choice(characters) for _ in range(length))
    return f'glpat-{token}'

# Example usage:
access_token = generate_gitlab_access_token()
print(access_token)

Example output:

glpat-yykIsrTg6RyKcFAvd2os

 

Posted by Uli Köhler in GitLab, Python

How to delete logs from Gitlab Docker instance (Omnibus)

You can often save multiple gigabytes of space by deleting old logs from Gitlab instances. It should be clear that the logs will be lost forever once deleting them, so ensure that you don’t really care about your content before deleting this.

First, enter the logs directory, that is the directory mapped to /var/log/gitlab. This should be mapped out from your container to a local directory or volume. In our Gitlab reference config for docker-compose, we have mapped it to the logs directory.

In that directory, run the following commands:

find . \( -name "*.gz" -o -name "*.log* -o -name "*.s" -o -name "*.u" \) -exec rm -v {} \;

This will delete all files with the given extensions.

Posted by Uli Köhler in GitLab

How to specify which docker image to use in .gitlab-ci.yml

The following .gitlab-ci.yml will build a native executable project using cmake with a custom docker image:

stages:
  - build

buildmyexe:
  stage: build
  image: 'ulikoehler/ubuntu-gcc-cmake:latest'
  script:
    - cmake .
    - make -j4

In this example, we have only one stage – if you have multiple stages, you can specify different images for each of them.

Posted by Uli Köhler in Docker, git, GitLab

How to register gitlab runner for multiple GitLab instances

One GitLab runner can support attaching to multiple GitLab instances. You can add additional instances by running gitlab-runner register.

gitlab-runner register

again, or running

docker-compose exec -T gitlab-runner gitlab-runner register

if you’re using our docker-compose-based GitLab runner setup.

This will append a new section to the config file.

Posted by Uli Köhler in GitLab

How to install gitlab-runner using docker-compose

First, choose a directory where the service will reside in. I recommend /opt/gitlab-runner.  Then create docker-compose.yml in said directory with this content:

version: '3'
services:
  gitlab-runner:
    image: 'gitlab/gitlab-runner:latest'
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./config:/etc/gitlab-runner
    restart: unless-stopped

then run this command to configure the runner:

docker-compose up -d
docker-compose exec gitlab-runner gitlab-runner register

It will ask you for details about the GitLab instance you want to attach to. You will find this information at https://<your-gitlab-domain>/admin/runners. This example is for my GitLab instance:

Runtime platform                                    arch=amd64 os=linux pid=38 revision=943fc252 version=13.7.0
Running in system-mode.

Enter the GitLab instance URL (for example, https://gitlab.com/):
https://gitlab.techoverflow.net/
Enter the registration token:
Loo2lahf9Shoogheiyae
Enter a description for the runner:
[148a53203df8]: My-Runner
Enter tags for the runner (comma-separated):

Registering runner... succeeded                     runner=oc-oKWMH
Enter an executor: custom, docker-ssh, shell, virtualbox, docker-ssh+machine, docker, parallels, ssh, docker+machine, kubernetes:
shell
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

Now, restart the runner that is running with the old config (i.e. with no gitlab instance being attached):

docker-compose down

After that’s finished, you can run the script from our previous post Create a systemd service for your docker-compose project in 10 seconds in the directory where docker-compose.yml is located.

curl -fsSL https://techoverflow.net/scripts/create-docker-compose-service.sh | sudo bash /dev/stdin

This will automatically generate a systemd service and start the runner (also on boot). For more details, see the corresponding blogpost. If your directory is named gitlab-runner, the service file will be stored in /etc/systemd/systemd/gitlab-runner.service, hence these are commands you can use to control the service:

Note that the script that creates the systemd service will automatically start the runner, so you don’t need to start it manually. !

Start by

sudo systemctl start gitlab-runner

Restart by

sudo systemctl restart gitlab-runner

Stop by

sudo systemctl stop gitlab-runner

View status:

sudo systemctl status gitlab-runner

View & follow logs:

sudo journalctl -xfu gitlab-runner

View logs in less:

sudo journalctl -xu gitlab-runner

Also see Mini systemd cheat-sheet

Also see How to register gitlab runner for multiple GitLab instances.

Note that you can also use

docker-compose logs -f

to view the logs (run this from the directory where docker-compose.yml) is located.

In case you see an error message like

error during connect: Post http://docker:2375/v1.40/auth: dial tcp: lookup docker on 192.168.178.1:53: no such host

in your jobs, see How to fix Gitlab CI error during connect: Post http://docker:2375/v1.40/auth: dial tcp: lookup docker on &#8230; no such host

Posted by Uli Köhler in GitLab