Steps to install openQA (a local copy of openqa.debian.net)

The goal is to install openQA in a local virtual machine with the capacity to run 2 workers.

Assumptions

In this document, the following assumptions are made:

Prepare the openQA server

  1. Host System Preparation (user=root on the host)

  2. Download the latest netinst image (user=you on the host)

  3. Start virtual machine manager (user=you on the host)

  4. Prepare virtual machine (user=you on the host)

  5. If you are using VirtualBox, add the Guest Additions for better performance

  6. Use dep822 for apt (user=root)

    apt --yes modernize-sources
rm /etc/apt/sources.list~
rm /etc/apt/sources.list.bak

  7. Prepare to use fasttrack (user=root)

    apt-get install fasttrack-archive-keyring

  8. Activate fasttrack (user=root)

    cat << EOF > /etc/apt/sources.list.d/fasttrack.sources
Types: deb
URIs: https://fasttrack.debian.net/debian-fasttrack/
Suites: trixie-fasttrack
Components: main
Signed-By: /usr/share/keyrings/fasttrack-archive-keyring.gpg
EOF

  9. Install required and preferred packages (user=root)

    rm /etc/apt/apt.conf # This contains the proxy that was used during the installer
apt-get update # Update the package list
apt-get install auto-apt-proxy
apt-get install aptitude # For looking at package dependencies etc.
apt-get install --yes ansible ssh-askpass rsync # For the configuration of the openqa server

  10. Install the Debian package openqa and friends (user=root)

    apt-get install --yes -t trixie-fasttrack openqa openqa-worker

  11. Configure openqa (for non-ssl) (user=root)

    /usr/share/openqa/script/configure-web-proxy
a2dissite 000-default
a2ensite openqa
systemctl reload apache2

  12. Configure the login procedure (user=root)

    cat << EOF > /etc/openqa/openqa.ini.d/50-download_domains.ini
[global]
# Allowed domains for ISO_URL
download_domains = reproducible-builds.org debian.org
EOF
cat << EOF > /etc/openqa/openqa.ini.d/50-auth.ini
[auth]
# Simple authentication
method = Fake
EOF

  13. Download and prepare the tests (user=root)

    su geekotest -s /bin/bash
cd /var/lib/openqa/share/tests
git clone https://salsa.debian.org/qa/openqa/openqa-tests-debian.git debian
ln -s debian debian-live
ln -s debian debian-edu
ln -s debian kali
ln -s debian windows
exit # To become root again

  14. Generate the documentation (user=root)

    markdown /var/lib/openqa/share/tests/debian/documentation/install.md > /usr/share/openqa/public/install.html
markdown /var/lib/openqa/share/tests/debian/documentation/work_on_tests.md > /usr/share/openqa/public/work_on_tests.html
markdown /var/lib/openqa/share/tests/debian/documentation/evolution_of_test_scenario.md > /usr/share/openqa/public/evolution_of_test_scenario.html

  15. Apply the Debian branding (user=root)

    mkdir -p /usr/share/openqa/templates/webapi/branding/openqa.debian.net
cat << EOF > /usr/share/openqa/templates/webapi/branding/openqa.debian.net/sponsorbox.html.ep
<div class="text-end" id="sponsorbox">
  <a href="https://www.debian.org" title="Debian"><img alt="Debian" src="https://www.debian.org/logos/openlogo.svg"></a>
</div>
EOF
cat << EOF > /usr/share/openqa/templates/webapi/branding/openqa.debian.net/docbox.html.ep
<h1>Welcome to the openQA instance of Debian</h1>
<p>Life is too short for manual testing!</p>
<p>
  <a class="btn btn-primary btn-lg" href="http://open.qa" role="button">Learn more &#xbb;</a>
  <a class="btn btn-primary btn-lg" href="/https/openqa.debian.net/install.html" role="button">Install your own copy of openQA &#xbb;</a>
  <a class="btn btn-primary btn-lg" href="/https/openqa.debian.net/work_on_tests.html" role="button">Work together on tests &#xbb;</a>
  <a class="btn btn-primary btn-lg" href="/https/openqa.debian.net/evolution_of_test_scenario.html" role="button">Evolution of a test scenario &#xbb;</a>
</p>
EOF
cat << EOF > /etc/openqa/openqa.ini.d/50-branding.ini
[global]
branding=openqa.debian.net
EOF

  16. Restart the openQA webui (user=root)

    systemctl reload openqa-webui

  17. (Re-)generate the default API key on the openQA server, valid for 24 hours (user=user)

    openqa-cli api --apibase http://localhost -X GET login | grep "Logged in as"

  18. Configure the API key, to avoid providing the values on each openqa-cli command (user=user)

    mkdir ~/.config/openqa
cat << EOF >> ~/.config/openqa/client.conf
[localhost]
key = 1234567890ABCDEF
secret = 1234567890ABCDEF
EOF

  19. Create a new SSH key for Ansible (user=user)

    ssh-keygen -t ed25519 -C "VM Debian-openQA"
cat ~/.ssh/id_ed25519.pub >> ~/.ssh/authorized_keys
# Make localhost known
ssh-keyscan localhost >> ~/.ssh/known_hosts

  20. Create the job groups (user=user)

    # Ugly, needs improvement: create the groups manually and remove the deleted ids
# At the time of writing this, the highest id was 24
for i in $(seq 1 24); do openqa-cli api -X POST job_groups name=dummy_group$i; done
for i in $(seq 1 9) 15; do openqa-cli api -X DELETE job_groups/$i; done

  21. Configure the openQA server using Ansible (user=user)

    cd /var/lib/openqa/share/tests/debian/configuration
ansible-playbook configure_openqa.yaml -i server_localhost.yaml

  22. Mount the pre-existing ISOs that have been made available (user=root)

Note: this mount point must be unmounted before updating openqa with apt-get

Prepare the openqa-worker

  1. Disable the (currently) not needed slirp VDE service (user=root)

    systemctl disable openqa-slirpvde

  2. Use the proxy when running installers under openQA (user=root)

    cat << EOF > /etc/openqa/workers.ini.d/50-http_proxy.ini
[global]
# Use the proxy (apt-cacher-ng) from the host
HTTP_PROXY = http://192.168.122.1:3142/
# The value http://localhost:9/ could be used for a 'network-access-denied' setting
EOF

  3. Configure a second worker with more memory (user=root)

    cat << EOF > /etc/openqa/workers.ini.d/55-bigmem-worker.ini
# x86_64 worker for jobs that require more memory
[2]
WORKER_CLASS=qemu_x86_64,qemu_x86_64_bigmem
EOF

  4. Configure the secrets (user=root)

    cat << EOF > /etc/openqa/client.conf
[localhost]
key = 1234567890ABCDEF
secret = 1234567890ABCDEF
EOF
chown _openqa-worker /etc/openqa/client.conf
chmod 0400 /etc/openqa/client.conf

  5. Start the cache service (user=root)

    systemctl restart openqa-worker-cacheservice
systemctl restart openqa-worker-cacheservice-minion

  6. Enable the workers (user=root)

    systemctl enable openqa-worker@1
systemctl enable openqa-worker@2

  7. Start the workers (user=root)

    systemctl start openqa-worker@1
systemctl start openqa-worker@2

Optional: add an arm64 worker

It is possible to test arm64 images using emulation. Be warned that this is slow.

  1. Install the suitable qemu package (user=root)

    apt-get install qemu-system-arm

  2. Configure a worker for arm64 (user=root)

    cat << EOF > /etc/openqa/workers.ini.d/60-arm-worker.ini
# arm64 worker
[10]
WORKER_CLASS=qemu_aarch64
QEMU_NO_KVM=1
QEMUCPU=max
QEMUMACHINE=virt,usb=off
EOF

  3. Activate the worker (user=root)

    systemctl enable openqa-worker@10
systemctl start openqa-worker@10

Run the first tests

Before running the tests, ensure that the API key is still valid and the workers are running

Basic test (amd64)

  1. Download the netinst image (on the host)

    cd /media/fast
wget --timestamping https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-13.1.0-amd64-netinst.iso

  2. Run the netinst tests (user=user)

    openqa-cli api -X POST isos ISO=debian-13.1.0-amd64-netinst.iso DISTRI=debian VERSION=stable FLAVOR=netinst-iso ARCH=x86_64 BUILD=1310

    Note: the tests with the prefix rescue will initially fail by design: they attempt to work on the previous installed image, which will not be present in the initial run. After xfce_HD and xfce_HD@64bit have finished, you can restart the rescue jobs and they will pass.

Cross-test (arm64)

The arm64 worker needs to be configured.

  1. Download the netinst image (on the host)

    cd /media/fast
wget --timestamping https://cdimage.debian.org/debian-cd/current/arm64/iso-cd/debian-13.1.0-arm64-netinst.iso

  2. Run the netinst tests (user=user)

    openqa-cli api -X POST isos ISO=debian-13.1.0-arm64-netinst.iso DISTRI=debian VERSION=stable FLAVOR=netinst-iso ARCH=aarch64 BUILD=1310

A live image (amd64)

  1. Download the live image (on the host)

    cd /media/fast
wget --timestamping https://get.debian.org/images/release/current-live/amd64/iso-hybrid/debian-live-13.1.0-amd64-gnome.iso

  2. Run the tests (user=user)

    openqa-cli api -X POST isos ISO=debian-live-13.1.0-amd64-gnome.iso DISTRI=debian-live VERSION=trixie FLAVOR=gnome ARCH=x86_64 BUILD=1310 CHECKSUM=1886e3fcfdbe60d01ed5e878e143e56ca0443afa8a655404cf3bc057f39a622c

    Note: the tests with the prefix dualboot will initially fail by design: they attempt to work on a Windows image, which will not be present in the initial run, they can be created later

Look at the running builds in openQA

  1. Log in as administrator on the openQA web interface (user=user)

  2. Explore the web interface of openQA

Special tests

Windows images (for the dualboot tests)

If you want to run dual boot tests, you'll need at least a Windows 11 installed image, and optionally a Windows 10 image. Note that these installed images require a lot of disk space.

  1. Install the TPM emulation (user=root)

    apt-get install swtpm

  2. Download the official Windows ISO files from Microsoft for Windows 10 (about 5GB) and Windows 11 (about 8GB) to /media/fast (user=you on the host)

    Note: on the production server the files are stored in the subdirectory fixed to ensure that the ISO file will never be deleted

  3. Verify that the download was successful by checking the checksums (the values are given on the download page) (user=you on the host)

  4. Prepare the Windows harddisk images in openQA (user=user)

    # For the Windows 10 image
openqa-cli api -X POST isos DISTRI=windows FLAVOR=windows10 VERSION=10 BUILD=22H2 ARCH=x86_64 ISO=Win10_22H2_English_x64v1.iso
# For the Windows 11 image
openqa-cli api -X POST isos DISTRI=windows FLAVOR=windows11 VERSION=11 BUILD=25H2 ARCH=x86_64 ISO=Win11_25H2_English_x64.iso

  5. Restart the dualboot tests -> they should pass now

Reset the openQA database (to be able to redo some steps)

If you need to reset the database and start from scratch (but you can keep the installed packages and configuration) follow the next steps.

Warning: this will remove all content from the database

  1. Stop all services (user=root)

    systemctl stop openqa-webui
systemctl stop openqa-gru
systemctl stop openqa-websockets
systemctl stop openqa-scheduler

  2. Drop the database (user=geekotest)

    dropdb openqa

    There should not be a line like DETAIL: There are N other sessions using the database., otherwise a few more openqa-related services need to be stopped

  3. Regenerate the empty database (user=root)

    systemctl start openqa-webui

  4. Recreate the job groups and redo the steps with Ansible

Missing sections

What's next?

Create and modify tests