This document describes the workflow to building and testing NCP.
Help is welcome and needed! The more testers there are the less each single one of them has to work. Please, write us in the forums or Telegram if you are interested.**
Procedure
@nachoparker is in charge of batch building all the appliances. The build system automatically uploads the images to downloads/testing.
When the images for a new release are ready, @nachoparker will create a post in the forums, quoting all the volunteer testers in this document. They will receive an email notification.
At least one tester per board will respond in the forum thread with ‘ack’, meaning that they will assign themselves the testing of that board for this pre-release. Then, if everything is ok, they will create another comment saying ‘ok’. After everything is tested, @nachoparker will move the images to downloads, and @nachoparker (or any volunteer) will anonunce the release.
Ideally we would have at least two volunteers per board, so nobody is pressured to do it in a short notice if they are busy with their life. A week would be a good period to wait to all images to be pretested before the official release announcement.
Initially just checking that the board boots, activation and wizard works, and nextcloud logs in should be enough. We will gradually replace this manual method for some automated testing scripts.
- rpi version @nachoparker @kanuahs
- rpi berryboot version @ovpso alias @Oliver_Van
- odroid version - @bit67 @nachoparker @kunibert
- rock64 version @nachoparker
- bananapi version @theCalcaholic
- orangepi version (new)
- docker version armhf @ushills
- docker version x86 @ushills @not_real_name
- curl installer (debian) @ovpso alias @Oliver_Van
Test cases
In the tests folders there are several tests that can be performed to thoroughly verify the build.
- Flash image
- Image boots
- Run automated tests
Automated tests
The tests are located in the tests folder. You can download them directly from github, or clone the repo
git clone git@github.com:nextcloud/nextcloudpi.git
cd nextcloudpi/tests
System tests
There are different tests available. We need python 3 and python selenium installed. The test results will be saved in test_log.txt
./system_tests.py [user@ip]
This test can be run from the board or from our computer. If we don’t specify arguments it will try to guess if the NCP instance is local, is a local docker container, or a remote instance at pi@nextclodpi.local. It will make sure that everything is installed and running.
Activation tests
./activation_tests [ip]
This test will go through the activation process and verify it. It will save the random credentials in test_cfg.txt and subsequent tests will use those credentials if found. If no IP is provided it will try localhost.
Nextcloud tests
./nextcloud_tests [--new] [ip]
This test will make sure that Nextcloud works and is well configured. It will use the random credentials in test_cfg.txt, or ask for credentials if not found. Use --new if you don’t want to use the stored credentials. If no IP is provided it will try localhost.
Writing automated tests
If you feel that you can help with some Python/Bash/Expect automated tests, please contribute to the code in the tests folder of this repo.
The following technologies are suggested, but other suggestions are welcome.
- TAP Test anything protocol. A standard testing protocol with bindings for Jenkins and many programming languages.
- BATS - a Bash testing suite that supports the TAP protocol
- python selenium - tool to test web based interfaces in python.
- expect, pexpect…