Developer’s Documentation¶
Contents
Setting up Automated Regression Testing for Octo¶
Note
Automated regression tests are run on every Octo source code update and install. Run BATS only if you are an advanced user who wants to contribute to Octo or run on a Linux distribution on which YottaDB is Supportable but not Supported.
Prerequisites for Automated Regression Testing¶
Octo uses BATS for automated integration and regression testing. To use BATS to run tests on Octo, BATS version 1.1+ must be installed:
git clone https://github.com/bats-core/bats-core.git cd bats-core sudo ./install.sh /usr
This will install BATS to /usr/bin. Note that installing to /usr may require root access or use of
sudo
. To specify an alternative path change the argument to your preferred location, e.g. “/usr/local” to install to /usr/local/bin.Details available in the BATS source repo.
Some bats tests also require go, java and expect. The appropriate libraries must installed:
# Ubuntu Linux/Debian Linux sudo apt-get install --no-install-recommends default-jdk expect golang-go # RHEL 8/Rocky Linux sudo yum install java-latest-openjdk expect golangAdditionally, some tests require a JDBC driver. The JDBC driver must be downloaded to the build directory and JDBC_VERSION must be set in the environment. Versions starting with 42.2.6 are tested, but earlier versions may work. For example, using 42.2.12 version:
export JDBC_VERSION=42.2.12 wget https://jdbc.postgresql.org/download/postgresql-$JDBC_VERSION.jar
(Debian/Ubuntu only) Install the en_US.utf8 locale
Octo tests should be run with the en_US.utf8 locale due to collation order differences in other locales that cause some test outputs to not match reference outputs.
# Debian locale -a # if "en_US.utf8" does not appear among the available locales listed by the above command, proceed to the steps below: sudo vi /etc/locale.gen # or use your preferred text editor # Uncomment the line in /etc/locale.gen that reads "en_US.UTF-8 UTF-8", then save and exit sudo locale-gen
Install cmocka unit testing framework
Octo uses cmocka for automated unit testing. To build and run Octo’s unit tests, cmocka must be installed:
# Ubuntu Linux/Debian Linux sudo apt-get install --no-install-recommends libcmocka-dev # RHEL 8/Rocky Linux sudo yum install libcmocka-devel
Install PostgreSQL client (psql)
Octo uses the psql PostgreSQL for some integration/regression tests. To build and run these tests, psql must be installed:
# Ubuntu Linux/Debian Linux sudo apt-get install --no-install-recommends postgresql-client # RHEL 8/Rocky Linux sudo yum install postgresql
Install PostgreSQL server
Octo uses the PostgreSQL server for some integration/regression tests. To build and run these tests, PostgreSQL must be installed:
# Ubuntu Linux/Debian Linux sudo apt-get install --no-install-recommends postgresql # RHEL 8/Rocky Linux sudo yum install postgresqlAdditionally, PostgreSQL must be set up for the user who will be running the tests:
sudo -u postgres createuser [username] sudo -u postgres psql <<PSQL alter user [username] createdb; create database [username] LC_COLLATE='C' template=template0; PSQL
Install MySQL server and client (mysql)
Octo uses the MySQL server for some integration/regression tests. To build and run these tests, MySQL must be installed:
# Ubuntu Linux/Debian Linux sudo apt-get install mysql-server mysql-client # RHEL 8/Rocky Linux yum install -y mysql-serverAdditionally, MySQL must be set up for the user who will be running the tests, using a password of ‘ydbrocks’. Assuming a
bash
shell, the following can be run as a single command to do the necessary setup:sudo mysql <<MYSQL CREATE USER '$USER'@'localhost' IDENTIFIED BY 'ydbrocks'; GRANT CREATE, ALTER, DROP, INSERT, UPDATE, DELETE, SELECT, REFERENCES, RELOAD on *.* TO '$USER'@'localhost' WITH GRANT OPTION; FLUSH PRIVILEGES; MYSQL
Install UnixODBC and the Postgres ODBC Shared Library
Octo runs ODBC driver tests if the UnixODBC package is installed. To build and run these tests, you need to do the following:
# Ubuntu Linux/Debian Linux sudo apt-get install unixodbc odbc-postgresql # RHEL 8/Rocky Linux sudo yum install unixODBC postgresql-odbc
Running the tests¶
Note
Make sure that YDB and YDBAIM are up to date and installed.
To generate a Debug build instead of a Release build (the default), add -DCMAKE_BUILD_TYPE=Debug
to the CMake line.
To additionally disable the generation of installation rules for the make install
, add -DDISABLE_INSTALL=ON
. This can be useful when doing testing in a temporary build directory only.
To build the full test suite rather than a subset of it, the FULL_TEST_SUITE
option needs to be set to ON
, e.g. cmake -D FULL_TEST_SUITE=ON ..
. In addition, there is a speed test that can be enabled by cmake -D TEST_SPEED=ON
. The speed test is intended for use in benchmarking and needs to be run separately from the full test suite. Run it with bats -T bats_tests/test_speed.bats
or with the equivalent command ctest -V -R test_speed
.
To show the output of failed tests, export the environment variable CTEST_OUTPUT_ON_FAILURE=TRUE
. Alternatively, you can show output for only a single run by passing the argument to make: make CTEST_OUTPUT_ON_FAILURE=TRUE test
.
For example, run the following commands to build and test the full test suite:
ydbuser@ydbdev:~/YDBOcto/build$ cmake -D FULL_TEST_SUITE=ON ..
...
ydbuser@ydbdev:~/YDBOcto/build$ make -j `getconf _NPROCESSORS_ONLN`
...
ydbuser@ydbdev:~/YDBOcto/build$ sudo make install
...
ydbuser@ydbdev:~/YDBOcto/build$ ctest -j `getconf _NPROCESSORS_ONLN`
...
100% tests passed, 0 tests failed out of 137
Total Test time (real) = 1111.17 sec
Contributing¶
To contribute or help with further development, fork the YDBOcto repository, clone your fork to a local copy and begin contributing!
Please also set up the pre-commit script to automatically enforce some coding conventions. Creating a symbolic link to YDBOcto/pre-commit will be enough for the setup. Assuming you are in the top-level directory of your local copy, the following will work:
ln -s ../../pre-commit .git/hooks
Note that this script will require tcsh
and clang-format-11
or a later release.
# Ubuntu 20.04
sudo apt install --no-install-recommends clang-format-11
# Any Debian-like distro; see also https://apt.llvm.org/
bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"
# RHEL 8/Rocky Linux
sudo yum install clang-tools-extra
clang-tidy¶
The CI pipeline will run the clang-tidy tool to catch common errors. You can replicate its behavior locally as follows:
# Ubuntu 20.04
sudo apt install --no-install-recommends clang-tidy
# Any Debian-like distro
bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"
# RHEL 8/Rocky Linux
sudo yum install clang-tools-extra
mkdir build
cd build
cmake -D CMAKE_EXPORT_COMPILE_COMMANDS=ON ..
clang-tidy ../src/octo_init.c # replace octo_init.c with the file you want to check
clang-tidy-8
and later are supported.
Dockerfiles¶
There are 4 Dockerfiles at the top of the source tree:
Dockerfile
Dockerfile-Tests.rocky
Dockerfile-Tests.ubuntu
Dockerfile-Tests.vista
Dockerfile
builds a docker container suitable for use for using Octo in
a testing capacity. The other files are all testing related, and are used to
replicate the Gitlab pipelines. There are instructions at the top of each file
for usage as well as current limitations.