Configuration

Configuration settings can be passed to the program with the following precedence:

  1. Flags passed to the program

  2. Environment settings

  3. octo.conf

  4. ~/octo.conf

  5. $ydb_dist/plugin/octo/octo.conf

Config files

Octo currently looks for a configuration file in the following directories:

  • $ydb_dist/plugin/octo/octo.conf

  • ~/octo.conf

  • ./octo.conf

If the same setting exists in more than one configuration file the setting in the later file (according to the list above) will prevail. An example config file can be found in $ydb_dist/plugin/octo/octo.conf.

Sample config file:

//////////////////////////////////////////////////////////////////
//								//
// Copyright (c) 2019-2020 YottaDB LLC and/or its subsidiaries.	//
// All rights reserved.						//
//								//
//	This source code contains the intellectual property	//
//	of its copyright holder(s), and is made available	//
//	under a license.  If you do not know the terms of	//
//	the license, please stop and do not read further.	//
//								//
//////////////////////////////////////////////////////////////////

/* Specifies the verbosity for logging. The valid options are, in order of decreasing verbosity:
 * TRACE, DEBUG, INFO, and ERROR. The default is ERROR.
 */
verbosity = "ERROR"
/* This is prepended to $zroutines on octo startup
 * Generated M routines (which represent SQL queries) are stored in the first source directory found
 * in octo_zroutines followed by $ydb_routines
 */
// octo_zroutines = "./"

// Settings related to the octod process
rocto = {
  // Address and port to listen on for connections
  address = "127.0.0.1"
  port = 1337
  // Set to use DNS to resolve numeric IP addresses
  use_dns = false
  // Set to true to enable usage of the Nagle algorithm on TCP connections and disable the TCP_NODELAY socket option.
  // Default is false (i.e. no delay) as it speeds up rocto response times
  tcp_delay = false
  // Authentication methods; supported options are "md5"
  authentication_method = "md5"
  // Set to 'true' to enable SSL/TLS session encryption for remote connections when clients requests it,
  // or to 'false' to disable SSL/TLS entirely. Note that even when set to 'true' unencrypted
  // connections are accepted when the client requests them. To force SSL/TLS connections, use ssl_required.
  ssl_on = false
  // Set to 'true' to require SSL/TLS session encryption for all remote connections and
  // refuse any client connection that does not request or support SSL/TLS encryption.
  // If set to 'false', unencrypted connections will be accepted.
  // Note that ssl_on must be set to 'true' when ssl_required is set to 'true'.
  ssl_required = false
}

/* Settings controlling YottaDB; these get set as environment variables during startup
 * Defined environment variables will take precedence
 */
/*yottadb = {
  ydb_gbldir = "yottadb.gld"
  ydb_ci = "calltab.ci"
  ydb_routines = "."
}*/

tls: {
  // Max number of certificate authorities to check when verifying certificate
  verify-depth: 7;
  // Name of certificate authority file
  CAfile: "server.crt";
  // Path to certificate authority file
  CApath: "/home/jon/YDBOcto/build/";
  // Session timeout limit, in seconds
  session-timeout: 600;

  /* List of certificate/key pairs specified by identifiers.
   * Currently only the OCTOSERVER identifier is supported.
   */
  OCTOSERVER: {
    format: "PEM";
    cert: "server.crt";
    key: "server.key";
  };
};

Environment variables

The following environment variables must be set:

  • ydb_dist

  • ydb_gbldir

  • ydb_routines

  • ydb_xc_ydbposix

The environment variables ydb_dist, ydb_gbldir, and ydb_routines can initially be set by sourcing ydb_env_set in your YottaDB installation directory.

Example setting of the environment variables (assuming default paths):

source /usr/local/lib/yottadb/r1.28/ydb_env_set
export ydb_routines=". $ydb_routines"
export ydb_xc_ydbposix=$ydb_dist/plugin/ydbposix.xc

Routines

Octo requires that $ydb_dist/plugin/o/_ydbocto.so and $ydb_dist/plugin/o/_ydbposix.so ($ydb_dist/plugin/o/utf8/_ydbocto.so and $ydb_dist/plugin/o/utf8/_ydbposix.so when using Octo in YottaDB’s UTF-8 mode) be included in $ydb_routines. This is necessary not only for running the octo and rocto executables, but also for correctly updating and maintaining the YottaDB triggers that are used to maintain cross references for Octo. Accordingly these paths should exist in the ydb_routines in your normal environment setup scripts.

Note

The source $(pkg-config --variable=prefix yottadb)/ydb_env_set command sets these up automatically for environments with the default structure under $ydb_dir (defaulting to $HOME/.yottadb). For UTF-8, set the environment variable ydb_chset to UTF-8, e.g., export ydb_chset=UTF-8 before sourcing ydb_env_set.

Logging

A config file can include instructions specifying verbosity for logging:

  • TRACE: A TRACE message is useful to help identify issues in the code

  • INFO: An INFO message is used to relay information to the user

  • DEBUG: A DEBUG message helps users debug configuration issues

  • WARNING: A WARNING message warns the user about potential problems in the code

  • ERROR : An ERROR message informs the user that an error has occurred

  • FATAL: A FATAL message terminates the program

Global Variables

All Octo related globals are prefixed with ^%ydbocto. Using normal global mapping procedures for an existing application global directory (where you want to run Octo), map the global variable namespace ^%ydbocto* to a separate region (and its associated database file) that meets the below requirements (the below example commands assume the separate region is named OCTO).

  • NULL_SUBSCRIPTS must be set to ALWAYS.

    Example: $ydb_dist/mupip set -null_subscripts=true -region 'OCTO'

  • KEY_SIZE must be tuned to your data - this can be set to the maximum allowed by YottaDB - “1019”

    Example: $ydb_dist/mupip set -key_size=1019 -region 'OCTO'

  • RECORD_SIZE must be tuned to your data/queries - a reasonable starting value is “300000”

    Example: $ydb_dist/mupip set -record_size=300000 -region 'OCTO'

Some of the globals used in Octo are:

  • ^%ydboctoocto: This global can refer to various functions, variables, octo “read only” table values (postgres mappings, oneRowTable, etc.), and some counts. It needs to persist between sessions.

  • ^%ydboctoxref: This global contains cross-references, row counts and other statistical information. It needs to persist between sessions.

  • ^%ydboctoschema: This global contains information about the tables loaded into the database. It needs to persist between sessions.

Since most of the Octo variables need to persist between sessions, it is necessary that the region(s) mapping the ^%ydbocto* namespace have replication turned on in a replicated environment (or journaling turned on in a non-replicated environment).

TLS/SSL Configuration

Note

The instructions provided in this section will help in setting up a self-signed certificate for TLS, but not for setting up TLS in production. Also, a full TLS setup will require certificates signed by a known and trusted certificate authority.

Enabling TLS/SSL requires several additional steps beyond installing the YottaDB encryption plugin - it requires creating a Certificate Authority (CA), generating a TLS/SSL certificate, and making additional changes to octo.conf.

Generate CA key and certificate

# In a directory in which you want to store all the certificates for Octo
# Be sure to create a strong passphrase for the CA
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out CA.key
# This creates a CA valid for 1-year and interactively prompts for additional information
openssl req -new -nodes -key CA.key -days 365 -x509 -out CA.crt

Create server key and certificate request

# This creates a 2048 bit private key
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out server.key
# This creates the certificate signing request
openssl req -new -key server.key -out server.csr

Sign certificate based on request and local CA

# Asks the CA to sign the certificate with a 1-Year validity time
openssl x509 -req -in server.csr -CA CA.crt -CAkey CA.key -CAcreateserial -out server.crt -days 365
# Mask the password for the certificate in a way YottaDB understands
$ydb_dist/plugin/gtmcrypt/maskpass
# This will need to be added to any startup scripts for octo/rocto
export ydb_tls_passwd_OCTOSERVER=[Masked Password from maskpass]
export ydb_crypt_config=/path/to/octo.conf

Update Octo configuration file

$ydb_dist/plugin/octo/octo.conf contains an outline of the minimum configuration options.

For TLS/SSL a configuration file is required, based on the changes below.

  1. In the “rocto” section, “ssl_on” must be set to “true” (no quotes needed in the configuration file).

  2. A “tls” section must be present and generally conform to the requirements specified for the TLS plugin itself. Other notes:

    • Octo doesn’t use any of the “dh*” settings, so those can be omitted.

    • The “format” specifier can also be omitted, as long as the certs are in PEM format.

    • The “CAfile” and “CApath” fields are mandatory and must point to valid files/locations with a full path.

    • A subsection named “OCTOSERVER” with “key”, and “cert” settings specifying the names of the private key and cert files.

  3. The ydb_tls_passwd_OCTOSERVER and ydb_crypt_config environment variables must be set correctly.

If you source $(pkg-config --variable=prefix yottadb)/ydb_env_set it provides reasonable default values of environment variables. Review the $ydb_dist/plugin/octo/octo.conf file to configure your own environment.

Usage

Before running Octo/Rocto make sure that the required YottaDB variables are set either by creating your own script or run source $(pkg-config --variable=prefix yottadb)/ydb_env_set.

To use the command-line SQL interpreter run: $ydb_dist/plugin/bin/octo.

To use rocto, the PostgreSQL protocol compatible server, run: $ydb_dist/plugin/bin/rocto -p XXXX where -p XXXX optionally specifies a TCP port at which rocto is to listen for connections. The default port number is 1337.