5. Database Management Tool (MUPIP)
Introduction to MUPIP
This chapter describes common database management operations such as creating database files, modifying database characteristics, database backup and restore, routine integrity checks, extracting or loading data, and optimizing performance.
YottaDB uses M Peripheral Interchange Program (MUPIP) for database management, database journaling, and logical multisite replication (LMS). This chapter summarizes the MUPIP commands pertaining to YottaDB database management and serves as a foundation for more advanced YottaDB functionality described for Journaling and LMS.
For MUPIP commands pertaining to database journaling, refer to Chapter 6: “YottaDB Journaling”.
For MUPIP commands pertaining to multisite database replication, refer to Chapter 7: “Database Replication”.
Note
Two MUPIP operations - INTRPT and STOP - perform process management functions. All other MUPIP operations relate to the operation of the database.
The YottaDB installation procedure places the MUPIP utility program in a directory specified by $ydb_dist.
Invoke MUPIP by executing the mupip program at the shell prompt. If this does not work, consult your system manager (MUPIP requires that the $ydb_dist point to the directory containing the MUPIP executable image).
$ydb_dist/mupip
MUPIP>
MUPIP asks for commands, with the MUPIP> prompt. Enter the EXIT command at the MUPIP> prompt to stop the utility. MUPIP performs one operation at a time, and automatically terminates after most operations.
When additional information appears on the command line after the mupip program name, MUPIP processes the additional information as its command, for example:
$ydb_dist/mupip stop 1158
This starts MUPIP and stops the process with Process ID (PID) 1158.
Some MUPIP commands require information contained in the global directory. Therefore, a process must have access to a valid global directory before using any MUPIP commands other than EXIT, INTRPT, JOURNAL, RESTORE, STOP and the -file option for any command that has that option.
The environment variable ydb_gbldir specifies the active global directory.
To access database files, MUPIP requires a global directory file, pointed to by the environment variable ydb_gblir. For more information on the global directory, refer to “Global Directory Editor (GDE)”.
Note
YottaDB recommends against running YottaDB components as root. When run as root, YottaDB components use the owner and group of the database file as the owner and group of newly created journal files, backup files, snapshot files, shared memory, and semaphores. In addition, they set the permissions on the resulting files, shared memory, and semaphores, as if running as the owner of the database file and as a member of the database file group.
Note
You can perform read operations on a YottaDB database residing on a read-only mounted filesystem. However, the filesystem must remain read-only for the duration of any process that opens a database file resident on it. If a read-only file system is switched to read-write while YottaDB processes have database files open on it, and other processes update those databases, the read-only processes are likely to read incorrect or corrupt data. When the filesystem is read-only, the shared memory resources - which are typically shared among multiple processes - become private to each process instead, so memory resource use increases with each additional concurrent process. M locks mapped to regions that map to database files on read-only filesystems are visible only to the process that owns the locks, and are invisible to other processes.
Operations - Standalone and Concurrent Access
While most MUPIP operations can be performed when YottaDB processes are actively accessing database files, some operations require stand-alone access. When using standalone access, no other process can access the database file(s). When using concurrent access, other processes can read or update the database file(s) while MUPIP accesses them. A few operations permit concurrent access to read database files, but not to update them. All MUPIP operations can be performed with stand-alone access - there is never a requirement for another process to be accessing database files when MUPIP operates on them.
Most MUPIP operations require write access to the database files with which they interact. The exceptions are INTRPT and STOP, which do not require database access, but may require other privileges; EXTRACT, which requires read access; and INTEG, which may require write access, depending on the circumstances it encounters and the qualifiers with which it is invoked. The following table displays some of the MUPIP operations and their database access requirements.
Operations |
MUPIP Command |
Database Access Requirements |
|---|---|---|
Backup database files |
MUPIP BACKUP |
Backup never requires standalone access and concurrent write access is controlled by [NO]ONLINE. |
Create and initialize database files |
MUPIP CREATE |
Standalone Access |
Convert a database file from one endian format to the other (BIG to LITTLE or LITTLE to BIG) |
MUPIP ENDIANCVT |
Standalone Access |
Recover database files (for example, after a system crash) and extract journal records |
MUPIP JOURNAL |
Standalone Access |
Restore databases from bytestream backup files |
MUPIP RESTORE |
Standalone access |
Properly close database files when processes terminate abnormally. |
MUPIP RUNDOWN |
Standalone access |
Modify database and/or journal file characteristics |
MUPIP SET |
Standalone access is required if the MUPIP SET command specifies ACCESS_METHOD, GLOBAL_BUFFERS, MUTEX_SLOTS, LOCK_SPACE or NOJOURNAL, or if any of the JOURNAL options ENABLE, DISABLE, or BUFFER_SIZE are specified. |
Grow the size of BG database files |
MUPIP EXTEND |
Concurrent Access |
Export data from database files into sequential (flat) or binary files |
MUPIP EXTRACT |
Although MUPIP EXTRACT command works with concurrent access, it implicitly freezes the database to prevent updates. Therefore, from an application standpoint, you might plan for a standalone access during a MUPIP EXTRACT operation. |
Prevent updates to database files |
MUPIP FREEZE |
Standalone access. |
Check the integrity of GDS databases |
MUPIP INTEG |
Concurrent access. However, standalone access is required if MUPIP INTEG specifies FILE |
Import data into databases |
MUPIP LOAD |
Although MUPIP LOAD works with concurrent access, you should always assess the significance of performing a MUPIP LOAD operation when an application is running because it may result in an inconsistent application state for the database. |
Defragment database files to improve performance |
MUPIP REORG |
Concurrent access. |
Send an asynchronous signal to a YottaDB process |
MUPIP INTRPT |
Non-database access. |
Reports information related to relinkctl files and their associated shared memory segments. |
MUPIP RCTLDUMP |
Non-database access. |
Stop YottaDB processes |
MUPIP STOP |
Non-database access. |
Note
MUPIP commands that need standalone access issue a MUUSERLBK error on a crashed replication-enabled database and MUUSERECOV error in case of a non-replicated-but-journaled database.
Synopsis
The general format of MUPIP commands is:
mupip command [-qualifier[...]] [object[,...]] [destination]
MUPIP allows the abbreviation of commands and qualifiers. In each section describing a command or qualifier, the abbreviation is also shown (for example, B[ACKUP]). The abbreviated version of B[ACKUP] you can use on the command line is B. To avoid future compatibility problems and improve readability, specify at least four characters when using MUPIP commands in scripts.
Although you can enter commands in both upper and lower case (the mupip program name itself must be in lower case on UNIX/Linux), the typographical convention used in this chapter is all small letters for commands. Another convention is in the presentation of command syntax. If the full format of the command is too long for a single line of print, the presentation wraps around into additional lines.
$ mupip backup -bytestream -transaction=1 accounts,history,tables,miscellaneous /var/production/backup/
When you enter a MUPIP command, one of its variable arguments is the region-list. region-list identifies the target of the command and may include the UNIX wildcards "?" and "*". Region-lists containing UNIX wildcard characters must always be quoted, for example, "*" to prevent inappropriate expansion by the UNIX shell. Similarly, for file and directory names you might want to avoid non-graphic characters and most punctuations except underscores (_), not because of YottaDB conventions but because of inappropriate expansion by UNIX shells.
MUPIP qualifier values are restricted only by the maximum size of the command input line, which is 4KB on some systems and upto 64KB on others.
Note
MUPIP sends its output to stderr not stdout. On shells such as bash stderr can be redirected to stdout by specifying 2>&1 on the command line.
MUPIP Commands and Qualifiers
Also see: Quick Reference Summary.
The MUPIP commands described in this section are used for common database operations and serves as the foundation for more advanced functionality like Journaling and Replication.
BACKUP
Saves the contents of the database. It provides a consistent application snapshot across all database regions involved in the backup operation. The format of the MUPIP BACKUP command is:
B[ACKUP]
[
-BK[UPDBJNL]={DISABLE|OFF}]
-BY[TESTREAM] [-NET[TIMEOUT]]
-DA[TABASE]
-DBG
-[NO]NEWJNLFILES[=[NO]PREVLINK],[NO]S[YNC_IO]]
-[NO]O[NLINE]
-REC[ORD]
-REPLA[CE]
-REPLIC[ATION]=ON
-REPLIN[STANCE]=target_location
-RET[RY]=count
-SH[OWPROGRESS]
-S[INCE]={DATABASE|BYTESTREAM|RECORD}
-T[RANSACTION]=hexadecimal_transaction_number
-VERBOSE
] region-list[,...] destination-list
Note
MUPIP BACKUP does a more comprehensive job of managing backup activities than other backup techniques such as a SAN backup, disk mirroring, or a file system snapshot because it integrates journal management, instance file management, and records timestamps in the database file headers. To use other techniques, you must first freeze all regions concurrently with a command such as MUPIP FREEZE in order to ensure a consistent copy of files with internal structural integrity. YottaDB neither endorses nor tests any third party products for backing up a YottaDB database.
MUPIP BACKUP supports two methods of database backup: BYTESTREAM and DATABASE. MUPIP BACKUP BYTESTREAM directs the output to a broad range of devices, including disks, TCP sockets, and pipes. MUPIP BACKUP DATABASE directs the output to files on filesystems with random access (e.g., disks).
[NO]ONLINE qualifier determines whether MUPIP BACKUP should suspend updates to regions. For example, MUPIP BACKUP NOONLINE suspends updates to all regions from the time it starts the first region until it finishes the last region. However, it does not suspend processes that only read from the database.
By default, MUPIP BACKUP is DATABASE ONLINE.
If any region name does not map to an existing accessible file, or if any element of the destination list is invalid, BACKUP rejects the command with an error.
region-list may specify more than one region of the current global directory in a list. Regions are case-insensitive, separated by a comma, and wildcards can be used to specify them. Any region-name may include the wildcard characters * and % (remember to escape them to protect them from inappropriate expansion by the shell). Any region name expansion occurs in M (ASCII) collation order. MUPIP BACKUP processes regions in the listed order after wildcard expansion.
Depending on the type of backup, destination-list may be a single directory, or a comma separated list of destinations including files, piped commands, or a TCP socket address (a combination of IPv4 or IPV6 hostname and a port number).
Region-list and destination-list items are matched in order - the first region is mapped to the first destination, the second to the second destination, and so on. If YottaDB encounters a region mapped to a directory, YottaDB treats that directory as the destination for all subsequent regions in the region-list.
YottaDB implicitly timestamps both BYTESTREAM and DATABASE backups using relative timestamps (transaction numbers). You can also explicitly specify a RECORD timestamp. You can use these timestamps as reference points for subsequent backups.
It takes approximately one (1) minute (per region) for BACKUP ONLINE to give up and bypass KILLs in progress; backup does not wait for Abandoned Kills to clear.
The environment variable ydb_baktmpdir specifies the directory where mupip backup creates temporary files. If ydb_baktmpdir is not defined, YottaDB uses the environment variable gtm_baktmpdir, and if that too is not defined, the deprecated GTM_BAKTMPDIR environment variable, if defined. If none of these is defined, MUPIP BACKUP uses the current working directory.
When you restrict access to a database file, YottaDB propagates those restrictions to shared resources associated with the database file, such as semaphores, shared memory, journals and temporary files used in the course of MUPIP BACKUP.
YottaDB supports only one concurrent ONLINE backup on a database. MUPIP BACKUP displays the BKUPRUNNING message if started when there is an already running BACKUP.
Unless the REPLACE option is used, MUPIP BACKUP protects against overwriting of existing destination files. However, it cannot protect other destinations, for example, if the destination is a pipe into a shell command that overwrites a file.
Before Starting a MUPIP Backup
Perform the following tasks before you begin a database backup.
Ensure adequate disk space for target location and temporary files. Set the environment variable ydb_baktmpdir to specify the directory where MUPIP BACKUP creates temporary files. If ydb_baktmpdir is not defined, YottaDB uses the environment variable gtm_baktmpdir, and if that too is not defined, the deprecated GTM_BAKTMPDIR environment variable, if defined. If none of these is defined, MUPIP BACKUP uses the current working directory. Do not place temporary files in the current directory for large databases in production environments.
If you are backing up any encrypted regions, you need to backup your $ydb_crypt_config / $gtmcrypt_config file and protect the encryption keys. You also need to backup the related algorithms used to retrieve or regenerate keys. For example, if you are using GnuPG for encryption and signing services, you must backup the relevant files in your $GNUPGHOME directory as part of your backup in addition to preserving the encryption key. Look in the manpages of GnuPG (man gpg) to see what files are required. Remember that if you cannot retrieve the encryption key which was in use at the time of the backup, there is no way to decrypt the data from the encrypted regions. Therefore, ensure that you have procedures in place to securely handle storage and retrieval of the encryption keys.
When using replication, ensure that a Source/Receiver process is alive (MUPIP REPLICATE SOURCE CHECKHEALTH). Always backup the replication instance file with the database (MUPIP BACKUP REPLINSTANCE).
If you intend to use a DATABASE backup at the same time in the same computer system as the source database, be sure to disable journaling in the backed up database with BKUPDBJNL=DISABLE.
We recommend switching journal files as part of the backup command using NEWJNLFILES. This aligns the journal files with the backup and simplifies journal file retention. Use the NOPREVLINK option for this qualifier with caution if the original database is used for replication. If the link to the previous generation journal file is cut, then the source server cannot supply transactions from the prior generation journal files.
If you follow separate procedures for backup and archival (moving to secondary storage), you can save time by starting archival as soon as MUPIP BACKUP completes the process of creating a backup database file for a region. You do not need to wait for MUPIP BACKUP to complete processing for all regions before starting archival. For example, a message like:
DB file /home/jdoe/.yottadb/r1.10/g/ydb.dat backed up in file /backup/ydb.dat
Transactions up to 0x0000000000E92E04 are backed up.
confirms that ydb.dat is backed up correctly and is ready for archival.
Determine an appropriate frequency, timing, and backup method (BYTESTREAM or DATABASE) based on business and operational considerations.
Ensure the user issuing backup commands has appropriate permissions before starting the backup. Backup files are owned by the user running MUPIP BACKUP.
There is one circumstance under which a MUPIP BACKUP is not advised. When your operational procedures call for taking backups of unmodified databases and journal files when rebooting a system after a crash, use an underlying operating system command (cp, cpio, gzip, tar, and so on) which will open the files read-only. Note that for ordinary system crashes where the system simply stops writing to open files at power down, you can use MUPIP JOURNAL to recover journaled database files, and taking backups on reboot should not be required. However, for system crashes with the possibility of damage to files already written to disk (for example, if the crash involved an IO controller with the potential for having written random data to disk immediately prior to power down), such backups on reboot are appropriate.
Example:
$ mupip backup "*" /ydb/bkup
This example creates ready-to-run database backup of all regions.
BACKUP Qualifiers
-BKupdbjnl
A backup database shares the same journaling characteristics of the source database. However, with BKUPDBJNL you can disable or turn off journaling in the backup database. Be sure to use BKUPDBJNL=DISABLE if you intend to open your backup database on the same system as the source database to ensure that two database files do not inadvertently point to the same journal file.
The format of the BKUPDBJNL qualifier is:
-BK[UPDBJNL]={DISABLE|OFF}
Specify DISABLE to disable journaling in the backup database.
Specify OFF to turn off journaling in the backup database.
Only one of the qualifiers DISABLE or OFF can be specified at any given point.
-BYtestream
Transfers MUPIP BACKUP output to a TCP connection, file (or a backup directory), or a pipe. If there are multiple .dat files, BYTESTREAM transfers output to a comma separated list of TCP connections, incremental backup files and/or directories, or pipes. When used with SINCE or TRANSACTION, MUPIP BACKUP allows incremental backup, that is, includes database blocks that have changed since a prior point specified by the SINCE or TRANSACTION.
Note
MUPIP BACKUP output to a TCP connection saves disk I/O bandwidth on the current system.
All bytestream backups needs to be restored to a random access file (with MUPIP RESTORE) before being used as a database file. BYTESTREAM can also send the output directly to a listening MUPIP RESTORE process via a TCP/IP connection or a pipe.
The format of the BYTESTREAM qualifier is:
-BY[TESTREAM]
BYTESTREAM is compatible with SINCE and TRANSACTION.
INCREMENTAL is deprecated in favor of BYTESTREAM. For upward compatibility, MUPIP temporarily continues to support the deprecated INCREMENTAL.
-Database
Creates a disk-to-disk backup copy of the files of all selected regions. DATABASE backup copy is a ready-to-use YottaDB database unlike BYTESREAM backups which must be restored.
The format of the DATABASE qualifier is:
-D[ATABASE]
By default, MUPIP BACKUP uses DATABASE.
The DATABASE qualifier is only compatible with the [NO]NEW[JNLFILES], ONLINE, and RECORD qualifiers.
COMPREHENSIVE is deprecated in favor of DATABASE. For upward compatibility, MUPIP temporarily continues to support the deprecated COMPREHENSIVE.
-DBG
Produces verbose output to help with debugging. We recommend using VERBOSE instead.
-NETtimeout
Specifies the timeout period when a bytestream BACKUP data is sent over a TCP/IP connection. The format of the NETTIMEOUT qualifier is:
NET[TIMEOUT]=seconds
The default value is 30 seconds.
Use only with BYTESTREAM and RESTORE.
-NEWJNLFILES
Determines the journaling characteristics of the database files as part of the backp. This qualifier is effective only for an ONLINE backup (the default), for database regions that have journaling enabled. When new journal files are created, journal records for all updates up to and including the last update in the backed up database files are in the prior journal files, and journal records for updates not in the backed up database files are in the new journal files.
The format of the NEWJNLFILES qualifier is:
-[NO]NEWJNLFILES[=[NO]PREVLINK],[NO]S[YNC_IO]]
NEWJNLFILES can take the following three values:
PREVLINK: New journal files are created with back links to the prior journal files. This is the default value.
NOPREVLINK: New journal files are created without back links to the prior journal files.
SYNC_IO: Specifies that the new journal files should use the SYNC_IO option.
NOSYNC_IO: Specifies that new journal files should not use the SYNC_IO option.
NONEWJNLFILES causes journaling to continue with the current journal files. It does not accept any arguments.
The default is NEWJNLFILES=PREVLINK, with the current values of SYNC_IO for each region retained by the new journal files.
-Online
Specifies that while the MUPIP BACKUP operation is active, processes can continue to update the database without affecting the result of the backup. The format of the ONLINE qualifier is:
-[NO]O[NLINE]
MUPIP BACKUP ONLINE creates a backup of the database as of the moment the backup starts. If running processes subsequently update the database, the backup does not reflect those updates.
MUPIP BACKUP ONLINE on region(s) waits for up to one minute so any concurrent KILL or MUPIP REORG operations can complete. If the KILL or MUPIP REORG operations do not complete within one minute, MUPIP BACKUP ONLINE starts the backup with a warning that the backup may contain incorrectly marked busy blocks. Such blocks waste space and can desensitize operators to much more dangerous errors, but otherwise do not affect database integrity. If you get such an error, consider whether to stop the backup and restart it when KILL or MUPIP REORG operations are less likely to interfere. MUPIP STOP on a process performing a KILL or MUPIP REORG operation may leave the database with incorrectly marked busy blocks. In this situation, YottaDB converts the ongoing KILLs flag to an Abandoned KILLs flag. If MUPIP BACKUP ONLINE encounters ADANDONED_KILLS, it gives a message and then starts the backup. An ABANDONED_KILLS error means that both the original database and the backup database possibly have incorrectly busy blocks which should be corrected.
By default, MUPIP BACKUP is ONLINE.
-Record
Timestamps (in the form of a transaction number) a database file to mark a reference point for subsequent bytestream, database, or custom backup protocols. Even though DATABASE and BYTESTREAM both mark their own relative timestamps, RECORD provides an additional timestamp option. MUPIP FREEZE also provides the RECORD qualifier because a FREEZE may be used to set the database up for a SAN or disk-mirror based backup mechanism.
The format of the RECORD qualifier is:
-R[ECORD]
Use RECORD (with the hyphen) to timestamp a reference point and use RECORD as a keyword (as in SINCE=RECORD) to specify the starting point for a MUPIP BACKUP operation.
RECORD replaces the previously RECORDed transaction identifier for the database file.
-REPLAce
Overwrites the existing destination files.
The format of the REPLACE qualifier is:
-[REPLA]CE
By default, MUPIP BACKUP protects against overwriting the destination files. REPLACE disables this default behavior.
REPLACE is compatible only with DATABASE.
-REPLICation
Recovering from the replication WAS_ON state involves taking a backup with new journal files and turning replication back on. The REPLICATION qualifier allows this to be performed as part of the backup.
-[REPLIC]ation=ON
REPLICATION is compatible only with DATABASE.
If replication in the database file being backed up is already in the ON state, this option is a no-op.
If replication in the database file being backed up is in the WAS_ON state, and NEWJNLFILES is specified or defaulted, REPLICATION=ON changes the replication state in the database file header to ON. NOPREVLINK is automatic in this case, because there is no prior generation journal file to which the new file can link. Note that even with new journal files, the system should not be considered in a safe state till the MUPIP BACKUP completes.
-REPLINstance
Specifies the target location, a directory or a file, for the backup of the replication instance file. If a directory is specified, the file has the same name as replication instance file being backed up.
Note
In replicated instances, always back the replication instance file along with database files. A Source Server for the instance must be started at least once before backing up the replication instance file.
The format of the REPLINSTANCE qualifier is:
-REPLIN[STANCE]=<target_location>
RETry
Specifies the number of times that MUPIP BACKUP should retry the backup if it fails. If RETRY is not specified, MUPIP BACKUP retries the file copy operation once if the first attempt fails on a retriable reason. In case of an error, retry attempts are always based on cp or pax even when another copy mechanism is available. If --retry=0 is specified, MUPIP BACKUP makes no attempt to retry when it encounters a failure.
The format of the RETRY qualifier is:
--ret[ry]=<count>
SHowprogress
When the backup copy mechanism supports monitoring, SHOWPROGRESS periodically displays the transfer progress, estimated time left, speed, and the number of transaction applied to the database during backup.
The format of the SHOWPROGRESS qualifier is:
-sh[owprogress]
Incompatible with BYTESTREAM.
-Since
Includes blocks changed since the last specified backup. The format of the SINCE qualifier is:
-S[ince]={Database|Bytestream|Record}
D[atabase] - Backup all changes since the last MUPIP BACKUP DATABASE.
B[ytestream] - Backup all changes since the last MUPIP BACKUP BYTESTREAM.
R[ecord] - Backup all changes since the last MUPIP BACKUP RECORD.
By default, MUPIP BACKUP BYTESTREAM operates as -since=database.
Incompatible with: TRANSACTION.
-Transaction
Specifies the transaction number of a starting transaction that causes BACKUP BYTESTREAM to copy all blocks that have been changed by that transaction and all subsequent transactions. The format of the TRANSACTION qualifier is:
-T[RANSACTION]=transaction-number
A Transaction number is always a 16 digit hexadecimal number. It appears in a DSE DUMP FILEHEADER with the label "Current transaction".
If the transaction number is invalid, MUPIP BACKUP reports an error and rejects the command.
It may be faster than a DATABASE backup, if the database is mostly empty.
Incompatible with: DATABASE, SINCE.
Note
A point in time that is consistent from an application perspective, is unlikely to have the same transaction number in all database regions. Therefore, except for TRANSACTION=1, this qualifier is not likely to be useful for any backup involving multiple regions.
-VERBOSE
Produces verbose output to help with debugging.
Examples for MUPIP BACKUP
Example:
$ mupip backup -bytestream MAMMALS,CRUSTACEANS bkup
If the global directory has regions MAMMALS and CRUSTACEANS that map to files called LINNAEUS.DAT and BRUNNICH.DAT (no matter which directory or directories the files reside in). Then the above example creates bytestream backup files MAMMALS.DAT and CRUSTACEANS.DAT in the bkup directory with updates since the last DATABASE backup.
Example:
$ mupip backup -bkupdbjnl="OFF" "*" bkup
This command backs up all regions in the bkup directory, with journaling turned off in the backup database files.
Example:
$ mupip backup -bytestream "*" tcp://philadelphia:7883,tcp://tokyo:8892
Assuming a global directory with two regions pointing to ACN.DAT and HIST.DAT, this example creates a backup of ACN.DAT to a possible MUPIP RESTORE process listening at port 7883 on server philadelphia and HIST.DAT to a possible MUPIP RESTORE process listening at port 8893 on server tokyo.
Always specify the <machine name> and <port> even if both backup and restore are on the same system, and ensure that the MUPIP RESTORE process is started before the MUPIP BACKUP process.
Example:
$ mupip backup -database -noonline "*" bkup
DB file /home/ydbnode1/yottadbuser1/yottadb.dat backed up in file bkup/yottadb.dat
Transactions up to 0x00000000000F42C3 are backed up.
BACKUP COMPLETED.
This command creates a disk-to-disk backup copy of all regions of the current database in directory bkup. YottaDB freezes all the regions during the backup operation.
Example:
$ mupip backup -bytestream -nettimeout=420 DEFAULT tcp://${org_host}:6200
This command creates a backup copy of the DEFAULT region with timeout of 420 seconds.
Example:
$ mupip backup -bytestream DEFAULT '"| gzip -c > online5pipe.inc.gz"'
This command sends (via a pipe) the backup of the DEFAULT region to a gzip command.
Example:
$ mupip backup -online DEFAULT bkup
DB file /ydbnode1/yottadbuser1/yottadb.dat backed up in file bkup/yottadb.dat
Transactions up to 0x00000000483F807C are backed up.
BACKUP COMPLETED.
This command creates a backup copy of the DEFAULT region of the current database in directory bkup. During the backup operation, other processes can read and update the database.
Example:
$ mupip backup -record DEFAULT bkup
This command sets a reference point and creates a backup copy of the DEFAULT region of the current database in directory bkup.
Example:
$ mupip backup -online -record DEFAULT bkup1921
DB file /home/mammals/yottadb.dat backed up in file bkup1921/yottadb.dat
Transactions up to 0x00000000000F4351 are backed up.
Example:
$ mupip backup -bytestream -since=record DEFAULT bkup1921onwards
MUPIP backup of database file /home/mammals/yottadb.dat to bkup1921onwards/yottadb.dat
DB file /home/mammals/yottadb.dat incrementally backed up in file bkup1921onwards/yottadb.dat
6 blocks saved.
Transactions from 0x00000000000F4351 to 0x00000000000F4352 are backed up.
BACKUP COMPLETED.
The first command sets a reference point and creates a backup copy of the DEFAULT region of the current database in directory bkup1921. The second command completes a bytestream backup starting from the reference point set by the first command.
Example:
$ mupip backup -bytestream -transaction=1 DEFAULT bkup_dir
MUPIP backup of database file /ydbnode1/yottadbuser1/yottadb.dat to bkup_dir/yotttadb.dat
DB file /ydbnode1/yottadbuser1/yottadb.dat incrementally backed up in file bkup/yottadb.dat
5 blocks saved.
Transactions from 0x0000000000000001 to 0x0000000000000003 are backed up.
BACKUP COMPLETED.
This command copies all in-use blocks of the DEFAULT region of the current database to directory bkup_dir.
Example:
$ mupip backup -newjnlfiles=noprevlink,sync_io "*" backupdir
This example creates new journal files for the current regions, cuts the previous journal file link for all regions in the global directory, enables the SYNC_IO option and takes a backup of all databases in the directory backupdir.
CREATE
Creates and initializes database files using the information in the Global Directory file specified by $ydb_gbldir. If a file already exists for any segment, MUPIP CREATE takes no action for that segment.
The format of the CREATE command is:
cr[eate] [-[no]v6] [-r[egion]=region-name | -r[egion] region name]
The single optional -region qualifier specifies a region for which to create a database file.
One YottaDB database file can grow to a maximum size of 17,179,869,184(16Gi) blocks. This means, for example, that with a 4KiB block size, the maximum single database file size is 64TiB (4KiB*16Gi). This is just the size of one database file – a logical database (a global variable namespace) can consist of an arbitrary number of database files, i.e., the size of a YottaDB database is limited only by available storage.
Note that a MUPIP CREATE command that explicitly specifies a region which is tagged as AutoDB, creates the database file for that region if it does not exist.
REGION
Specifies a single region for creation of a database file. By default, MUPIP CREATE creates database files for all regions in the current Global Directory that do not already have a database file.
The format of the REGION qualifier is:
-r[egion]=region-name | -r[egion] region-name
The region-name is case-insensitive.
V6
Directs MUPIP to create an r1.x (GT.M V6.x) compatible database file. When the environment variable ydb_db_create_ver specifies the creation of r1.x compatible database files, this option can be negated (nov6) to create r2.x database files.
The format of the V6 qualifier is:
-[no]v6
Examples for MUPIP CREATE
Example:
$ mupip create -region=YDBOCTO
This command creates the database file specified by the Global Directory for region YDBOCTO.
DOWNGRADE
The MUPIP DOWNGRADE command changes the file header format to a previous version number. The format of the MUPIP DOWNGRADE command is:
D[OWNGRADE] -V[ERSION]={r1.10|r1.20} file-name
Note
You must perform a database integrity check using the -noonline parameter prior to downgrading a database. The integrity check verifies and clears database header fields required for an orderly downgrade. If an integrity check is not possible due to time constraints, please rely on a rolling upgrade scheme using replication and/or take a backup prior to upgrading the database.
-VERSION={version}
For more information on the downgrade criteria for your database, refer to the release notes document of your current YottaDB version.
Examples for MUPIP DOWNGRADE
Example:
$ mupip downgrade yottadb.dat
This command changes the file-header of yottadb.dat to the format in the previous version.
DUMPFHEAD
The MUPIP DUMPFHEAD command displays information about one or more database files. The format of the MUPIP DUMPFHEAD command is:
DU[MPFHEAD] {-F[ILE] file-name | -R[EGION]=region-list | -R[EGION] region-list}
-FILE file-name
Specifies the name of the database file for the MUPIP DUMPFHEAD operation. FILE does not require a Global Directory. The format of the FILE qualifier is:
-F[ILE] file-name
The database filename must include the absolute or relative path.
The FILE qualifier is incompatible with the REGION qualifier.
-REGION region-list
Specifies that the INTEG parameter identifies one or more regions rather than a database file. The format of the REGION qualifier is:
-R[EGION]=region-list | -R[EGION] region-list
The region-list identifies the target of DUMPFHEAD. The region-list may specify more than one region of the current global directory in a list. Regions are case-insensitive, separated by a comma, and wildcards can be used to specify them. Any region-name may include the wildcard characters * and ? (remember to escape them to protect them from inappropriate expansion by the shell). Any region name expansion occurs in M (ASCII) collation order. INTEG processes regions based on the order specified in the list after wild-card expansion.
The REGION qualifier is incompatible with the FILE qualifier.
Examples for MUPIP DUMPFHEAD
Example:
$ mupip dumpfhead -file yottadb.dat
This command lists information about the database file yottadb.dat in the current working directory.
$ mupip dumpfhead -region "*"
This command lists information about all the database files mapped by the global directory specified by $ydb_gbldir.
ENDIANCVT
Converts a database file from one endian format to the other (BIG to LITTLE or LITTLE to BIG). The format of the MUPIP ENDIANCVT command is:
ENDIANCVT [-OUTDB=<outdb-file>] -OV[ERRIDE] <db-file>
<db-file> is the source database for endian conversion. By default ENDIANCVT converts <db-file> in place.
outdb writes the converted output to <outdb-file>. In this case, ENDIANCVT does not modify the source database <db-file>.
ENDIANCVT produces a <outdb-file> of exactly the same size as <db-file>.
Note
Ensure adequate storage for <outdb-file> to complete the endian conversion successfully.
ENDIANCVT requires standalone access to the database.
YottaDB displays a confirmation request with the "from" and "to" endian formats to perform the conversion. Conversion begins only upon receiving positive confirmation, which is a case-insensitive "yes".
In a multi-site replication configuration, the receiver server automatically detects the endian format of an incoming replication stream and converts it into the native endian format. See the Database Replication chapter for more information.
Encrypted database files converted with ENDIANCVT require the same key and the same cipher that were used to encrypt them.
Note
YottaDB on a big endian platform can convert a little endian database into big endian and vice versa; as can YottaDB on a little endian platform. YottaDB (run-time and utilities other than MUPIP ENDIANCVT) on a given endian platform opens and processes only those databases that are in the same endian format. An attempt to open a database of a format other than the native endian format produces an error.
-OVERRIDE
Enables MUPIP ENDIANCVT to continue operations even if YottaDB encounters the following errors:
"minor database format is not the current version"
"kills in progress"
"a GT.CM server is accessing the database"
Note that the OVERRIDE qualifier does not override critical errors (database integrity errors, and so on) that prevent a successful endian format conversion.
Examples for MUPIP ENDIANCVT
$ mupip endiancvt yottadb.dat -outdb=yottadb_cvt.dat
Converting database file yottadb.dat from LITTLE endian to BIG endian on a LITTLE endian system
Converting to new file yottadb_cvt.dat
Proceed [yes/no] ?
This command detects the endian format of yottadb.dat and converts it to the other endian format if you type yes to confirm.
EXIT
Stops a MUPIP process and returns control to the process from which MUPIP was invoked.
The format of the MUPIP EXIT command is:
EXI[T]
The EXIT command does not accept any qualifiers.
EXTEND
Increases the size of a database file. By default, YottaDB automatically extends a database file when there is available space.
The format of the MUPIP EXTEND command is:
EXTE[ND] [-BLOCKS=<data-blocks-to-add>] region-name
The only qualifier for MUPIP EXTEND is BLOCKS.
The required region-name parameter specifies the name of the region to expand.
EXTEND uses the Global Directory to map the region to the dynamic segment and the segment to the file.
-Blocks
Specifies the number of GDS database blocks by which MUPIP should extend the file. GDS files use additional blocks for bitmaps. MUPIP EXTEND adds the specified number of blocks plus the bitmap blocks required as overhead. For more information about bitmaps, refer to Chapter 9: “Database Structure (GDS file)”.
The format of the BLOCK qualifier is:
-BLOCKS=data-blocks-to-add
By default, EXTEND uses the extension value in the file header as the number of GDS blocks by which to extend the database file. You can specify as many blocks as needed as long as you are within the maximum total blocks limit (which could be as high as 224 million GDS blocks).
Examples for MUPIP EXTEND
$ mupip extend DEFAULT -blocks=400
This command adds 400 GDS database blocks to region DEFAULT.
Example:
$ mupip extend MAMMALS -blocks=100
This command adds 100 GDS database blocks to the region MAMMALS.
EXTRACT
Backs up certain globals or extracts data from the database for use by another system. The MUPIP EXTRACT command copies globals from the current database to a sequential output file in one of three formats - GO, BINARY, or ZWR. The format of the MUPIP EXTRACT command is:
EXTR[ACT]
[
-FO[RMAT]={GO|B[INARY]|Z[WR]}
-FR[EEZE]
-LA[BEL]=text
-[NO]L[OG]
-[NO]NULL_IV
-R[EGION]=region-list | -R[EGION] region-list
-SE[LECT]=global-name-list]
]
{-ST[DOUT]|file-name}
By default, the FORMAT of MUPIP EXTRACT is ZWR.
MUPIP EXTRACT uses the Global Directory to determine which database files to use.
MUPIP EXTRACT supports user collation routines. When used without the FREEZE qualifier, EXTRACT may operate concurrently with normal YottaDB database access.
To ensure that MUPIP EXTRACT reflects a consistent application state, suspend the database updates to all regions involved in the extract, typically with the FREEZE qualifier, or backup the database with the ONLINE qualifier and extract files from the backup.
EXTRACT places its output in the file defined by the file-name.
In UTF-8 mode, MUPIP EXTRACT writes a sequential output file in the UTF-8 character encoding. Ensure that the MUPIP EXTRACT commands and corresponding MUPIP LOAD commands execute with the same setting for the environment variable ydb_chset.
The GO format is not supported for UTF-8 mode. Use BINARY or ZWR formats in UTF-8 mode.
For information on extracting globals with the %GO utility, refer to the "Utility Routines" chapter of the Programmer's Guide. MUPIP EXTRACT is typically faster, but %GO can be customized.
The following sections describe the qualifiers of MUPIP EXTRACT command.
-FORMAT
Specifies the format of the output file. The format of the FORMAT qualifier is:
-FO[RMAT]=format_code
The format code is any one of the following:
B[INARY] - Binary format, used for database reorganization or short term backups. MUPIP EXTRACT FORMAT=BINARY works much faster than MUPIP EXTRACT FORMAT=GO and MUPIP EXTRACT FORMAT=ZWR. Note: There is no defined standard to transport binary data from one YottaDB implementation to another. Furthermore, YottaDB reserves the right to modify the binary format in new versions. The first record of a BINARY format data file contains the header label. The header label is 102 characters long. The following table illustrates the components of the header label.
Characters
Explanation
1-2
Hexadecimal representation of the length of the label (by default 64 - decimal 100).
3-28
Fixed-length (26 bytes) ASCII text containing:
"GDS BINARY EXTRACT LEVEL 6": when no region is encrypted.
"GDS BINARY EXTRACT LEVEL 8": when one more regions are encrypted using null IVs.
"GDS BINARY EXTRACT LEVEL 9": when one or regions are encrypted using non-null IVs.
29-42
Fixed-length (14 bytes) ASCII text: Date and time of extract in the $ZDATE() format: "YEARMMDD2460SS"
43-49
Fixed-length (7 bytes) ASCII text: Decimal maximum block size of each region from which data was extracted
50-56
Fixed-length (7 bytes) ASCII text: Decimal maximum record size of each region from which data is extracted
57-63
Fixed-length (7 bytes) ASCII text: Decimal maximum key size of each region from which data is extracted
64-70
Fixed-length (7 bytes) ASCII text:Boolean indicator of Standard NULL collation (1) or historical null collation (0).
71-102
Fixed-length (32 bytes) ASCII text: Space-padded label specified by the -LABEL qualifier; the default LABEL is "MUPIP EXTRACT". For extracts in UTF-8 mode, YottaDB prefixes UTF-8 and a space to -LABEL.
GO - Global Output format, used for files to transport or archive. FORMAT=GO stores the data in record pairs. Each global node produces two records - the first contains the key and the second contains the value. GO format is only supported in M mode.
ZWR - ZWRITE format, used for files to transport or archive that may contain non-graphical information. Each global node produces one record with both a key and data. Note that for non-ASCII data, M mode and UTF-8 mode extracts can differ, as the definition of printable characters differs.
GO and ZWR format output files have two header records. The first is a text label (refer to the LABEL qualifier), defaulting to: "YottaDB MUPIP EXTRACT" followed by the command line used to generate the extract, including the full path to the mupip executable, followed by UTF-8 if the process ran in UTF-8 mode; the second is the date and time of extract in $ZDATE() format DD-MON-YEAR 24:60:SS, and, for ZWR extracts, the text "ZWR".
Note
ZWR format is suitable for all data. Use GO format for data that contains only printable characters and spaces, as some characters (such as linefeed) can corrupt the output file format.
The GO and ZWR format output header was enhanced in release r1.30.
-FREEZE
Prevents database updates to all database files from which the MUPIP EXTRACT command is copying records. FREEZE ensures that a MUPIP EXTRACT operation captures a "sharp" image of the globals, rather than one "blurred" by updates occurring while the copy is in progress.
The format of the FREEZE qualifier is:
-FR[EEZE]
By default, MUPIP EXTRACT does not "freeze" regions during operation.
-LABEL
Specifies the text string that becomes the first record in the output file. MUPIP EXTRACT FORMAT=BINARY truncates the label text to 32 characters. The format of the LABEL qualifier is:
-LA[BEL]=text
By default, EXTRACT uses the label "MUPIP EXTRACT."
For more detailed information about the FORMAT=BINARY header label, refer to the description of EXTRACT FORMAT=BINARY.
-LOG
Displays a message on stdout for each global extracted with the MUPIP EXTRACT command. The message displays the number of global nodes, the maximum subscript length and maximum data length for each global. The format of the LOG qualifier is:
-[NO]LO[G]
By default, EXTRACT operates -LOG.
-NULL_IV
Creates an encrypted binary extract with null IVs from a database with non-null IVs, which can be restored to a version that does not support non-null IVs. The format of the NULL_IV qualifier is:
-[NO]NULL_IV
Older versions of YottaDB used empty (all zeros or "NULL_IV") initialization vectors(IVs) to encrypt or decrypt FORMAT="BINARY" extracts.
The current and later versions use non-zero IVs.
Use the NULL_IV qualifier only on encrypted databases to create an encrypted binary extract in GDS BINARY EXTRACT LEVEL 8 format. This format can load data on any encrypted YottaDB database created with an older version.
The default is NONULL_IV which produces a binary extract in GDS BINARY EXTRACT LEVEL 9 format.
-REGION
Restricts MUPIP EXTRACT to a set of regions. The format of the REGION qualifier is:
-R[EGION]=region-list | -R[EGION] region-list
region-list may specify more than one region of the current global directory in a list. Regions are case-insensitive, separated by a comma, and wildcards can be used to specify them. Any region-name may include the wildcard characters * and % (remember to escape them to protect them from inappropriate expansion by the shell). Any region name expansion occurs in M (ASCII) collation order.
-SELECT
Specifies globals for a MUPIP EXTRACT operation. The format of the SELECT qualifier is:
-S[ELECT]= global-specification
By default, EXTRACT selects all globals, as if it had the qualifier SELECT=*
The caret symbol (^) in the specification of the global name is optional.
The global-specification can be:
A global name, such as MEF. In this case, MUPIP EXTRACT selects only global ^MEF.
A range of global names, such as A7:B6. In this case, MUPIP EXTRACT selects all global names between ^A7 and ^B6, inclusive.
A list, such as A,B,C. In this case, MUPIP EXTRACT selects globals ^A, ^B, and ^C.
A suffix with a global name. For example, PIGEON* selects all global names from ^PIGEON through ^PIGEONzzzzz. You can use suffixes with a global name or a list.
Note
If the rules for selection are complex, it may be easier to construct an ad hoc Global Directory that maps the global variables to be extracted to the database file. This may not be permissible if the database file is part of a replicated instance. If this is the case, work with a backup of the database.
-STDOUT
Redirects the database extract to the standard output stream. The format of the STDOUT qualifier is:
-ST[DOUT]
Examples for MUPIP EXTRACT
Example:
$ mupip extract -format=go -freeze big.glo
This command prevents database updates during a MUPIP EXTRACT operation.
Example:
$ mupip extract -format=GO yottadb_i.go
This command creates an extract file called yottadb_i.go in "Global Output" format. Use this format to transport or archive files. The first record of a GO format file contains the header label, "MUPIP EXTRACT," as text.
Example:
$ mupip extract -format=BINARY v5.bin
This command creates an extract file called v5.bin in Binary format. Use this format for reorganizing a database or for short-term backups.
Example:
$ mupip extract -format=ZWR -LABEL=My_Label My_Extract_File
This example extracts all globals from the current database to file My_Extract_File (in ZWRITE format) with label My_Label.
Example:
$ mupip extract -nolog FL.GLO
This command creates a global output file, FL.GLO, (which consists of all global variables in the database) without displaying statistics on a global-by-global basis. As there is no label specified, the first record in FL.GLO contains the text string "MUPIP EXTRACT."
Example:
$ mupip extract -select=Tyrannosaurus /dev/tty
This command instructs EXTRACT to dump the global ^Tyrannosaurus to the device (file-name) /dev/tty.
FREEZE
Temporarily suspends (freezes) updates to the database after ensuring a consistent state between memory and secondary storage, which, with ACCESS_METHOD=BG, means after flushing global buffers. If you prefer a non-YottaDB utility to perform a backup or reorganization, you might use this facility to provide standalone access to your YottaDB database. You might use MUPIP FREEZE to suspend (and later resume) database updates for creating mirrored disk configuration or re-integrating a mirror.
BACKUP, INTEG, and REORG operations may implicitly freeze and unfreeze database regions. However, for most operations, this freeze/unfreeze happens internally and is transparent to the application.
The format of the MUPIP FREEZE command is:
F[REEZE] {-OF[F] [-OV[ERRIDE]]|-ON [[-ONL[INE] [-[NO]AUTORELEASE]] | [-NOONL[INE]] [-R[ECORD]]]} region-list
The region-list identifies the target of the FREEZE. region-list may specify more than one region of the current global directory in a list. Regions are case-insensitive, separated by a comma, and wildcards can be used to specify them. Any region-name may include the wildcard characters * and % (remember to escape them to protect them from inappropriate expansion by the shell). Any region name expansion occurs in M (ASCII) collation order.
MUPIP FREEZE waits for up to one minute so that concurrent KILL or MUPIP REORG operations can complete. If the KILL or MUPIP REORG commands do not complete within one minute, MUPIP FREEZE unfreezes any regions it had previously marked as frozen and terminates with an error.
To ensure that a copy or reorganized version of a database file contains a consistent set of records, concurrent MUPIP utilities, such as BACKUP (without the ONLINE qualifier) and EXTRACT, include mechanisms to ensure that the database does not change while the MUPIP utility is performing an action. YottaDB recommends the use of the ONLINE qualifier with BACKUP.
A MUPIP FREEZE can be removed only by the user who sets the FREEZE or by using OVERRIDE.
A MUPIP FREEZE ON can specify either NOONLINE, the default, or ONLINE, and if ONLINE, can specify either AUTORELEASE, the default, or NOAUTORELEASE.
A FREEZE specifying ONLINE attempts to minimize the impact of the FREEZE on concurrently updating processes.
A FREEZE specifying ONLINE AUTORELEASE allows updates to continue immediately when YottaDB needs to update the database file. The processes release the freeze if they cannot find global buffers to do their work.
After MUPIP FREEZE ON NOONLINE, processes that are attempting updates "hang" until the FREEZE is removed by the MUPIP FREEZE OFF command or DSE. Make sure that procedures for using MUPIP FREEZE, whether manual or automated, include provisions for removing the FREEZE in all appropriate cases, including when errors disrupt the normal flow.
MUPIP FREEZE sends a DBFREEZEON/DBFREEZEOFF message to the system log for each region whose freeze state is changed.
A RECOVER/ROLLBACK for a database reverts to a prior database update state. Therefore, a RECOVER/ROLLBACK immediately after a MUPIP FREEZE ON removes the freeze. However, RECOVER/ROLLBACK does not succeed if there are processes attached (for example when a process attempts a database update immediately after a MUPIP FREEZE ON) to the database.
FREEZE must include one of the qualifiers:
-OF[F]
-ON
The optional qualifiers are:
-[NO]A[UTORELEASE] - only valid with -ONLINE
-DBG
-ON[LINE] - only valid with -ON
-OV[ERRIDE]
-R[ECORD] - only valid with -ON
-VERBOSE
-OFF
Clears a freeze set by another process with the same userid.
The format of the OFF qualifier is:
OF[F]
A FREEZE OFF which turns off a FREEZE ONLINE AUTORELEASE produces a OFRZNOTHELD warning to indicate that the freeze was automatically released and therefore did not protect whatever concurrent actions it was intended to guard.
When used with OVERRIDE, OFF stops a freeze operation set by a process with a different userid.
Incompatible with: ON, RECORD
-ON
Specifies the start of a MUPIP FREEZE operation. The format of the ON qualifier is:
-ON
Incompatible with: OFF, OVERRIDE
-[NO]A[UTORELEASE]
Controls the behavior of a FREEZE specified with -ONLINE when YottaDB must write to a database file. The format of the AUTORELEASE qualifier is:
-[NO]A[UTORELEASE]
AUTORELEASE, the default, causes YottaDB to release the freeze if it needs to update the file before a FREEZE OFF.
When FREEZE -ONLINE is in effect, MUPIP rejects all SET actions and produces the OFRZACTIVE warning; if you need to perform any such SET actions, you must release the freeze first.
NOAUTORELEASE causes YottaDB to hold off actions that need to update the database file until someone issues a MUPIP FREEZE OFF.
The actions that require YottaDB to write to the database file are:
Insufficient global buffers to hold updates - YottaDB must flush buffers to make space to do any additional updates
Insufficient space in the database to hold updates - YottaDB must extend the file
The journal file reaches its maximum size or someone issues a MUPIP SET JOURNAL command - YottaDB must create a new journal file
An epoch comes due - YottaDB must create a checkpoint
Someone issues a MUPIP BACKUP command - YottaDB must record state information to mark the beginning of the backup
When an AUTORELEASE abandons a FREEZE, any actions that depend on the stability of the database file on secondary storage, such as a database copy, lose that protection and are not reliable, so they likely need to be repeated at a time when an AUTORELEASE is less likely or when NOONLINE is more appropriate.
An AUTORELEASE action produces an OFRZAUTOREL message in the operator log.
An AUTORELEASE action requires a FREEZE OFF to reestablish a normal database state.
Incompatible with: OFF, NOONLINE
-DBG
Produces verbose output to help with debugging. We recommend using VERBOSE instead.
-ONLINE
Controls the potential impact of a FREEZE on concurrently updating processes. The format of the ONLINE qualifier is:
-[NO]ONL[INE]
ON NOONLINE, the default, causes the freeze to last until OFF, and makes management of the FREEZE straightforward.
ON ONLINE, causes YottaDB to attempt to minimize the impact of the FREEZE on concurrently updating processes by taking a number of actions, as appropriate:
Switching journal files to provide maximum space
Performing an epoch to provide maximum time to the next epoch
Flushing the global buffers to make all available to hold updates
Incompatible with: AUTORELEASE, OFF
After performing these preparations, ONLINE allows updating processes to make updates to global buffers but defer flushing them to the database file.
ONLINE cannot apply to MM databases, so a FREEZE ONLINE skips any MM regions it encounters.
Refer to AUTORELEASE above for additional information.
Incompatible with: OFF
Note
If any database region is nearly full, run MUPIP EXTEND before attempting a MUPIP FREEZE ON ONLINE. Otherwise, should a database file extension become necessary after the MUPIP FREEZE takes effect, the freeze will be released (if AUTORELEASE was specified or implicitly assumed by default) or all updates will be blocked (if NOAUTORELEASE was specified) effectively turning the ONLINE freeze into a NOONLINE freeze.
-OVERRIDE
Release a freeze set by a process with a different userid. YottaDB provides OVERRIDE to allow error recovery in case a procedure with a freeze fails to release. The format of the OVERRIDE qualifier is:
-OV[ERRIDE]
OVERRIDE should not be necessary (and may even be dangerous) in most schemes.
Incompatible with: AUTORELEASE, ON, ONLINE, RECORD
-RECORD
Specifies that a MUPIP FREEZE operation should record an event as a reference point. You might use MUPIP FREEZE to set up your database for a custom-backup mechanism (SAN or mirror-based).
The format of the RECORD qualifier is:
-R[ECORD]
You might use RECORD to integrate MUPIP BACKUP BYTESTREAM with an external backup mechanism.
RECORD replaces the previously RECORDed transaction identifier for the database file.
Incompatiable with: OFF and OVERRIDE.
-VERBOSE
Produces verbose output to help with debugging.
Examples for MUPIP FREEZE
Example:
$ mupip freeze -off DEFAULT
This command stops an ongoing MUPIP FREEZE operation on the region DEFAULT.
Example:
$ mupip freeze -on "*"
This command prevents updates to all regions in the current Global Directory.
Example:
$ set +e
$ mupip freeze -on -record "*"
$ tar cvf /dev/tape /prod/appl/*.dat
$ mupip freeze -off
$ set -e
The set +e command instructs the shell to attempt all commands in the sequence , regardless of errors encountered by any command. This ensures that the FREEZE OFF is processed even if the tar command fails. FREEZE prevents updates to all database files identified by the current Global Directory. The RECORD qualifier specifies that the current transaction in each database be stored in the RECORD portion of the database file header. The tar command creates a tape archive file on the device /dev/tape, containing all the files from /prod/app that have an extension of .dat. Presumably all database files in the current Global Directory are stored in that directory, with that extension. The second FREEZE command re-enables updates that were suspended by the first FREEZE. The set -e command re-enables normal error handling by the shell.
Example:
$ mupip freeze -override -off DEFAULT
This command unfreezes the DEFAULT region even if the freeze was set by a process with a different userid.
FTOK
Produces the "public" (system generated) IPC Keys (essentially hash values) of a given file-list or the journal/receiver pool in a table form.
The format of the MUPIP FTOK command is:
FT[OK] [-DB] [-JNLPOOL] [-RECVPOOL] [-ID] [-ONLY] [-[NO]HEADER] file-list
where file-list is a space delimited list of files, such as that provided by the use of the * and ? shell wildcard characters.
With JNLPOOL or RECVPOOL, MUPIP FTOK ignores any files in the list.
With JNLPOOL or RECVPOOL, MUPIP FTOK uses the entire replication instance file path.
-DB
Specifies that the file-name is a database file. By default, MUPIP FTOK uses DB.
-JNLPOOL
Specifies that the reported key is for the Journal Pool of the instance created by the current Global Directory.
-RECVPOOL
Specifies that the reported key is for the Receive Pool of the instance created by the current Global Directory.
-ID
Specified the signature use for the FTOK; if unspecified, it defaults to the signature for a YottaDB database file.
ID qualifier was added to YottaDB effective release r1.36.
-ONLY
Restricts the output to only the FTOK key; if a file is not valid and accessible, FTOK reports -1 values. ONLY does not report the table header even when HEADER is specified. If the database file is unavailable, the utility defaults to the ONLY behavior.
ONLY qualifier was added to YottaDB effective release r1.36.
-HEADER
Displays or omits the header of the table. By default, MUPIP FTOK displays the header.
Example:
$ mupip ftok -id mumps.dat
File :: Semaphore Id :: Shared Memory Id :: FTOK Key :: FileId
-------------------------------------------------------------------------------------------------------------------------------------------
mumps.dat :: 1044382209 [0x3e400201] :: 613417274 [0x2490013a] :: 725039842 [0x2b373ae2] :: 0x00000000000e09818000002900000001
$ mupip ftok -id -only mumps.dat
mumps.dat :: 725039842 [ 0x2b373ae2 ]
$ mupip ftok -id -noheader mumps.dat
mumps.dat :: 1044382209 [0x3e400201] :: 613417274 [0x2490013a] :: 725039842 [0x2b373ae2] :: 0x00000000000e09818000002900000001
$ mupip ftok -jnlpool
File :: Semaphore Id :: Shared Memory Id :: FTOK Key :: FileId
-------------------------------------------------------------------------------------------------------------------------------------------
multi.repl :: -1 [0xffffffff] :: -1 [0xffffffff] :: 743218009 [0x2c4c9b59] :: 0x00000000000e09838000002900000001
jnlpool :: 636486196 [0x25f00234] :: 578814256 [0x22800130]
HEADER qualifier was added to YottaDB effective release r1.36.
HASH
Uses a 128 bit hash based on the MurmurHash3 algorithm to provide the hash of source files from the command line.
The format of the MUPIP HASH command is:
MUPIP HASH <file-names>
INTEG
Performs an integrity check on a YottaDB database file. You can perform structural integrity checks on one or more regions in the current Global Directory without bringing down (suspending database updates) your application. However, a MUPIP INTEG on a single file database requires standalone access but does not need a Global Directory. The order in which the MUPIP INTEG command selects database regions is a function of the file system layout and may vary as files are moved or created. Execute MUPIP INTEG operations one database file at a time to generate an report where the output always lists database files in a predictable sequence. For example, to compare output with a reference file, run INTEG on one file at a time.
Always use MUPIP INTEG in the following conditions:
Periodically - to ensure ongoing integrity of the database(s); regular INTEGs help detect any integrity problems before they spread and extensively damage the database file.
After a crash - to ensure the database was not corrupted. (Note: When using before-image journaling, when the database is recovered from the journal file after a crash, an integ is not required).
When database errors are reported - to troubleshoot the problem.
Improving the logical and physical adjacency of global nodes may result in faster disk I/O. A global node is logically adjacent when it is stored within a span of contiguous serial block numbers. A global node is physically adjacent when it resides on adjacent hard disk sectors in a way that a single seek operation can access it. Database updates (SETs/KILLs) over time affect the logical adjacency of global nodes. A MUPIP INTEG reports the logical adjacency of your global nodes which may indicate whether a MUPIP REORG could improve the database performance. A native file system defragmentation improves physical adjacency.
Note
Most modern SAN and I/O devices often mask the performance impact of the adjustments in logical and physical adjacency. If achieving a particular performance benchmark is your goal, increasing the logical and physical adjacency should be only one of many steps that you might undertake. While designing the database, try to ensure that the logical adjacency is close to the number of blocks that can physically reside on your hard disk's cylinder. You can also choose two or three cylinders, with the assumption that short seeks are fast.
The format of the MUPIP INTEG command is:
I[NTEG]
[
-A[DJACENCY]=integer
-BL[OCK]=hexa;block-number
-BR[IEF]
-DBG
-FA[ST]
-FU[LL]
-[NO]K[EYRANGES]
-[NO]MAP[=integer]
-[NO]MAXK[EYSIZE][=integer]
-[NO]O[NLINE]
-S[UBSCRIPT]=subscript]
-TN[_RESET]
-[NO]TR[ANSACTION][=integer]
-VERBOSE
]
{[-FILE] file-name|-REG[ION] region-list}
MUPIP INTEG requires specification of either file(s) or region(s).
Press <CTRL-C> to stop MUPIP INTEG before the process completes.
The file-name identifies the database file for a MUPIP INTEG operation. The region-list identifies one or more regions that, in turn, identify database files through the current Global Directory.
MUPIP INTEG operation keeps track of the number of blocks that do not have the current block version during a non-fast integ (default or full) and matches this value against the blocks to upgrade counters in the file-header. It issues an error if the values are unmatched and corrects the count in the file header if there are no other integrity errors.
Note
Promptly analyze and fix all errors that MUPIP INTEG reports. Some errors may be benign while others may be signs of corruption or compromised database integrity. If operations continue without fixes to serious errors, the following problems may occur: Invalid application operation due to missing or incorrect data, Process errors (including inappropriate indefinite looping when a database access encounters an error), and degrading application level consistency as a result of incomplete update sequences caused by pre-existing database integrity issues.
YottaDB strongly recommends fixing the following errors as soon as they are discovered:
Blocks incorrectly marked free - these may cause accelerating damage when processes make updates to any part of the database region.
Integrity errors in an index block - these may cause accelerating damage when processes make updates to that area of the database region using the faulty index. For more information, refer to Chapter 11: “Maintaining Database Integrity”.
MUPIP INTEG FAST and the "regular" INTEG both report these errors (These qualifiers are described later in this section). Other database errors do not pose the threat of rapidly spreading problems in GDS files. After the YottaDB database repair, assess the type of damage, the risk of continued operations, and the disruption in normal operation caused by the time spent repairing the database. For information on analyzing and correcting database errors, refer to Chapter 11: “Maintaining Database Integrity”. Contact your YottaDB support channel for help assessing INTEG errors.
INTEG Qualifiers
The following sections describe the qualifiers of the INTEG command.
-ADJACENCY
Specifies the logical adjacency of data blocks that MUPIP INTEG should assume while diagnosing the database. By default, MUPIP INTEG operates with ADJACENCY=10 and reports the logical adjacency in the "Adjacent" column of the MUPIP INTEG report.
The complexity of contemporary disk controllers and the native file system may render this report superfluous. But when it is meaningful, this report measures the logical adjacency of data.
A MUPIP REORG improves logical adjacency and a native file system defragmentation improves physical adjacency.
The format of the ADJACENCY qualifier is:
-AD[JACENCY]=integer
-BLOCK
Specifies the block for MUPIP INTEG command to start checking a sub-tree of the database. MUPIP INTEG BLOCK cannot detect "incorrectly marked busy errors".
The format of the BLOCK qualifier is:
-BL[OCK]=block-number
Block numbers are displayed in an INTEG error report or by using DSE.
Incompatible with: SUBSCRIPT and TN_RESET
-BRIEF
Displays a single summary report by database file of the total number of directory, index and data blocks. The format of the BRIEF qualifier is:
-BR[IEF]
By default, MUPIP INTEG uses the BRIEF qualifier.
Incompatible with: FULL
-DBG
Produces verbose output to help with debugging. We recommend using VERBOSE instead.
-FAST
Checks only index blocks. FAST does not check data blocks.
The format of the FAST qualifier is:
-FA[ST]
FAST produces results significantly faster than a full INTEG because the majority of blocks in a typical database are data blocks.
While INTEG FAST is not a replacement for a full INTEG, it very quickly detects those errors that must be repaired immediately to prevent accelerating database damage.
By default, INTEG checks all active index and data blocks in the database.
FAST reports include adjacency information.
Incompatible with: TN_RESET.
-FILE
Specifies the name of the database file for the MUPIP INTEG operation. FILE requires exclusive (stand-alone) access to a database file and does not require a Global Directory. The format of the FILE qualifier is:
-FI[LE]
With stand-alone access to the file, MUPIP INTEG FILE is able to check whether the reference count is zero. A non-zero reference count indicates prior abnormal termination of the database.
The FILE qualifier is incompatible with the REGION qualifier.
By default, INTEG operates on FILE.
-FULL
Displays an expanded report for a MUPIP INTEG operation. With FULL specified, MUPIP INTEG displays the number of index and data blocks in the directory tree and in each global variable tree as well as the total number of directory, index and data blocks. The format of the FULL qualifier is:
-FU[LL]
The FULL qualifier is incompatible with the BRIEF qualifier.
By default, INTEG reports are BRIEF.
Use FULL to have INTEG report all global names in a region or list of regions.
-KEYRANGES
Specify whether the MUPIP INTEG report includes key ranges that it detects which identify the data suspected of problems. The format of the KEYRANGES qualifier is:
-[NO]K[EYRANGES]
By default, INTEG displays KEYRANGES.
-MAP
Specifies the maximum number of "incorrectly marked busy errors" that MUPIP INTEG reports. The format of the MAP qualifier is:
-[NO]MAP[=max_imb_errors]
<max_imb_errors> specifies the threshold limit for the number of incorrectly marked busy errors.
NOMAP automatically sets a high threshold limit of 1000000 (1 million) incorrectly marked busy errors (MAP=1000000).
By default, INTEG reports a maximum of 10 map errors (MAP=10).
Note
MUPIP INTEG reports all "incorrectly marked free" errors as they require prompt action. MAP does not restrict their reports.
An error in an index block prevents INTEG from processing potentially large areas of the database. A single "primary" error may cause large numbers of "secondary" incorrectly marked busy errors, which are actually useful in identifying valid blocks that have no valid index pointer. Because "real" or primary incorrectly marked busy errors only make "empty" blocks unavailable to the system, they are low impact and do not require immediate repair.
Note
After a database recovery with RECOVER (for example, using BEFORE_TIME) or ROLLBACK (for example, using FETCHRESYNC), the database may contain incorrectly marked busy errors. Although these errors are benign, they consume available space. Schedule repairs on the next opportunity.
-MAXKEYSIZE
Specifies the maximum number of "key size too large" errors that a MUPIP INTEG operation reports. The format of the MAXKEYSIZE qualifier is:
-[NO]MAX[KEYSIZE][=integer]
By default, INTEG reports a maximum of 10 key size errors (MAXKEYSIZE=10).
NOMAXKEYSIZE removes limits on key size reporting so that INTEG reports all "key size too large" errors.
NOMAXKEYSIZE does not accept assignment of an argument.
"Key size too large" errors normally only occur if a DSE CHANGE FILEHEADER KEY_MAX_SIZE command reduces the maximum key size.
-ONLINE
Specifies that while a MUPIP INTEG operation is active, other processes can update the database without affecting the result of the operation. Allows checking database structural integrity to run concurrently with database updates. The format of the ONLINE qualifier is:
-[NO]O[NLINE]
NOONLINE specifies that the database should be frozen during MUPIP INTEG.
By default, MUPIP INTEG is online.
Since MUPIP INTEG ONLINE does not freeze database updates, it cannot safely correct errors in the "blocks to upgrade" and "free blocks" fields in the file header, while MUPIP INTEG NOONLINE can correct these fields.
As it checks each database file, MUPIP INTEG ONLINE creates a sparse file of the same size as the database. As each YottaDB process updates the database, it places a copy of the old block in the sparse file before updating the database. For any database blocks with a newer transaction number than the start of the INTEG, MUPIP uses the copy in the sparse file. Thus, analogous with MUPIP BACKUP ONLINE, INTEG reports on the state of the database as of when it starts, not when it completes. Note: a command such as ls -l shows sparse files at their full size, but does not show actual disk usage. Use a command such as
du -shto see actual disk usage.The environment variable ydb_snaptmpdir can be used to indicate a directory where MUPIP should place the snapshot files (used by MUPIP INTEG ONLINE). If ydb_snaptmpdir does not exist, INTEG uses the location specified by ydb_baktmpdir and if neither of those environment variables is defined, INTEG places the snapshot files in the current directory at the time you issue the INTEG command. MUPIP and YottaDB processes automatically clean up these temporary snapshot files under a wide variety of conditions.
Temporary directory security settings must allow write access by the MUPIP process and by all processes updating the database. MUPIP creates the temporary file with the same access as the database file so processes updating the database can write to the temporary file. If the database is encrypted, the updating processes write encrypted blocks to the snapshot file and the MUPIP INTEG process must start with access to appropriate key information as it does even NOONLINE.
MUPIP INTEG NOONLINE [FAST] {REGION|FILE} clears the KILLs in progress and the Abandoned Kills flags if the run includes the entire database and there are no incorrectly marked busy blocks.
Only one online integ can be active per database region. If an online integ is already active, a subsequent one issues an error and immediately terminates. If an online integ does not successfully complete, YottaDB cleans it up in one of the following ways:
A subsequent online integ detects that an earlier one did not successfully complete and releases the resources held by the prior online integ before proceeding.
If a MUPIP STOP was issued to the online integ process, the process cleans up any resources it held. Note: since the process was stopped the results of the integ may not be valid.
subsequent MUPIP RUNDOWN ensures the release of resources held by prior unsuccessful online integs for the specified regions.
For every 64K transactions after the online integ initiation, online integ checks YottaDB's health for improperly abandoned online integs and releases resources held by any it finds.
Incompatible with: FILE, TN_RESET (there should be no need to use TN_RESET on a YottaDB database).
-REGION
Specifies that the INTEG parameter identifies one or more regions rather than a database file. The format of the REGION qualifier is:
-R[EGION]=region-list | -R[EGION] region-list
The region-list identifies the target of INTEG. region-list may specify more than one region of the current global directory in a list. Region names in the list are case-insensitive, separated by commas, and wildcards can be used to specify them. Any region-name may include the wildcard characters * and ? (remember to escape them to protect them from inappropriate expansion by the shell). Any region name expansion occurs in M (ASCII) collation order. INTEG processes regions based on the order specified in the list after wild-card expansion.
INTEG REGION requires the environment variable ydb_gbldir to specify a valid Global Directory. For more information on defining ydb_gbldir, refer to Chapter 4: “Global Directory Editor”.
Because a KILL may briefly defer marking the blocks it releases "free" in the bit maps, INTEG REGION may report spurious "block incorrectly marked busy" errors. These errors are benign. If these errors occur in conjunction with a "Kill in progress" error, resolve the errors after the "Kill in progress" error is no longer present.
By default, INTEG operates FILE.
Incompatible with: FILE, TN_RESET
-SUBSCRIPT
Specifies a key (an unsubscripted or subscripted global name) or a range of keys to INTEG.
An unsubscripted global name key may be enclosed in single or double quotes (for example, ^a or '^a' or "^a" are all valid). A subscripted global name with numeric subscripts must be enclosed in single or double quotes to avoid the shell from parsing the parentheses (for example, '^a(1)' or "^a(1)" are both valid). A subscripted global name with string subscripts must be enclosed in a mix of single and double quotes and use 2 double quotes for every double quote in the string subscript (for example, '"^a(1,""string"")"' is valid). See example section below for more details.
Identify a range by separating the beginning key and ending key with a colon (:).
SUBSCRIPT cannot detect incorrectly marked busy errors. The format of the SUBSCRIPT qualifier is:
-SU[BSCRIPT]=subscript
Specify SUBSCRIPT only if the path to the keys in the subscript is not damaged. If the path is questionable or known to be damaged, use DSE to find the block(s) and INTEG BLOCK.
Incompatible with: BLOCK, TN_RESET
-STATS
Specifies INTEG to check any active statistics database associated with the region(s) specified for the command. The format of the STATS qualifier is:
-[NO]ST[ATS]
Specify STATS only if you have reason to understand that statistics reporting is failing with database errors or reporting incorrect results. Because FILE requires standalone access and statistic databases are automatically created and removed it is incompatible with STATS. The default is NOSTATS.
Incompatible with: BLOCK, FILE, TN_RESET
-TN_RESET
Resets block transaction numbers and backup event recorded transaction numbers to one (1), and the current transaction number to two (2) which makes the backup event recorded transaction numbers more meaningful and useful. It also issues an advisory message to perform a backup.
The format of the TN_RESET qualifier is:
-TN[_RESET]
Transaction numbers can go up to 18,446,744,073,709,551,615. This means that a transaction processing application that runs flat out at a non-stop rate of 1,000,000 updates per second would need a TN reset approximately every 584,554 years.
The TN_RESET qualifier rewrites all blocks holding data. If the transaction overflow resets to zero (0) database operation is disrupted.
The TN_RESET qualifier is a protective mechanism that prevents the transaction overflow from resetting to 0.
By default, INTEG does not modify the block transaction numbers.
Note
There should never be a need for a TN_RESET on the database, even when cleaning up after a runaway process.
The TN_RESET qualifier is incompatible with the FAST, BLOCK, REGION, and SUBSCRIPT qualifiers.
Note
Any time a YottaDB update opens a database file that was not properly closed, YottaDB increments the transaction number by 1000. This automatic increment prevents problems induced by abnormal database closes, but users must always consider this factor in their operational procedures. The rate at which YottaDB "uses up" transaction numbers is a function of operational procedures and real database updates.
-TRANSACTION
Specifies the maximum number of block transaction-number-too-large errors that MUPIP INTEG reports. The format of the TRANSACTION qualifier is:
-[NO]TR[ANSACTION][=integer]
NOTRANSACTION removes limits on transaction reporting so MUPIP INTEG reports all transaction number errors.
NOTRANSACTION does not accept assignment of an argument.
A system crash may generate many "block transaction number too large" errors. These errors can cause problems for BACKUP INCREMENTAL and for transaction processing. Normally, the automatic increment of 1000 blocks that YottaDB adds when a database is reopened averts these errors. If a problem still exists after the database is reopened, users can use a value in the DSE CHANGE FILEHEADER CURRENT_TN= command to quickly fix "block transaction number too large" errors.
By default, INTEG reports a maximum of 10 block transaction errors (TRANSACTION=10).
-VERBOSE
Produces verbose output to help with debugging.
Examples for MUPIP INTEG
Example:
$ mupip integ -block=4 yottadb.dat
This command performs a MUPIP INTEG operation on BLOCK 4 of yottadb.dat.
Example:
$ mupip integ -adjacency=20
A sample output from the above command follows:
Type Blocks Records % Used Adjacent
Directory 2 110 25.732 NA
Index 1170 341639 88.298 6
Data 340578 519489 99.268 337888
Free 6809 NA NA NA
Total 348559 861238 NA 337894
[Spanning Nodes:3329 ; Blocks:341403]
This example performs a MUPIP INTEG operation assuming that logically related data occupies 20 data blocks in the current database. The sample output shows that out of 1137 data blocks, 1030 data blocks are adjacent to each other. One may be able to improve the performance of a database if all blocks are as adjacent as possible. "% Used" is the amount of space occupied across the in-use blocks divided by the space available in the in-use blocks, and thus represents the packing density for the in-use blocks (excluding local bit maps). Higher "% Used" may actually be undesirable from a performance perspective as they indicate a higher likelihood of block splits with upcoming updates.
Example:
$ mupip integ -brief yottadb.dat
This command performs a MUPIP INTEG operation on the database yottadb.dat. A sample output from the above command follows:
No errors detected by integ.
Type Blocks Records % Used Adjacent
Directory 2 110 25.732 NA
Index 1170 341639 88.298 4
Data 340578 519489 99.268 337617
Free 6809 NA NA NA
Total 348559 861238 NA 337621
[Spanning Nodes:3329 ; Blocks:341403]
Example:
$ mupip integ -fast yottadb.dat
This command performs a MUPIP INTEG operation only on the index block of the database file yottadb.dat. A sample output from the above command follows:
No errors detected by fast integ.
Type Blocks Records % Used Adjacent
Directory 2 110 25.732 NA
Index 1170 341639 88.298 4
Data 340578 NA NA 337617
Free 6809 NA NA NA
Total 348559 NA NA 337621
Note the NA entries for Data type. It means that the MUPIP INTEG FAST operation checked only index blocks.
$ mupip integ -full yottadb.dat
The sample output from the above command follows:
Directory tree
Level Blocks Records % Used Adjacent
1 1 1 0.585 NA
0 1 109 50.878 NA
Global variable ^#t
Level Blocks Records % Used Adjacent
1 1 1 0.585 0
0 1 80 49.609 1
Global variable ^versionContent
Level Blocks Records % Used Adjacent
1 1 1 0.585 0
0 1 1 94.018 0
Global variable ^x
Level Blocks Records % Used Adjacent
1 1 2 1.464 0
0 2 109 52.551 1
Example:
$ mupip integ -map=20 -maxkeysize=20 -transaction=2 yottadb.dat
This command performs a MUPIP INTEG operation and restricts the maximum number of "key size too large" errors to 20.
Example:
$ mupip integ -map=20 -transaction=2 yottadb.dat
This command performs a MUPIP INTEG operation and restricts the maximum number of "block transaction number too large" errors to 2.
$ mupip integ -file yottadb.dat -tn_reset
This command resets the transaction number to one in every database block.
Example:
$ mupip integ -subscript="^Parrots" yottadb.dat
This example performs a MUPIP INTEG operation on the global variable ^Parrots in the database file yottadb.dat.
Example:
$ mupip integ -subscript='"^Amsterdam(100)"':'"^Bolivia(""Chimes"")"' -region DEFAULT
This example performs a MUPIP INTEG operation on all global variable nodes/keys greater than or equal to ^Amsterdam(100) and less than or equal to ^Bolivia("Chimes") in the DEFAULT region.
Note
To specify a literal in the command string, use two double quotation marks for example, ^b(""c"").
INTRPT
Sends an interrupt signal [POSIX] SIGUSR1 to the specified process, whose signal handler determines how it is handled. M processes execute the $ZINTERRUPT intrinsic special variable. The format of the MUPIP INTRPT command is:
INTRPT process-id
Note
Ensure that signal SIGUSR1 is not be used by any C external function calls or any (initially non-YottaDB) processes that use call-in support, as it is interpreted by YottaDB as a signal to trigger the $ZINTERRUPT mechanism.
To INTRPT a process belonging to its own account, a process requires no UNIX privileges.
The implementation of INTRPT uses signals, and on Linux a non-root process can only send signals to other processes of the same userid. Superuser privilege is required to send signals to processes of other userids, regardless of group membership.
JOURNAL
Analyzes, extracts, reports, and recovers data using journal files. For a description of the JOURNAL command, refer to Chapter 6: “YottaDB Journaling”.
LOAD
Puts the global variable names and their corresponding data values into a YottaDB database from a sequential file.
The format of the LOAD command is:
L[OAD]
[-BE[GIN]=integer -E[ND]=integer
-FI[LLFACTOR]=integer
-FO[RMAT]={GO|B[INARY]|Z[WR]]}
-I[GNORECHSET]
-O[NERROR]={STOP|PROCEED|INTERACTIVE}
-S[TDIN]] file-name
Note
From an application perspective, performing a MUPIP LOAD operation while an application is running may result in an inconsistent application state for the database.
MUPIP LOAD uses the Global Directory to determine which database files to use.
LOAD supports user collation routines.
LOAD takes its input from the file defined by the file-name, which may be a UNIX file on any device that supports such files.
LOAD accepts files with DOS style termination.
MUPIP LOAD command considers a sequential file as encoded in UTF-8 if the environment variable ydb_chset is set to UTF-8. Ensure that MUPIP EXTRACT commands and the corresponding MUPIP LOAD commands execute with the same setting for the environment variable ydb_chset.
For information on loading with an M "percent utility," refer to the %GI section of the "M Utility Routines" chapter in the Programmer's Guide. LOAD is typically faster, but the %GI utility can be customized.
Press <CTRL-C> to produce a status message from LOAD. Entering <CTRL-C> twice in quick succession stops LOAD. A LOAD that is manually stopped or stops because of an internal error is incomplete and may lack application level integrity, but will not adversely affect the database structure unless terminated with a
kill -9.
Note
The MUPIP EXTRACT or MUPIP LOAD procedure for large databases is time consuming due to the volume of data that has to be converted from binary to ZWR format (on source) and vice versa (on target). One must also consider the fact that the extracted file can be very large for a large database. Users must ensure there is adequate storage space to support the size of the extract file and the space occupied by the source and target databases. In order to reduce the total time and space it takes to transfer database content from one endian platform to another, it is efficient to convert the endian format in-place for a database and transfer the converted database. See MUPIP ENDIANCVT for more information on converting the endian format of a database file.
The following sections describe the optional qualifiers of the MUPIP LOAD command.
-FORMAT
Specifies the format of the input file. If the format of the input file is not specified, MUPIP LOAD automatically detects the file format (BINARY/ZWR/GO) based on the file header or (when the header is absent) the key and value information in the file. If the format is specified, it must match the actual format of the input file for LOAD to proceed.
The format codes are:
B[INARY] - Binary format
GO - Global Output format
Z[WR] - ZWRITE format
MUPIP LOAD detects the file format (BINARY/ZWR/GO) based on the file header of the extracted files from MUPIP EXTRACT, ^%GO and DSE.
-format=binaryonly applies to GDS files. A BINARY format file loads significantly faster than a GO or ZWR format file. FORMAT=BINARY works with data in a proprietary format. FORMAT=BINARY has one header record, therefore LOAD FORMAT=BINARY starts active work with record number two (2).FORMAT={ZWR|GO} applies to text files produced by tools such as MUPIP EXTRACT or %GO.
For FORMAT={ZWR|GO} UTF-8 files not produced by MUPIP EXTRACT or %GO, the first line of the label must contain the case insensitive text "UTF-8".
For all FORMAT={ZWR|GO} files not produced by MUPIP EXTRACT or %GO, the second line should contain the case insensitive text "ZWR" for zwr format or "GLO" for GO format and the two label lines must contain a total of more than 10 characters.
FORMAT=GO expects the data in record pairs. Each global node requires one record for the key and one for the data.
FORMAT=ZWR expects the data for each global node in a single record.
-BEGIN
Specifies the record number of the input file with which LOAD should begin. Directing LOAD to begin at a point other than the beginning of a valid key causes an error. The format of the BEGIN qualifier is:
-BE[GIN]=integer
Note
Always consider the number of header records for choosing a BEGIN point. See FORMAT qualifier for more information.
For FORMAT=GO input, the value is usually an odd number. As FORMAT=BINARY requires important information from the header, this type of load requires an intact file header regardless of the BEGIN value.
For FORMAT=ZWR input, each record contains a complete set of reference and data information. The beginning values are not restricted, except to allow two records for the header.
By default, LOAD starts at the beginning of the input file.
-END
Specifies the record number of the input file at which LOAD should stop. END integer value must be greater than the BEGIN integer value for LOAD to operate. LOAD terminates after processing the record of the number specified by END or reaching the end of the input file. The format of the END qualifier is:
-E[ND]=integer
The value of FORMAT=GO input should normally be an even number. By default, LOAD continues to the end of the input file.
-FILLFACTOR
Specifies the quantity of data stored in a database block. Subsequent run-time updates to the block fill the remaining available space reserved by the FILL_FACTOR. Blocks that avoid block splits operate more efficiently. The format of the FILL_FACTOR qualifier is:
-FI[LL_FACTOR]=integer
Reserves room and avoids unnecessary block splits to accommodate the forecasted growth in a global variable that may experience significant rate of additions over a period of time.
Users having database performance issues or a high rate of database updates must examine the defined FILL_FACTORs. Unless the application only uses uniform records, which is not typical for most applications, FILL_FACTORs do not work precisely.
By default, LOAD uses FILL_FACTOR=100 for maximum data density.
Note
FILL_FACTOR is useful when updates add or grow records reasonably uniformly across a broad key range. If updates are at ever-ascending or ever-descending keys, or if the record set and record sizes are relatively static in the face of updates, FILL_FACTOR does not provide much benefit.
-IGNORECHSET
Notifies MUPIP LOAD to load the extract file even if it was created by a MUPIP process in another mode (UTF-8 mode vs. M mode). The format of the IGNORECHSET qualifier is:
-I[GNORECHSET]
Note
As using IGNORECHSET bypasses YottaDB checks, use it only if you are sure that the extract file can be loaded correctly.
IGNORECHSET was added to YottaDB effective release r1.30.
-ONERROR
Determines the MUPIP LOAD behavior when it encounters an error. The format of the ONERROR qualifier is:
-O[NERROR]={STOP|PROCEED|INTERACTIVE}
STOP causes MUPIP LOAD to exit immediately.
PROCEED proceeds to the next record.
INTERACTIVE prompts to continue or stop.
By default MUPIP LOAD exits on encountering an error.
-STDIN
Specifies that MUPIP LOAD takes input from standard input (stdin). The format of the STDIN qualifier is:
-S[TDIN]
Examples for MUPIP LOAD
Example:
$ mupip load ex_file.go
This command loads the content of the extract file ex_file.go to the current database.
Example:
$ mupip load -format=go big.glo
This command loads an extract file big.glo in the current database.
Example:
$ mupip load -begin=5 -end=10 rs.glo
This command begins the MUPIP LOAD operation from record number 5 and ends at record number 10. Note that the value for BEGIN is an odd number. A sample output from the above command follows:
MUPIP EXTRACT
02-MAR-2017 18:25:47 ZWR
Beginning LOAD at record number: 5
LOAD TOTAL Key Cnt: 6
Max Subsc Len: 7
Max Data Len: 1
Last LOAD record number: 10
Example:
$ mupip load -fill_factor=5 reobs.glo
This command sets the FILL_FACTOR to 5 for loading an extract file in the current database.
Example:
$cat big.glo | mupip load -stdin
$mupip load -stdin < big.glo
These commands loads the extract file big.glo using -stdin.
RCTLDUMP
Reports information related to relinkctl files and their associated shared memory segments. The format of the MUPIP RCTLDUMP command is:
MUPIP RCTLDUMP [dir1]
If the optional parameter dir1 is not specified, MUPIP RCTLDUMP dumps information on all its active auto-relink-enabled directories (those with with a *-suffix) identified by $ydb_routines. With a directory path specified for dir1, MUPIP RCTLDUMP reports information on that directory. An example output follows. It lists the full path of the Object directory; its corresponding relinkctl file name; the number of routines currently loaded in this relinkctl file; the number of processes including the reporting MUPIP process that have this relinkctl file open; the shared memory id and length of the relinkctl shared memory segment; one or more rtnobj shared memory segment(s); and a listing of all the routine names loaded in this file (lines starting with rec#…).
The Rtnobj shared memory line : All the length fields are displayed in hexadecimal. shmlen is the length of the allocated shared memory segment in bytes. shmused is the length that is currently used. shmfree is the length available for use. objlen is the total length of all the objects currently loaded in this shared memory. As YottaDB allocates blocks of memory with sizes rounded-up to an integer power of two bytes, shmused is always greater than objlen; for example with an objlen of 0x1c0, the shmused is 0x200.
Lines of the form rec#… indicate the record number in the relinkctl file. Each relinkctl file can store a maximum of 1,000,000 records, i.e., the maximum number of routines in a directory with auto-relink enabled is one million. Each record stores a routine name (rtnname:), the current cycle for this object file record entry (cycle:) which gets bumped on every ZLINK or ZRUPDATE command, the hash of the object file last loaded for this routine name (objhash:), the number of different versions of object files loaded in the Rtnobj shared memory segments with this routine name (numvers:), the total byte-length of the one or more versions of object files currently loaded with this routine name (objlen:), the total length used up in shared memory for these object files where YottaDB allocates each object file a rounded-up perfect 2-power block of memory (shmlen:).
Given a relinkctl file name, one can find the corresponding directory path using the Unix "strings" command on the Relinkctl file. For example, strings /tmp/ydb-relinkctl-f0938d18ab001a7ef09c2bfba946f002, corresponding to the above MUPIP RCTLDUMP output example, would output /obj the corresponding directory name.
Example:
$ mupip rctldump .
Object Directory : /tmp
Relinkctl filename : /tmp/yottadb/r1.20_x86_64/ydb-relinkctl-61f9eb418212a24a75327f53106c1656
# of routines : 1
# of attached processes : 2
Relinkctl shared memory : shmid: 11534344 shmlen: 0x57c6000
Rtnobj shared memory # 1 : shmid: 11567113 shmlen: 0x100000 shmused: 0x200 shmfree: 0xffe00 objlen: 0x1c0
rec#1: rtnname: abcd cycle: 1 objhash: 0xedbfac8c7f7ca357 numvers: 1 objlen: 0x1c0 shmlen: 0x200
REORG
Improves database performance by defragmenting and reorganizing database files and attempts to reduce the size of the database file. MUPIP REORG runs concurrently with other database activity, including updates. Competing activity generally increases the time required to perform a REORG, as well as that of the competing operations.
MUPIP REORG can also encrypt a database and/or change the encryption keys for database files "on the fly" while the database is in use.
The format of the MUPIP REORG command is:
REO[RG]
[
-D[OWNGRADE]
-ENCR[YPT]=key
-E[XCLUDE]=global-name-list
-FI[LL_FACTOR]=integer
-I[NDEX_FILL_FACTOR]=integer
-NOCO[ALESCE]
-NOSP[LIT]
-NOSW[AP]
-R[ESUME]
-S[ELECT]=global-name-list
-T[RUNCATE][=percentage]
-UP[GRADE]
-REG[ION] region-list
]
Note
While REORG optimizes the GDS structure of database files, it does not handle native file system file fragmentation. In most cases, fragmentation at the native file system level is more likely than fragmentation at the GDS structure level. Therefore, YottaDB recommends users create files with appropriate allocations and extensions, on disks with large amounts of contiguous free space. Use native utilities and MUPIP utilities (depending on operational procedures) to eliminate file fragmentation when database files have been extended more than a dozen times.
As REORG is IO intensive, running a REORG concurrently with normal database access may impact the operation of normal processes. As the YottaDB database engine has a daemonless architecture, attempts to reduce the impact by reducing the priority of REORG can (perhaps counter-intuitively) exacerbate rather than alleviate the impact. To reduce the impact REORG has on other processes, use the ydb_poollimit environment variable to limit the number of global buffers used by the REORG.
MUPIP REORG does not change the logical contents of the database, and can run on either the originating instance or replicating instance of an LMS application. In such cases, resuming REORGs in process should be part of the batch restart. See the "Database Replication" chapter for more information about running REORG on a dual site application.
Use MUPIP STOP (or <Ctrl-C> for an interactive REORG) to terminate a REORG process. Unless terminated with a
kill -9, a REORG terminated by operator action or error is incomplete but does not adversely affect the database.
Note
REORG focuses on optimum adjacency and a change to even a single block can cause it to perform a large number of updates with only marginal benefit. Therefore, YottaDB recommends not running successive REORGs close together in time, as much as can provide minimal benefit for a significant increase in database and journal activity. For the same reason, YottaDB recommends careful research and planning before using the RESUME qualifier or complex uses of EXCLUDE and SELECT.
Assume two scenarios of putting values of ^x(1) to ^x(10000). In the first scenarios, fill values in a sequential manner. In the second scenario, enter values for odd subscripts and then enter values for the even subscripts.
Scenario 1:
At the prompt, execute the following command sequence:
YDB>for i=1:1:10000 set ^x(i)=$justify(i,200)
Then, execute the following MUPIP INTEG command.
$ mupip integ -region "*"
This command produces an output like the following:
Integ of region DEFAULT
No errors detected by integ.
Type Blocks Records % Used Adjacent
Directory 2 2 2.490 NA
Index 29 2528 95.999 1
Data 2500 10000 82.811 2499
Free 69 NA NA NA
Total 2600 12530 NA 2500
Note the high density (percent used) for index and data blocks from the report.
Scenario 2:
At the YottaDB prompt, execute the following command sequence:
YDB>for i=1:2:10000 s ^x(i)=$justify(i,200)
YDB>for i=2:2:10000 set ^x(i)=$justify(i,200)
Then, execute the following command:
$ mupip integ -region "*"
This command produces an output like the following:
Integ of region DEFAULT
No errors detected by integ.
Type Blocks Records % Used Adjacent
Directory 2 2 2.490 NA
Index 153 3902 29.211 57
Data 3750 10000 55.856 1250
Free 95 NA NA NA
Total 4000 13904 NA 1307
Note that there are more and less dense index and data blocks used than in scenario 1. MUPIP REORG addresses such issues and makes the database (depending on the FILL_FACTOR) more compact.
The optional qualifiers for MUPIP REORG are:
-DOWNGRADE
MUPIP REORG DOWNGRADE is deprecated, and not supported as of r2.00.
-ENCRYPT
Encrypts an unencrypted database or changes the encryption key of a database while the database continues to be used by applications. Whether or not the prior encryption uses non-zero initialization vectors (IVs), database blocks encrypted with the new key use non-zero IVs. The format of the ENCRYPT qualifier is:
-ENCR[YPT]=<key>
MUPIP provides <key> to the encryption plugin. The reference implementation of the plugin expects a key with the specified name in the encryption configuration file identified by $ydb_crypt_config. The configuration file must contain an entry in the database section for each database file mapping to a region specified in <region-list> that names the specified key as its key. The ENCRYPT flag is incompatible with all other command line flags of MUPIP REORG except REGION, and performs no operation other than changing the encryption key. If the specified key is already the encryption key of a database region, MUPIP REORG ENCRYPT moves on to the next region after displaying a message (on stderr, where MUPIP operations send their output).
As MUPIP REORG ENCRYPT reads, re-encrypts, and writes every in-use block in each database file, its operations take a material amount of time on the databases of typical applications, and furthermore add an additional IO load to the system on which it runs. You can use the environment variable ydb_poollimit to ameliorate, but not eliminate the impact, at the cost of extending execution times. To minimize impact on production instances, YottaDB recommends running this operation on replicating secondary instances, rather than on originating primary instances.
ENCRYPT switches the journal file for each database region when it begins operating on it, and again when it completes, and also records messages in the syslog for both events.
As is the case under normal operation when MUPIP REORG ENCRYPT is not active, journaled databases are protected against system crashes when MUPIP REORG ENCRYPT is in operation: MUPIP JOURNAL ROLLBACK/RECOVER recovers journaled database regions (databases that use NOBEFORE journaling continue to require RECOVER/ROLLBACK FORWARD).
Because a database file utilizes two keys while MUPIP REORG ENCRYPT is underway, the database section of the configuration file provides for a single database file entry to specify multiple keys. For example, if the keys of database regions CUST and HIST, mapping to database files cust.dat and hist.dat in directory /var/myApp/prod, are to be changed from key1 to key2 using the command:
MUPIP REORG -ENCRYPT=key2 -REGION CUST,HIST
then the database section of the configuration file must at least have the following entries:
database: {
keys: ({
dat: "/var/myApp/cust.dat";
key: "key1";
},{
dat: "/var/myApp/cust.dat";
key: "key2";
},{
dat: "/var/myApp/hist.dat";
key: "key1";
},{
dat: "/var/myApp/hist.dat";
key: "key2";
})
};
In other words, each database file entry can have multiple keys, and a key can be associated with multiple database files. With a configuration file that has multiple keys associated with the same database file, MUPIP CREATE uses the last entry. Other database operations use whichever key associated with the database file has a hash matching one in the database file header, reporting an error if no key matches. To improve efficiency when opening databases, you can delete entries for keys that are no longer used from the configuration file.
MUPIP REORG ENCR[YPT] can encrypt an unencrypted database only if the following command:
MUPIP SET -ENCRYPTABLE -REGION <region-list>
has previously marked the database "encryptable".
The command requires standalone access to the database. It performs some basic encryption setup checks and requires the ydb_passwd environment variable to be defined and the GNUPGHOME environment variable to point to a valid directory in the environment. Just as encrypted databases use global buffers in pairs (for encrypted and unencrypted versions of blocks), a database marked as encryptable has global buffers allocated in pairs (i.e., the actual number of global buffers is twice the number reported by DSE DUMP FILEHEADER) and requires correspondingly larger shared memory segments. To revert unencrypted but encryptable databases back to the "unencryptable" state, use the command:
MUPIP SET -NOENCRYPTABLE -REGION <region-list>
The above command also requires standalone access, and the result depends on the state of the database. It:
is a no-op if the database is encrypted;
is disallowed if the database is partially (re)encrypted; and
prohibits encryption if the database is not encrypted.
Under normal operation, a database file has only one key. Upon starting a MUPIP REORG ENCRYPT to change the key, there are two keys, both of whose hashes YottaDB stores in the database file header. With a MUPIP REORG ENCRYPT operation underway to change the key, normal database operations can continue, except for another MUPIP REORG ENCRYPT or MUPIP EXTRACT in binary format. Other MUPIP operations, such as MUPIP BACKUP and MUPIP EXTRACT in ZWR format can occur. A MUPIP REORG ENCRYPT operation can resume after an interruption, either unintentional, such as after a system crash and recovery, or intentional, i.e., an explicit MUPIP STOP of the MUPIP REORG ENCRYPT process. To resume the REORG operation, reissue the original command, including the key parameter. (Note that supplying a key other than the one used in the original command produces an error.)
After the MUPIP REORG ENCRYPT process completes, subsequent MUPIP REORG ENCRYPT operations on the same region(s) are disallowed until the following command is run:
MUPIP SET -ENCRYPTIONCOMPLETE -REGION <region-list>
Blocking subsequent MUPIP REORG ENCRYPT operations after one completes, provides time for a backup of the entire database before enabling further key changes. MUPIP SET ENCRYPTIONCOMPLETE reports an error for any database region for which MUPIP REORG ENCRYPT has not completed.
Note
YottaDB recommends rotating (changing) the encryption key of the database for better security. The frequency of encryption key rotation depends on your security requirements and policies.
Note
MUPIP REORG ENCRYPT does not enable switching between encryption algorithms. To migrate databases from Blowfish CFB to AES CFB requires that the data be extracted and loaded into newly created database files. To minimize the time your application is unavailable, you can deploy your application in a Logical Multi-Site (LMS) configuration, and migrate using a rolling upgrade technique. Refer to the Chapter 7: “Database Replication” for more complete documentation.
-EXCLUDE
Specifies that REORG not handle blocks that contain information about the globals in the associated list–this means they are neither reorganized nor swapped in the course of reorganizing other globals; EXCLUDE can reduce the efficiency of REORG because it complicates and interferes with the block swapping actions that try to improve adjacency.
The format of the EXCLUDE qualifier is:
-E[XCLUDE]=global-name-list
Assume that a single MUPIP command organizes a subset of the globals in a database or region. If a second MUPIP REORG command selects the remaining globals, it may tend to disrupt the results of the first REORG by de-optimizing the previously organized blocks. This is because there is no information passed from the previous MUPIP REORG command to the next command. The EXCLUDE qualifier allows users to list the name of the previously REORGed globals, so that the MUPIP REORG bypasses the GDS blocks containing these globals.
If global-name-list contains globals that do not exist, REORG issues a message to the terminal and continues to process any specified globals that exist. If REORG is unable to process any globals, it terminates with an error.
Global-name-list can be an individual global name, a range of global names, or a list of names and prefixes followed by the wildcard symbol. For example:
A global name, such as ACN.
A range of global names, such as A7:B7.
A list, such as A,B,C.
Global names with the same prefix such as TMP*.
In the first case, REORG only excludes global ^ACN. In the second case, REORG excludes all global names in the collating sequence A7 to B7. For the third case, REORG excludes A, B, and C. In the last case, REORG excludes all globals prefixed with TMP.
Enclose wildcards in double-quotes ("") to prevent inappropriate expansion by the shell. The caret symbol (^) in the specification of the global is optional.
By default, REORG does not EXCLUDE any globals.
In case any global appears in the argument lists of both SELECT and EXCLUDE, REORG terminates with an error.
-FILL_FACTOR
Specifies how full you want each database block to be. This is a target number. Individual blocks may be more or less full than the fill factor. The format of the FILL_FACTOR qualifier is:
F[ILL_FACTOR]=integer
The arguments for the FILL_FACTOR qualifier must be integers from 30 to 100. These integers represent the percentage of the data block that REORG can fill. By default, the FILL_FACTOR value is 100 for maximum data density.
Users who come upon database performance issues or a high rate of database updates must examine the defined FILL_FACTORs. Unless the application uses entirely uniform records, which is not typical for most applications, FILL_FACTORs do not work precisely.
The FILL_FACTOR for data that is relatively static, or grows by the addition of new nodes that collate before or after pre-existing nodes, should be 100 percent. The FILL_FACTOR for data that is growing by additions to existing nodes may be chosen to leave room in the typical node for the forecast growth for some period. Generally, this is the time between the LOAD and first REORG, or between two REORGs. This is also true for additions of nodes that are internal to the existing collating sequence.
-INDEX_FILL_FACTOR
Directs REORG to leave free space within index blocks for future updates. Arguments to this qualifier must be integers from 30 to 100 that represent the percentage of the index block that REORG can fill. REORG uses this number to decide whether to place more information in an index block, or create space by moving data to another block. The format of the INDEX_FILL_FACTOR qualifier is:
-I[NDEX_FILL_FACTOR]=integer
Under certain conditions, especially with large database block sizes, it may be possible to achieve faster throughput by using a smaller fill factor for index blocks than for data blocks. By default, the INDEX_FILL_FACTOR is the value of FILL_FACTOR regardless of whether that value is explicitly specified or implicitly obtained by default.
KEEP
For a MUPIP REORG TRUNCATE, KEEP allows the specification, as a decimal integer number of blocks or a percentage from 0% to 99% of the starting total blocks, the total amount of space the truncate operation should leave (i.e., not truncate) in the database file at the completion of other REORG operations. The format of the KEEP qualifier is:
[-keep=blocks|percent%]
-NOCOALESCE
-NOCOALESCE specifies to MUPIP REORG to skip actions that increase block density. The format of the NOCOALESCE qualifier is :
-NOCO[ALESCE]
By default, MUPIP REORG attempts to rearrange blocks which are below the specified fill-factor to pack data more densely. More tightly packed data can increase block I/O and reduce the storage requirements.
-NOSPLIT
NOSPLIT specifies to MUPIP REORG to skip actions that decrease block density. The format of the NOSPLIT qualifier is:
-NOSP[LIT]
By default, MUPIP REORG attempts to rearrange blocks which are above the specified fill-factor to pack data less densely. Having empty space in blocks can reduce the need for block splits going forward, which may improve performance.
-NOSWAP
NOSWAP specifies to MUPIP REORG to skip actions that increase block adjacency. The format of the NOSWAP qualifier is:
-NOSW[AP]
By default, MUPIP REORG attempts to rearrange blocks so that logically related blocks have adjacent block numbers. On rotating storage, this tends to improve performance; on solid state storage the results are device specific and may increase device wear. Swap activities tend to generate a lot of journal file volume.
-RESUME
For an interrupted REORG operation, RESUME allows the user to resume the REORG operation from the point where the operation stopped. REORG stores the last key value in the database file header. The format of the RESUME qualifier is:
-R[ESUME]
With RESUME specified, the program retrieves the last key value, from the database file header, and restarts operations from that key.
-REGION
Specifies that REORG operate in the regions in the associated list and restricts REORG to the globals in those regions that are mapped by the current global directory; it does not have the same interactions as EXCLUDE and SELECT, but it does not mitigate those interactions when combined with them.
The format of the REGION qualifier is:
-R[EGION=region-list | -R[EGION] region-list
region-list may specify more than one region of the current global directory in a list. Regions are case-insensitive, separated by a comma, and wildcards can be used to specify them. Any region-name may include the wildcard characters * and % (remember to escape them to protect them from inappropriate expansion by the shell). Any region name expansion occurs in M (ASCII) collation order.
-SELECT
Specifies that REORG reorganizes only the globals in the associated list; globals not on the list may be modified by block swaps with selected globals unless they are named with EXCLUDE; SELECT can be difficult to use efficiently because it tends to de-optimize unselected globals unless they are named in an EXCLUDE list (which introduces inefficiency).
The format of the SELECT qualifier is:
-S[ELECT]=global-name-list
By default, REORG operates on all globals in all database files identified by the current Global Directory for the process executing the MUPIP command.
One of the functions performed by REORG is to logically order the globals on which it operates within the file. Unless the EXCLUDE and SELECT qualifiers are properly used in tandem, repeating the command with different selections in the same file wastes work and leaves only the last selection well organized.
If you enter the REORG SELECT=global-name-list command and the specified globals do not exist, REORG issues a message to the screen and continues to process any specified globals that exist. If REORG is unable to process any globals, it terminates with an error.
Arguments for this qualifier may be an individual global name, a range of global names, or a list of names and prefixes followed by the wildcard symbol. The caret symbol (^) in the specification of the global is optional.
The global name can be:
A global name, such as ACN
A range of global names, such as A7:B7
A list, such as A,B,C.
Global names with the same prefix such as TMP*.
In the first case, REORG only includes global ^ACN. In the second case, REORG includes all global names in the collating sequence A7 to B7. For the third case, REORG includes A, B, and C. In the last case, REORG includes all globals prefixed with TMP.
By default, REORG selects all globals.
TRUNCATE
Specifies that REORG, after it has rearranged some or all of a region's contents, should attempt to reduce the size of the database file and return free space to the file system. TRUNCATE only applies to BG databases. The format of the TRUNCATE qualifier is:
-t[runcate][=percentage]
The optional percentage (0-99) provides a minimum amount for the reclamation; in other words, REORG will not truncate the file unless it can release at least this percentage of the file; the default (0) has it release anything it can. TRUNCATE always returns space aligned with bit map boundaries, which fall at 512 database block intervals. TRUNCATE analyses the bit maps, and if appropriate, produces before image journal records as needed for recycled (formerly used) blocks. The journal extract of a truncated database file may contain INCTN records having the inctn opcode value 9 indicating that the specific block was marked from recycled to free by truncate.
Note
TRUNCATE does not complete if there is a concurrent online BACKUP or use of the snapshot mechanism, for example by INTEG.
Examples for MUPIP REORG
Example:
$ mupip reorg
Fill Factor:: Index blocks 100%: Data blocks 100%
Global: CUST (region DEFAULT)
Blocks processed : 667340
Blocks coalesced : 601487
Blocks split : 0
Blocks swapped : 319211
Blocks freed : 646964
Blocks reused : 298814
Blocks extended : 0
Global: HIST (region HIST)
%YDB-I-FILERENAME, File /var/myApp/prod/journals/hist.mjl is renamed to /var/myApp/prod/journals/hist.mjl_2015289165050
Blocks processed : 337069
Blocks coalesced : 12888
Blocks split : 0
Blocks swapped : 329410
Blocks freed : 315998
Blocks reused : 308337
Levels Eliminated : 1
Blocks extended : 0
$
In this output:
Blocks processed - the number of blocks originally used by the global variable
Blocks coalesced - the number of blocks that were sufficiently compacted enough to free a block
Blocks split - the number of blocks expanded enough to require the allocation of a new block
Blocks swapped - the number of blocks moved to improve adjacency; this can exceed the number of blocks processed as a consequence of the movement of blocks
Blocks freed - the number of blocks formerly used that were released by a combination of swaps and coalesces
Blocks reused - blocks freed, and then reused
Levels eliminated - reduction in the depth of the global variable tree
Blocks extended - the number of blocks the database grew during the reorg
Note also that owing the database update activity of REORG, the hist.mjl journal file reached its limit, requiring MUPIP to switch the journal file.
Example:
$ mupip reorg -exclude="^b2a,^A4gsEQ2e:^A93"
This example performs a MUPIP REORG operation on all globals excluding ^b2a and all globals ranging from ^A4gsEQ2e to ^A93.
Example:
If the forecasted growth of a global is 5% per month from relatively uniformly distributed updates, and REORGs are scheduled every quarter, the FILL_FACTOR for both the original LOAD and subsequent REORGs might be 80 percent 100 - ((3 months + 1 month "safety" margin) * five percent per month). The REORG command based on the above assumptions is as follows:
$ mupip reorg -fill_factor=80
Example:
The following example uses REORG ENCRYPT to encrypt a database "on the fly". This is a simple example created for demonstration purposes. It is NOT recommended for production use. Consult your YottaDB support channel for specific instructions on encrypting an unencrypted database.
Create an empty default unencrypted database.
$ydb_dist/yottadb -r ^GDE exit
$ydb_dist/mupip create
Setup the GNUPG home directory.
export GNUPGHOME=$PWD/.helengnupg3
mkdir $GNUPGHOME # Ensure that you protect this directory with appropriate permissions.
chmod go-rwx $GNUPGHOME
Create a new key. Enter demo values. Accept default values. Choose a strong passphrase.
gpg --gen-key
Edit the key to add a new sub-key:
gpg --edit-key helen.keymaster@yottadb
Type addkey, select option 6 RSA (encrypt only), and accept default values and execute the following commands:
gpg --gen-random 2 32 | gpg --encrypt --default-recipient-self --sign --armor > ydb_workshop_key.txt
gpg --decrypt < ./ydb_workshop_key.txt | gpg --encrypt --armor --default-recipient-self --output ydb.key
Refer to the 'man gpg; a description on the qualifiers for gpg.
Create a gtmcrypt_config file as following:
$ cat config
database: {
keys: (
{
dat: "/path/to/yottadb.dat" ;
key: "/path/to/ydb.key" ;
}
);
}
Set the environment variable gtmcrypt_config to point to this config file.
export gtmcrypt_config=$PWD/config
Set the environment variable ydb_passwd.
echo -n "Enter passphrase for ydb.key: " ; export ydb_passwd=`$ydb_dist/plugin/gtmcrypt/maskpass|cut -f 3 -d " "`
Execute the following commands:
$ mupip set -encryptable -region DEFAULT
$ mupip reorg -encrypt="ydb.key" -region DEFAULT
mupip reorg -encrypt="ydb.key" -region DEFAULT
Region DEFAULT : MUPIP REORG ENCRYPT started
Region DEFAULT : Database is now FULLY ENCRYPTED with the following key: ydb.key
Region DEFAULT : MUPIP REORG ENCRYPT finished
Execute the following command when encryption completes.
$ mupip set -encryptioncomplete -region DEFAULT
Database file /home/gtc_twinata/staff/nitin/tr11/yottadb.dat now has encryption marked complete
Always keep the keys in a secured location. Always set gtmcrypt_config and ydb_passwd to access the encrypted database.
-UPGRADE
MUPIP REORG UPGRADE is not supported in YottaDB r2.00. YottaDB intends to support it in a future release. See UPGRADE.
USER_DEFINED_REORG
The major REORG operations are COALESCE, SPLIT and SWAP, in terms of how database files are defragmented and reorganized.
As defined above:
Blocks coalesced - the number of blocks that were sufficiently compacted enough to free a block
Blocks split - the number of blocks expanded enough to require the allocation of a new block
Blocks swapped - the number of blocks moved to improve adjacency; this can exceed the number of blocks processed as a consequence of the movement of blocks
USER_DEFINED_REORG gives the user more control over which operations are performed during the REORG process.
As REORG is IO intensive, an organization may opt to skip swapping or splitting files, especially as these operations may only yield a minimal benefit while also impacting the operation of normal processes.
Examples of USER_DEFINED_REORG are as follows:
mupip reorg -user_defined_reorg="DETAIL"
This example records every block and every operation (coalesce/swap/split) and gives detailed information about the REORG.
mupip reorg -user_defined_reorg="NOCOALESCE"
This example carries out the REORG but does not coalesce (combine) any blocks.
mupip reorg -user_defined_reorg="NOSPLIT"
This example carries out the REORG but does not split any blocks.
mupip reorg -user_defined_reorg="NOSWAP"
This example carries out the REORG but does not swap any blocks.
mupip reorg -user_defined_reorg="NOSWAP,NOCOALESCE"
This example combines two specifications, and carries out the REORG without swapping or coalescing any blocks.
REPLICATE
Control the logical multi-site operation of YottaDB. For more information on the qualifiers of the MUPIP REPLICATE command, refer to Chapter 7: “Database Replication” .
RESTORE
Integrates one or more BACKUP INCREMENTAL files into a corresponding database. The transaction number in the first incremental backup must be one more than the current transaction number of the database. Otherwise, MUPIP RESTORE terminates with an error.
The format of the RESTORE command is:
RE[STORE] [-NET[TIMEOUT]] [-[NO]E[XTEND]] file-name bytestrm-bkup-list
file-name identifies the name of the database file that RESTORE uses as a starting point.
bytestrm-bkup-list specifies one or more bytestream backup files (generated with BACKUP BYTESTREAM) to RESTORE into the database file specified with file-name. The file-list are separated by commas (,) and must be in sequential order, from the oldest transaction number to the most recent transaction number. RESTORE may take its input from a UNIX file on any device that supports such files.
bytestrm-bkup-list may also include a comma separated list of pipe commands or TCP socket addresses (a combination of IPv4 or IPV6 hostname and a port number) from where MUPIP RESTORE receives bytestream backup files (produced with BACKUP BYTESTREAM).
The current transaction number in the database must match the starting transaction number of each successive input to the RESTORE.
If the BACKUP BYTESTREAM was created using TRANSACTION=1, create a new database with MUPIP CREATE and do not access it, except the standalone MUPIP commands INTEG FILE, EXTEND, and SET before initiating the RESTORE.
-EXTEND
Specifies whether a MUPIP RESTORE operation should extend the database file automatically if it is smaller than the size required to load the data.
The format of the EXTEND qualifier is:
-[NO]E[XTEND]
M activity between backups may automatically extend a database file. Therefore, the database file specified as the starting point for a RESTORE may require an extension before the RESTORE. If the database needs an extension and the command specifies NOEXTEND, MUPIP displays a message and terminates. The message provides the sizes of the input and output database files and the number of blocks by which to extend the database. If the RESTORE specifies more than one incremental backup with a file list, the database file may require more than one extension.
By default, RESTORE automatically extends the database file.
Examples for MUPIP RESTORE
$ mupip restore backup.dat $backup_dir/backup.bk1, $backup_dir/backup.bk2, $backup_dir/backup.bk3
This command restores backup.dat from incremental backups stored in directory specified by the environment variable backup_dir.
$ mupip restore ydb.dat '"gzip -d -c online5pipe.inc.gz |"'
This command uses a pipe to restore ydb.dat since its last DATABASE backup from the bytestream backup stored in online5pipe.inc.gz.
RUNDOWN
When database access has not been properly terminated, RUNDOWN properly closes currently inactive databases, removes abandoned YottaDB database semaphores, and releases any IPC resources used. Under normal operations, the last process to close a database file performs the RUNDOWN actions, and a MUPIP RUNDOWN is not required. If a database file is already properly rundown, a MUPIP RUNDOWN has no effect. If in doubt, it is always to safe to perform a rundown. YottaDB recommends the following method to shutdown a YottaDB application or the system:
Terminate all YottaDB processes, and
Rundown any and all database files that may be active.
MUPIP RUNDOWN checks for version mismatch. If there is a mismatch, it skips the region and continues with the next region. This makes it easier for multiple (non-interacting) YottaDB versions to co-exist on the same machine. Note that YottaDB does not support concurrent access to the same database file by multiple versions of the software.
The format of the MUPIP RUNDOWN command is:
RU[NDOWN] {-FILE file-name|-REGION region-list|-RELINKCTL [dir]|-OVERRIDE}
MUPIP RUNDOWN clears certain fields in a file that is already closed. This facilitates recovery from a system crash or any other operational anomaly.
Use RUNDOWN after a system crash or after the last process accessing a database terminates abnormally. RUNDOWN ensures that open databases are properly closed and ready for subsequent use. RUNDOWN has no effect on any database that is actively being accessed at the time the RUNDOWN is issued.
A successful MUPIP RUNDOWN of a database region removes any current MUPIP FREEZE.
RUNDOWN FILE can be directed to a statistics database file and works even if the corresponding actual database file does not exist.
To ensure database integrity, all system shutdown algorithms should include scripts that stop at YottaDB processes and perform RUNDOWN on all database files.
The RUNDOWN command may include one of the following qualifiers:
-F[ILE]
-R[EGION]=region-list | -R[EGION] region-list
-REL[INKCTL] [dir1]
-O[VERRIDE]
If the RUNDOWN command does not specify either FILE or REGION, it checks all the IPC resources (shared memory) on the system and if they are associated with a YottaDB database, attempts to rundown that file. MUPIP RUNDOWN with no argument removes any statistics database file resources associated with actual database file resources it can remove.
-FILE
Specifies that the argument is a file-name for a single database file. The FILE qualifier is incompatible with the REGION qualifier. If the rundown parameter consists of a list of files, the command only operates on the first item on the list.
Incompatible with: REGION
-OVERRIDE
Overrides the protection that prevents MUPIP RUNDOWN from performing a rundown of a replication-enabled (with BEFORE_IMAGE) database or a non-replicated NOBEFORE-journaled database that was abnormally shutdown. The protection involves issuing the MUUSERLBK error for a previously crashed replication-enabled (with BEFORE IMAGE journaling) database and the MUUSERECOV error for a non-replicated or NOBEFORE-journaled database. Both these errors prevent complications related to data recovery from a journal file or a replication-enabled database.
-REGION
The region-list identifies the target of the RUNDOWN. region-list may specify more than one region of the current global directory in a list. Regions are case-insensitive, separated by a comma, and wildcards can be used to specify them. Any region-name may include the wildcard characters * and % (remember to escape them to protect them from inappropriate expansion by the shell). Any region-name expansion occurs in M (ASCII) collation order.
Use the wildcard "*" to rundown all inactive regions in a global directory.
Incompatible with: FILE
When MUPIP RUNDOWN has no qualifier, it performs rundown on all inactive database memory sections on the node. Because this form has no explicit list of databases, it does not perform any clean up on regions that have no abandoned memory segments but may not have been shutdown in a crash.
-RELINKCTL
Cleans up orphaned Relinkctl files. YottaDB strongly recommends avoiding actions that tend to make such cleanup necessary - for example, kill -9 of YottaDB processes or ipcrm -m of active Relinkctl and/or Rtnobj shared memory segments.
If the optional dir1 is not specified, MUPIP RUNDOWN RELINKCTL examines the environment variable $ydb_routines, attempts to verify and correct their attach counts and runs down all its inactive auto-relink-enabled directories (those with with a *-suffix). Alternatively, one can specify a directory path for the parameter dir1 and MUPIP RUNDOWN RELINKCTL treats it as an auto-relink-enabled directory and runs down the resources associated with this directory. It prints a RLNKCTLRNDWNSUC message on a successful rundown and a RLNKCTLRNDWNFL message on a failure (usually because live processes are still accessing the Relinkctl file).
SEMAPHORE
Reports the details of a space delimited list of semaphore IDs.
The format of the MUPIP SEMAPHORE command is:
MUPIP SEMAPHORE <sem-ids-list>
SEMAPHORE command was added to YottaDB effective release r1.36.
SET
Use MUPIP SET for performance tuning and/or modifying certain database and journal file attributes.
The format of the SET command is:
SE[T] {-FI[LE] file-name|-JN[LFILE] journal-file-name|-REG[ION] region-list}
-AC[CESS_METHOD]={BG|MM}
-[NO]AS[YNCIO]
-[NO]DE[FER_TIME][=seconds]
-[NO]DEFER_ALLOCATE
-EPOCHTAPER
-E[XTENSION_COUNT]=integer(no of blocks)
-F[LUSH_TIME]=integer
-FU[LLBLKWRT]={0|1|2}
-G[LOBAL_BUFFERS]=integer
-H[ARD_SPIN_COUNT]=integer
-[NO]INST[_FREEZE_ON_ERROR]
-K[EY_SIZE]=bytes
-L[OCK_SPACE]=integer
-M[UTEX_SLOTS]=integer
-N[ULL_SUBSCRIPTS]=value
-[NO]LCK_SHARES_DB_CRIT
-PA[RTIAL_RECOV_BYPASS]
-PROBLKSPLIT=N
-[NO]Q[DBRUNDOWN]
-[NO]REA[D_ONLY]
-REC[ORD_SIZE]=bytes
-REORG_SLEEP_NSEC=integer
-REP[LICATION]={ON|OFF}
-RES[ERVED_BYTES]=integer]
-SL[EEP_SPIN_COUNT]=integer
-SPIN_SLEEP_M[ASK]=hex_mask
-STAN[DALONENOT]
-[NO]STAT[S]
-[NO]STD[NULLCOLL]
-T[RIGGER_FLUSH]=integer
-V[ERSION]={V4|V6}
-W[AIT_DISK]=integer
-WR[ITES_PER_FLUSH]=integer
Exclusive access to the database is required if the MUPIP SET command specifies ACCESS_METHOD, GLOBAL_BUFFERS, LOCK_SPACE or NOJOURNAL, or if any of the JOURNAL options ENABLE, DISABLE, or BUFFER_SIZE are specified.
The file-name, journal_file_name, region-list or REPLICATION qualifier identify the target of the SET.
The SET command must include one of the following target qualifiers which determine whether the argument to the SET is a file-name or a region-list.
SET Qualifiers
-FILE
Specifies that the argument is a file-name for a single database file. The format of the FILE qualifier is:
-F[ILE]
Incompatible with: JNLFILE, REGION and REPLICATION
-JNLFILE
Specifies that the argument is a journal-file-name. The format of the JNLFILE qualifier is:
-JNLF[ILE] journal-file-name
Incompatible with: FILE, REGION and REPLICATION
-REGION
Specifies that the argument is a region-list which identifies database file(s) mapped by the current Global Directory. The format of the REGION qualifier is:
-R[EGION] region-list | -R[EGION]=region-list
In other words, either a space or an equal sign can follow the qualifier. The region-list identifies the target of SET. region-list may specify more than one region of the current global directory in a list. Regions are case-insensitive, separated by a comma, and wild-cards can be used to specify them. Any region-name may include the wild-card characters * and % (remember to escape them to protect them from inappropriate expansion by the shell). Any region name expansion occurs in M (ASCII) collation order. Note that region-list must immediately follow the REGION qualifier. Older versions of YottaDB used only a space and permitted other qualifiers between the REGION qualified and region-list.
Incompatible with: FILE, JNLFILE and REPLICATION
-REPLICATION
Specifies whether replication is on or off. The format of the REPLICATION qualifier is:
-REP[LICATION]={ON|OFF}
Incompatible with: JNLFILE
The following sections describe the action qualifiers of the MUPIP SET command exclusive of the details related to journaling and replication, which are described in Chapter 6: “YottaDB Journaling” and Chapter 7: “Database Replication”. All of these qualifiers are incompatible with the JNLFILE and REPLICATION qualifiers.
-ACCESS_METHOD
Specifies the access method (YottaDB buffering strategy) for storing and retrieving data from the global database file. The format of the ACCESS_METHOD qualifier is:
-AC[CESS_METHOD]=code
For more information on specifying the ACCESS_METHOD,refer to Segment Qualifiers.
-ASYNCIO
Specifies whether to use asynchronous I/O for an access method BG database, rather than using synchronous I/O through the file system cache. ASYNCIO is incompatible with the MM access method and an attempt to combine the two with MUPIP SET produces a ASYNCIONOMM error. The format of the ASYNCIO qualifier is:
-[NO]AS[YNCIO]
For more information on specifying ASYNCIO,refer to Segment Qualifiers.
-DEFER_TIME
Specifies, in MM access mode, the multiplying factor applied to the flush time to produce a wait after an update before ensuring a journal buffer write to disk; the default is 1. A value of 2 produces a wait of double the flush time. NODEFER_TIME or a value of -1 turns off timed journal writing, leaving the journal, under light update conditions, to potentially get as stale as the epoch time. Note that, in MM mode without the sync_io option set, absent a VIEW("JNLFLUSH") from the application, YottaDB only fsyncs the journal at the epoch. The format of the DEFER_TIME qualifier is:
-[NO]D[efer_time][=seconds]
-DEFER_ALLOCATE
With DEFER_ALLOCATE, YottaDB instructs the file system to create the database file as a sparse file. Before using DEFER_ALLOCATE, ensure that your underlying file system supports sparse files. By default UNIX file systems, and YottaDB, use sparse (or lazy) allocation, which defers actual allocation until blocks are first written. The format of the DEFER_ALLOCATE qualifier is:
-[NO]DEFER_ALLOCATE
Utilities such as du report typically show lower disk space usage for a database file with DEFER_ALLOCATE because YottaDB instructs the file system to defer disk space allocation to the time when there is an actual need. With NODEFER_ALLOCATE, such utilities report higher disk space usage count as YottaDB instructs the file system to preallocate disk space without waiting for a need to arise.
DEFER_ALLOCATE makes database file extensions lighter weight. However, disk activity may tend towards causing fragmentation.
To switch an existing database file so it immediately preallocates all blocks, first use MUPIP SET NODEFER_ALLOCATE to set the switch in the database file header, followed by MUPIP EXTEND BLOCKS=n, where n >= 0. Failures to preallocate space produce a PREALLOCATEFAIL error.
The default is DEFER_ALLOCATE.
-EPOCHTAPER
Tries to minimize epoch duration by reducing the number of buffers to flush by YottaDB and the file system (via an fsync()) as the epoch (time-based or due to journal file auto-switch) approaches. Epoch tapering reduces the impact of I/O activity during an epoch event. Application that experience high load and/or need to reduce latency may benefit from epoch tapering. The format of the EPOCHTAPER qualifier is:
-[NO]EPOCHTAPER
-ENCRYPTABLE
Performs some basic encryption checks and marks the database as encryptable. Note that marking a database as encryptable does not encrypt the database. For more information on encrypting a database, refer to -ENCRYPT. The format of the ENCRYPTABLE qualifier is:
-[NO]ENCRYPTA[BLE]
-EXTENSION_COUNT
Specifies the number of GDS blocks by which an existing database file extends. A file or region name is required. This qualifier requires standalone access. The format of the EXTENSION_COUNT qualifier is:
-E[XTENSION_COUNT]=integer
For more information on specifying the EXTENSION_COUNT, refer to Segment Qualifiers.
-FLUSH_TIME
Specifies the amount of time between deferred writes of stale cache buffers. The default value is 1 second and the maximum value is 1 hour. The format of the FLUSH_TIME qualifier is:
-F[LUSH_TIME]=[[[HOURS:]MINUTES:]SECONDS:]CENTISECONDS
-FULLBLKWRT
Determines whether YottaDB writes bytes in the database or filesystem block past the last valid database record. Full block writes can be more efficient with some secondary storage because they avoid the potential read-before-write that can penalize partial block writes. The format of the FULLBLKWRT qualifier is:
-FU[LLBLKWRT]={0|1|2}
When FULLBLKWRT=2, when writing data to a newly allocated block, a process writes a full database block. For example, if the database block size is 8KiB, the filesystem block size is 4KiB, and there are 1587 bytes of database records, 8Ki bytes are written to the filesystem in an attempt to ensure that the database block occupies two contiguous blocks in the filesystem. If a subsequent write to that database block has 1259 bytes of database records, 1259 bytes are written to the filesystem.
When FULLBLKWRT=1, a process writes entire filesystem blocks, even when the database records occupy less than a full block. This may improve filesystem load and improve response time on filesystems where writing less than a full block involves a read-modify-write sequence, and writing a full filesystem block avoids the read.
When FULLBLKWRT=0, a process writes only the actual database records when writing to the filesystem. This is the default, and is appropriate for most situations. Evaluate all three values when you need to maximize throughput.
YottaDB ignores FULLBLKWRT for MM databases. Since MM databases are mapped to virtual memory, YottaDB has no control over actual database writes.
FULLBLKWRT was added to YottaDB effective release r1.36.
-GLOBAL_BUFFERS
Specifies the number of cache buffers for a BG database. This qualifier requires standalone access.The format of the GLOBAL_BUFFERS qualifier is:
-G[LOBAL_BUFFERS]=integer
For more information on ways to determine good working sizes for GLOBAL_BUFFERS, refer to Segment Qualifiers.
In general, increasing the number of global buffers improves performance by smoothing the peaks of I/O load on the system. However, increasing the number of global buffers also increases the memory requirements of the system, and a larger number of global buffers on memory constrained systems can increase the probability of the buffers getting swapped out. If global buffers are swapped out, any performance gain from increasing the number of global buffers will be more than offset by the performance impact of swapping global buffers. Most applications use from 1,000 to 4,000 global buffers for database regions that are heavily used. YottaDB does not recommend using fewer than 256 buffers except under special circumstances.
The minimum is 64 buffers and the maximum is 2,097,151 buffers. By default, MUPIP CREATE establishes GLOBAL_BUFFERS using information entered in the Global Directory.
On many UNIX systems, default kernel parameters may be inadequate for YottaDB global buffers, and may need to be adjusted by a system administrator.
-HARD_SPIN_COUNT
The mutex hard spin count specifies the number of attempts to grab the mutex lock before initiating a less CPU-intensive wait period. The format of HARD_SPIN_COUNT is:
-HARD_SPIN_COUNT=integer
The default value is 128. Except on the advice of your YottaDB support channel, YottaDB recommends leaving the default values unchanged in production environments, until and unless, you have data from testing and benchmarking that demonstrates a benefit from a change.
-INST_FREEZE_ON_ERROR
Enables or disables custom errors in a region to automatically cause an Instance Freeze. This flag modifies the "Inst Freeze on Error" file header flag. The format of the INST_FREEZE_ON_ERROR qualifier is:
-[NO]INST[_FREEZE_ON_ERROR]
For more information on creating a list of custom errors that automatically cause an Instance Freeze, refer to Instance Freeze.
For more information on promptly setting or clearing an Instance Freeze on an instance irrespective of whether any region is enabled for Instance, refer to the Starting the Source Server section of the Database Replication chapter.
-JOURNAL
Specifies whether the database allows journaling and, if it does, characteristics for the journal file.
Note
In regions that have journaling enabled and on, users can switch journal files without either requiring standalone access or freezing updates.
The format of the JOURNAL qualifier is:
-[NO]J[OURNAL][=journal-option-list]
NOJOURNAL specifies that the database does not allow journaling. And also it does not accept an argument assignment.
JOURNAL specifies journaling is allowed. It takes one or more arguments in a journal-option-list.
For detailed description of the all JOURNAL qualifiers and its keywords, refer to SET Action Qualifiers.
-KEY_SIZE
Specifies the maximum key size in bytes for storing and retrieving data from the global database file. The maximum supported size is 1019 bytes. The format of the KEY_SIZE qualifier is:
-K[EY_SIZE]=bytes
For more information on KEY_SIZE, refer to Region Qualifiers.
-LOCK_SPACE
Specifies the number of pages allocated to the management of M locks associated with the database. The size of a page is always 512 bytes. The format of the LOCK_SPACE qualifier is:
-L[OCK]_SPACE=integer
The maximum LOCK_SPACE is 262144 pages.
The minimum LOCK_SPACE is 10 pages.
The default LOCK_SPACE is 40 pages.
For more information on LOCK_SPACE, refer to Segment Qualifiers.
This qualifier requires standalone access.
-MUTEX_SLOTS
Sets the size of a structure that YottaDB uses to manage contention for the principal critical section for a database. Performance issues may occur when there are many processes contending for database access and if this structure cannot accommodate all waiting processes. Therefore, YottaDB recommends setting this value to a minimum of slightly more than the maximum number of concurrent processes you expect to access the database.
The minimum value is 64 and the maximum value is 32768. The default value is 1024. The format of the MUTEX_SLOTS qualifier is:
-M[UTEX_SLOTS]=integer
-NULL_SUBSCRIPTS
Controls whether YottaDB accepts null subscripts in database keys.
Usage:
-N[ULL_SUBSCRIPTS]=value
value can either be T[RUE], F[ALSE], ALWAYS, NEVER, or EXISTING. See GDE chapter for more information on these values of null_subscript.
Prohibiting null subscripts can restrict access to existing data and cause YottaDB to report errors.
The default value is never.
PROBLKSPLIT
Specifies the approximate number of records a block in the region must exceed before YottaDB considers a proactive block split. The format of the PROBLKSPLIT qualifier is:
-PRO[BLKSPLIT]=integer
In order to minimize restarts YottaDB can proactively split data blocks when updating an data block more than once within a TP transaction or mini-transaction owing to contention. Proactive block splits tend to make globals require more blocks and potentially deeper trees.
A value of one (1) indicates to always consider proactive splits, which might be useful for regions used to hold globals with a first subscript of, for example, $job.
The default value of five (5) indicates that YottaDB considers proactive block split only when the number of records in a block exceed 5.
-QDBRUNDOWN
Shortens normal process shutdown when a large number of processes accessing a database file need to shutdown almost simultaneously, for example, in benchmarking scenarios or emergencies. The format of the QDBRUNDOWN qualifier is:
-[NO]Q[DBRUNDOWN]
When a terminating YottaDB process observes that a large number of processes are attached to a database file and QDBRUNDOWN is enabled, it bypasses checking whether it is the last process accessing the database. Such a check occurs in a critical section and bypassing it also bypasses the usual RUNDOWN actions which accelerates process shutdown removing a possible impediment to process startup. By default, QDBRUNDOWN is disabled.
Note that with QDBRUNDOWN there is a possibility that the last process to exit might leave the database shared memory and IPC resources in need of cleanup. Except after the number of concurrent processes exceeds 32Ki, QDBRUNDOWN minimizes the possibility of abandoned resources, but it cannot eliminate it. When using QDBRUNDOWN, use an explicit MUPIP command such as RUNDOWN or JOURNAL RECOVER or ROLLBACK of the database file after the last process exits, to ensure the cleanup of database shared memory and IPC resources; not doing so risks database damage.
When a database has QDBRUNDOWN enabled, if the number of attached processes ever exceeds 32Ki, YottaDB stops tracking the number of attached processes, which means that it cannot recognize when the number reaches zero (0) and the shared resources can be released. The process that detects this event issues a NOMORESEMCNT in the system log. This means that an orderly, safe shutdown requires a MUPIP JOURNAL -ROLLBACK -BACKWARD for replicated databases, a MUPIP JOURNAL RECOVER BACKWARD for unreplicated journaled databases and a MUPIP RUNDOWN for journal-free databases.
-PARTIAL_RECOV_BYPASS
Sets the CORRUPT_FILE flag in the database file header to FALSE. The CORRUPT_FILE flag indicates whether a region completed a successful recovery. The format of the PARTIAL_RECOV_BYPASS qualifier is:
-PA[RTIAL_RECOV_BYPASS]
For more information, refer to the CORRUPT_FILE qualifier.
-READ_ONLY
Indicates whether YottaDB should treat an MM access method segment as read only for all users, including root. This designation augments UNIX authorizations and prevents any state updates that normally might require an operational action for a database with no current accessing (attached) processes. MUPIP emits an error on attempts to set READ_ONLY on databases with the BG access method, or to set the access method to BG on databases with READ_ONLY set. The YottaDB help databases have READ_ONLY set by default. The format of the READ_ONLY qualifier is:
-[NO]REA[D_ONLY]
Note
When the first process connects to a database, it creates a access-control semaphore as part of the management of the shared resource. However, when processes connect to a READ_ONLY database , each creates a private copy of the in-memory structures for the database and thus a private semaphore.
-RECORD_SIZE
Specifies the maximum record size in bytes for storing and retrieving data from the global database file. The maximum supported size is 1MiB bytes. The format of the RECORD_SIZE qualifier is:
-REC[ORD_SIZE]=bytes
For more information on KEY_SIZE, refer to Region Qualifiers.
-REORG_SLEEP_NSEC
Specifies the number of nanoseconds that a MUPIP REORG process operating between blocks takes to process, with default value of 0 and a maximum of 999999999 (i.e. 999,999,999, or 1 nanosecond less than 1 second). Using non-zero values reduces the IO impact of MUPIP REORG, at the cost of increasing the duration of the operation. Note that the existing environment variable ydb_poollimit is the appropriate technique to limit the impact of MUPIP REORG on global buffers; the -reorg_sleep_nsec can be used to limit the impact on the IO subsystem.
-RESERVED_BYTES
Specifies the size to be reserved in each database block. RESERVED_BYTES is generally used to reserve room for compatibility with other implementations of M or to observe communications protocol restrictions. The format of the RESERVED_BYTES qualifier is:
-RES[ERVED_BYTES]=size
RESERVED_BYTES may also be used as a user-managed fill factor.
The minimum RESERVED_BYTES is 0 bytes. The maximum RESERVED_BYTES is the block size minus the size of the block header which is 7 or 8 depending on your platform. Realistic determinations of this amount should leave room for at least one record of maximum size.
-SLEEP_SPIN_COUNT
Specifies the number of times a process suspends its activity while waiting to obtain critical sections for shared resources, principally those involving databases. The format of the SLEEP_SPIN_COUNT qualifier is:
-SLEEP_SPIN_COUNT=integer
integer is the number of times the process yields to the OS scheduler or sleeps (depending on the SPIN_SLEEP_LIMIT) after exhausting its hard spin count and before enqueuing itself to be awakened by another process releasing the shared resource mutex.
The default is 128.
Except on the advice of your YottaDB support channel, YottaDB recommends leaving the default values unchanged in production environments, until and unless you have data from testing and benchmarking that demonstrates benefits from a change.
-SPIN_SLEEP_MASK
Specifies the maximum number of nanoseconds for processes to sleep while waiting to obtain critical sections for shared resources, principally those involving databases. The format of the SPIN_SLEEP_MASK qualifier is:
-SPIN_SLEEP_MASK=hex_mask
hex_mask is a hexadecimal mask that controls the maximum time (in nanoseconds) the process sleeps on a sleep spin.
The default is zero (0) which causes the process to return control to the UNIX kernel to be rescheduled with no explicit delay. When the value is non-zero, the process waits for a random value between zero (0) and the maximum value permitted by the mask.
Except on the advice of your YottaDB support channel, YottaDB recommends leaving the default values unchanged in production environments, until and unless you have data from testing and benchmarking that demonstrates a benefit from a change.
-STATS
Specifies whether YottaDB should permit statistics sharing for this region. This characteristic permits operational exclusion of statistics sharing for a region. The format of the STATS qualifier is:
-[NO]STAT[S]
At database creation, GDE controls this characteristic, which by default is specified as STATS (on). When on, this characteristic causes YottaDB to create a small MM database for the associated region to hold the shared statistics.
A process disables itself from maintaining the shared statistics when it fails to open a statsDB. It does not, however, disable subsequently starting processes from maintaining the shared statistics.
-STDNULLCOLL
Specifies whether YottaDB uses standard or historical null collation for null-subscripted keys. YottaDB strongly recommends that you use STDNULLCOLL and not the historical null collation. The format of the STDNULLCOLL qualifier is:
-[NO]STD[NULLCOLL]
-TRIGGER_FLUSH
Specifies the decimal value, in buffers, for the threshold at which processes start flushing dirty buffers after each update. The format of the TRIGGER_FLUSH qualifier is:
-T[RIGGER_FLUSH]=integer
-VERSION
Sets the block format version (Desired DB Format field in the file header) for all subsequent new blocks. The format of the VERSION qualifier is:
-V[ERSION]={version}
MUPIP UPGRADE and MUPIP REORG UPGRADE set the Desired DB Format field in the database file header to the latest version while MUPIP REORG DOWNGRADE sets it to the previous version.
For more information on the upgrading or downgrading your database, refer to the release notes document of your current YottaDB version(s).
-WAIT_DISK
Specifies the seconds to wait for disk space before giving up on a database block write, where zero (0) means to give an error immediately without waiting. The format of the WAIT_DISK qualifier is:
-W[AIT_DISK]=seconds
-WRITES_PER_FLUSH
Specifies the decimal number of blocks to write in each flush. The default value is 7. The format of the WRITES_PER_FLUSH qualifier is:
-WR[ITES_PER_FLUSH]=integer
Examples for MUPIP SET
Example:
$ mupip set -journal=on,nobefore -region "*"
This example enables NOBEFORE image journaling and turns on journaling for all regions.
$ mupip set -version=r120 -file yottadb.dat
Database file yottadb.dat now has desired DB format r120
This example sets the block format to r1.20 for all subsequent new blocks in r1.10 database file yottadb.dat.
Example:
$ mupip set -version=r110 -file yottadb.dat
Database file yottadb.dat now has desired DB format r110
This example sets the block format to r1.10 for all subsequent new blocks in r1.00 database file yottadb.dat.
Example:
mupip set -flush_time=01:00:00:00 -region DEFAULT
This example sets flush time to 1 hour. You can also specify flush time in any combination of [[[HOURS:]MINUTES:]SECONDS:]CENTISECONDS. MUPIP interprets FLUSH_TIME=360000 or FLUSH_TIME=00:60:00:00 as FLUSH_TIME=01:00:00:00.
Example:
$ mupip set -region MAMMALS -inst_freeze_on_error
This example enables custom errors in region MAMMALS to cause an Instance Freeze.
SIZE
Estimates and reports the size of global variables using a format that is similar to the one that appears at the end of the MUPIP INTEG FULL report. In comparison with MUPIP INTEG FAST FULL, MUPIP SIZE provides the option of choosing any one of the three estimation techniques to estimate the size of global variables in a database file. These techniques vary in measurement speed and estimate accuracy. The format of the MUPIP SIZE command is:
MUPIP SI[ZE] [-h[euristic]=estimation_technique] [-s[elect]=global-name-list] [-a[djacency]=integer] [-su[bscript]]=global-list [-r[egion]=region-list | r[egion] region-list]
The optional qualifiers of MUPIP SIZE are:
-HEURISTIC=estimation_technique
Specifies the estimation technique that MUPIP SIZE should use to estimate the size of global variables. The format of the HEURISTIC qualifier is:
-h[euristic]={sc[an][,level=<lvl>] | a[rsample][,samples=<smpls>] | i[mpsample][,samples=<smpls>]}
smpls is the number of samples and must be greater than zero (0)
lvl is a positive or negative tree level designation and -(level of the root block) <= lvl <= (level of the root block)
estimation-technique is one of the following:
scan,level=<lvl> : Traverses the global variable tree and counts the actual number of records and blocks at levels from the root down to the level specified by lvl (default is 0, the data blocks). If the given level is non-negative, it is the lowest block level of the global for which the count is requested. So, 0 means all blocks, 1 means all index blocks, 2 means all index blocks of level 2 and above, and so on. SCAN counts a negative level from the root of the global tree where -1 means children of the root. The technique reports the results for levels other than 0 and shows the adjacency for the next lower (one less) level.
arsample,samples=<smpls> : Uses acceptance/rejection sampling of random tree traversals to estimate the number of blocks at each level. It continues until the specified number of samples (default is 1,000) is accepted.
impsample,samples=<smpls> : Uses importance sampling of random tree traversals to weight each sample of the specified number of samples (default is 1,000) in order to estimate the size of the tree at each level.
If HEURISTIC is not specified, MUPIP SIZE uses the ARSAMPLE,SAMPLE=1000 estimation technique.
The 2 sigma column for the two sampling techniques shows the dispersion of the samples (in blocks) and the probability (rounded to a whole percentage) that the actual value falls farther away from the reported value by more than two sigma. With the scan method the "sample" is "complete," so any inaccuracy comes from concurrent updates.
Note
For large databases, MUPIP SIZE is faster than MUPIP INTEG FAST FULL. IMPSAMPLE is expected to be the fastest estimation technique, followed by ARSAMPLE and then SCAN. In terms of accuracy, MUPIP INTEG FAST FULL is the most accurate.
-ADJACENCY=integer
Specifies the logical adjacency of data blocks that MUPIP SIZE should assume during estimation. By default, MUPIP SIZE assumes ADJACENCY=10 and reports the logical adjacency in the "Adjacent" column of the MUPIP SIZE report. Note that adjacency is only a proxy for database organization and its usefulness may be limited by the technology and configuration of your secondary storage. See the INTEG section of this chapter for additional comments on adjacency.
-SELECT
Specifies the global variables on which MUPIP SIZE runs. SELECT is incompatible with SUBSCRIPT. If neither SELECT nor SUBSCRIPT is specified, MUPIP SIZE selects all global variables in the specified region(s).
The format of the SELECT qualifier is:
-s[elect]=global-name-list
global-name-list can be:
A comma separated list of global variables.
A range of global variables denoted by start:end syntax. For example,
-select="g1:g4".A global variable with wildcards, for example, "g*" (the name must be escaped to avoid shell filename expansion)
"*" to select all global variables.
-REGION
Specifies the region on which MUPIP SIZE runs. If REGION is not specified, MUPIP SIZE selects all regions. The format of the REGION qualifier is:
-R[EGION]=region-list | -R[EGION] region-list
The regions in the region-list are case-insensitive. The specified region-list is converted into upper case before processing.
-SUBSCRIPT
Specifies the subscripted/non-subscripted global variables on which MUPIP SIZE runs. SUBSCRIPT is incompatible with SELECT.
The format of the SUBSCRIPT qualifier is:
-SU[BSCRIPT]=global-list
global-list can be:
A range of global variables denoted by start:end syntax. For example,
-subscript="^g1:^g4".A range of subscripted global variables denoted by start:end syntax. For example,
-subscript="^g(1):^g(4)"
Note
Apart from randomness caused by sampling heuristics, MUPIP SIZE also has randomness from concurrent updates because it does not use the snapshot technique that MUPIP INTEG uses.
SUBSCRIPT qualifier was added to YottaDB effective release r1.32.
Examples for MUPIP SIZE
$ mupip size -heuristic="impsample,samples=2000" -select="y*" -region="AREG"
This example estimates the size of all global variable starting with "y". It uses importance sampling with 2000 samples on the region AREG.
$ mupip size -heuristic="scan,level=-1"
This example counts the number of blocks and records at 1 level below the root of the database tree.
$ mupip size -heuristic="arsample" -select="g1:g3"
This example estimates the size of global variables g1, g2 and g3 using accept/reject sampling with the default number of samples regardless of the region in which they reside.
Note
Apart from randomness caused by sampling heuristics, MUPIP SIZE also has randomness from concurrent updates because it does not use the snapshot technique that MUPIP INTEG uses.
STOP
Terminates a YottaDB image. The image executes an orderly disengagement from all databases that are currently open by the process, and then exits. A MUPIP STOP performs a kill -15 and therefore may also be used to stop non-YottaDB images.
The format of the STOP command is:
MUPIP ST[OP] process-id
Use the shell command ps to display a list of active process names and process identifiers (PIDs).
To STOP a process belonging to its own account, a process requires no privileges. To STOP a process belonging to another account, MUPIP STOP must execute as root.
Note
On receipt of a MUPIP STOP signal, a YottaDB process cleans up its participation in managing the database before shutting down. On receipt of three MUPIP STOP signals in a row, a YottaDB process shuts down forthwith without cleaning up - the equivalent of a kill -9 signal. This can result in structural database damage, because YottaDB does not have sufficient control of what happens in response to an immediate process termination to protect against database damage under all circumstances. In all cases, on receipt of a MUPIP STOP, a process will eventually terminate once it gets the resources needed to clean up. Use three MUPIP STOPs in a row only as a last resort, and when you do, perform a MUPIP INTEG at your earliest opportunity thereafter to check for database structural damage, and repair any damage following the procedures in Chapter 11 (Maintaining Database Integrity).You may never have to perform a MUPIP STOP if your application is designed in a way that it reduces or eliminates the probability of a process getting in the final try of a transaction. For more information, refer to the Programmers Guide.
TRIGGER
Examines or loads trigger definitions. The format of the MUPIP TRIGGER command is:
TRIGGER {-STDIN|-TRIG[GERFILE]=<trigger_definitions_file>
[-NOPR[OMPT]]|[-SELE[CT][=name-list|*][-STDOUT|<select-output-file>]|-UPGRADE}
Before you run the MUPIP TRIGGER command:
Set the value of the environment variable ydb_gbldir: to specify the value of a current global directory.
Ensure that the key size, record size, block size of your database is sufficient for storing all planned trigger definitions. You may have to set the key and record sizes larger than the database content would otherwise require.
The qualifiers of the MUPIP TRIGGER command are as follows:
-TRIGGERFILE=<trigger_definitions_file>
Loads a trigger definition file to the database. The format of the TRIGGERFILE qualifier is:
-TRIG[GERFILE]=<trigger_definitions_file> [-NOPR[OMPT]]
For information on the syntax and usage of a trigger definition file, refer to the Triggers chapter and the $ZTRIGGER() section in the Functions chapter of the Programmer's Guide.
A MUPIP TRIGGER TRIGGERFILE operation occurs within a transaction boundary, therefore, if even one trigger from the trigger definition file fails to parse correctly, MUPIP TRIGGER rolls back the entire trigger definition file load. Trigger maintenance operations reserve their output until the transaction commits at which time they deliver the entire output in a consistent way. MUPIP TRIGGER operations have an implicit timeout of zero (0), meaning the read must succeed on the first try or the command will act as if it received no input.
MUPIP TRIGGER TRIGGERFILE ignores blank lines and extra whitespace within lines. It treats lines with a semi-colon in the first position as comments and ignores their content.
If the NOPROMPT option is specified and the trigger definition file contains a
-*line, then YottaDB will delete all triggers without user confirmation.MUPIP TRIGGER compiles the XECUTE action string and rejects the load if the compilation has errors.
Always specify the same value for the environment variable ydb_chset during loading and executing triggers. If you specify different values of ydb_chset during loading and executing triggers, MUPIP TRIGGER generates a run-time error (TRIGINVCHSET). YottaDB does not prevent a process from updating different nodes with triggers using a different character set, however, YottaDB prevents a process from updating the same triggering node with different character sets. Your coding practice, for all database updates, should be to ensure that you provide the same value for ydb_chset during load compilation and run-time compilation.
MUPIP TRIGGER replicate trigger definitions as logical actions from an originating/primary instance to a replicating/secondary instance based on LGTRIG journal records. This permits the instances to have different sets of triggers and differing database layouts (for example, different # of regions, different block sizes, different maximum-record-size, and so on).
MUPIP TRIGGER error messages associated with loading triggers limit trigger expression source lines to 80 characters including a trailing ellipsis to indicate there was more text, and they also replace any non-graphic characters with a dot (.)
YottaDB triggers apply to spanning regions. When $ZTRIGGER() or MUPIP TRIGGER define triggers that apply to globals spanning multiple regions, each of the spanned regions install a definition.
Incompatible with: SELECT
Note
The trigger update summary reports count not only names and option changes as "modified" but also cases where a COMMANDS list changed, even though those are functionally additions or deletions of separate trigger definitions.
-SELECT=name-list
Provides a facility to examine the current trigger definition. SELECT produces a list of the current triggers for a comma-separate list of global variables or trigger names. The format of the SELECT qualifier is:
-SELE[CT][=name-list*][ <select-output-file>]
Name-list can include global names, delimited with a leading caret (^), and/or trigger names (user-defined or auto-generated) with no leading caret. You can specify a trailing asterisk(*) with either.
With no arguments specified, YottaDB treats SELECT as
-select="*"and extracts a list of all current triggers.Optionally, you can specify a file name to redirect the output of the command. If you do not specify a file name, MUPIP TRIGGER prompts for a file name. If you respond with an empty string (RETURN), MUPIP TRIGGER directs the output to STDOUT.
MUPIP TRIGGER SELECT displays all output including errors on STDOUT.
For Trigger definition reporting operations, $ZTRIGGER("SELECT") and MUPIP TRIGGER SELECT, return a non-zero exit status when their selection criteria encounter an error in the select.
MUPIP TRIGGER SELECT works even if a multi-line XECUTE string does not terminate with a newline character. For more information on multi-line XECUTE strings, refer to the -xecute="|<<strlit1"|>> section under Trigger Definition File in the Triggers chapter and the $ZTRIGGER() section in the Functions chapter of the Programmer's Guide.
Note
The output from the MUPIP TRIGGER SELECT command may not be identical to your trigger definition file. This is because YottaDB converts semantically identical syntax into a single internal representation; while SELECT output may not be identical to the TRIGGERFILE input, it has the same meaning. Additionally, MUPIP TRIGGER SELECT displays a field called "Cycle" as part of a comment. Cycle is the number of trigger definition updates (addition, modification, or deletion) performed on a global node. MUPIP TRIGGER treats the deletion of a non-existent trigger as a success; if that is the only operation, or one of a set of successful operations, it returns success 0 to the shell. Also, MUPIP TRIGGER returns failure in case of trigger selection using trigger names where the number after the pound-sign (#) starts with a 0 (which is an impossible auto-generated trigger name).
-UPGRADE
Upgrades older trigger definitions into current format.
The format of the UPGRADE qualifier is:
-UPGRADE
If YottaDB encounters an old trigger definition it produces a NEEDTRIGUPGRD message. To preserve the possibility of a straightforward downgrade to an earlier version, perform a select "*" action with MUPIP TRIGGER (or $ZTRIGGER() and save the result. Note that TRIGGER UPGRADE assumes that the existing trigger definitions are properly defined; if the prior release has produced defective triggers, delete them with a wild-card ("*"), and redefine the triggers in the new release. In the event of a downgrade, delete "*" all triggers before the downgrade and insert the saved version from before the upgrade. Attempting to perform a MUPIP TRIGGER UPGRADE on a database without write authorization to the database produces a TRIGMODREGNOTRW error. The UPGRADE qualifier is not compatible with any other MUPIP TRIGGER qualifier. Trigger upgrades from older versions may produce journal records based on the prior format that a MUPIP JOURNAL RECOVER cannot process correctly, therefore, YottaDB recommends you do them with journaling off, and start with a backup and fresh journal files after the trigger upgrade.
-STDIN
Reads input triggers to be set from stdin.
The format of the STDIN qualifier is:
-STDIN
STDIN is incompatible with any other option.
STDIN qualifier was added to YottaDB effective release r1.32.
-STDOUT
Reports on triggers from the database to stdout.
The format of the STDOUT qualifier is:
-STDOUT
STDOUT is only valid for SELECT and therefore is incompatible with the STDIN, TRIGGERFILE, and UPGRADE options.
STDOUT qualifier was added to YottaDB effective release r1.32.
Examples for MUPIP TRIGGER
This section provides step-by-step instructions for creating, modifying, and deleting triggers. Triggers affect all processes updating a database unlike, for example, environment variables such as $ydb_routines which work on a per process basis. Therefore, YottaDB recommends that you should always have carefully planned procedures for changing triggers in your production environment.
To create a new trigger for global node ^Acct("ID"):
Using your editor, create a trigger definition file called triggers.trg with the following entry:
+^Acct("ID") -name=ValidateAccount -commands=S -xecute="Write ""Hello Earth!"""
Execute a command like the following:
$ mupip trigger -triggerfile=triggers.trg
This command adds a trigger for ^Acct("ID"). On successful trigger load, this command displays an output like the following:
File triggers.trg, Line 1: ^Acct trigger added with index 1
=========================================
1 triggers added
0 triggers deleted
0 trigger file entries not changed
0 triggers modified
=========================================
Now, every S[et] operation on the global node ^Acct("ID") executes the trigger.
Execute a command like the following:
$ mupip trigger -select="^Acct*"
This command displays the triggers. A sample output looks like the following:
;trigger name: ValidateAccount# cycle: 1
+^Acct("ID") -name=ValidateAccount -commands=S -xecute="Write ""Hello Earth!"""
To modify an existing trigger for global node ^Acct("ID"):
You cannot directly replace an existing trigger definition with a new one. With the exception of NAME and OPTIONS, to change an existing trigger, you have to delete the existing trigger definition and then add the modified trigger definition as a new trigger. Note that YottaDB performs two different trigger comparisons to match trigger definitions depending on whether or not S[ET] is the trigger invocation command. If there is a S[ET], then the comparison is based on the global name and subscripts, PIECES, [Z]DELIM, and XECUTE. If there is no SET, YottaDB compares only the global node with subscripts and the XECUTE code value.
Begin by executing the following command:
$ mupip trigger -select="^Acct*"Output file:
Specify trigger_mod.trg as the output file. This file contains entries like the following:
;trigger name: ValidateAccount# cycle: 1
+^Acct("ID") -name=ValidateAccount -commands=S -xecute="Write ""Hello Earth!"""
Using your editor, open trigger_mod.trg and change + (plus) to - (minus) for the trigger definition entry for ValidateAccount and add a new trigger definition for ^Acct("ID"). To avoid inconsistent application behavior, it is important to replace an old trigger with a new one in the same transaction (Atomic). The trigger_mod.trg file should have entries like:
;trigger name: ValidateAccount# cycle: 1
-^Acct("ID") -name=ValidateAccount -commands=Set -xecute="Write ""Hello Earth!"""
;trigger name: ValidateAccount#
+^Acct("ID") -name=ValidateAccount -commands=Set -xecute="Write ""Hello Mars!"""
Execute a command like the following:
$ mupip trigger -triggerfile=trigger_mod.trg
This command displays an output like the following:
File trigger_mod.trg, Line 1: ^Acct trigger deleted
File trigger_mod.trg, Line 3: ^Acct trigger added with index 1
=========================================
1 triggers added
1 triggers deleted
0 trigger file entries not changed
0 triggers modified
=========================================
Congratulations! You have successfully modified the xecute string of ValidateAccount with the new one.
To delete an existing trigger for global node ^Acct("ID"):
Begin by executing the following command:
$ mupip trigger -select="^Acct*"Output file:
Specify trigger_delete.trg as the output file. This file contains entries like the following:
;trigger name: ValidateAccount# cycle: 3
+^Acct("ID") -name=ValidateAccount -commands=S -xecute="Write ""Hello Mars!"""
Using your editor, change + (plus) to - (minus) for the trigger definition entry for ValidateAccount. Alternatively, you can create a file with an entry like -ValidateAccount.
Now, execute a command like the following:
$ mupip trigger -triggerfile=trigger_delete.trg
This command displays an output like the following:
File trigger_delete.trg, Line 2: ^Acct trigger deleted
=========================================
0 triggers added
1 triggers deleted
0 trigger file entries not changed
0 triggers modified
=========================================
You have successfully deleted trigger "ValidateAccount".
To change a trigger name for global node ^Acct("ID"):
Using your editor, create a new file called trigger_rename.trg and add a trigger definition entry for ValidateAcct with the same trigger signature as ValidateAccount. Your trigger definition would look something like:
+^Acct("ID") -name=ValidateAcct -commands=S -xecute="Write ""Hello Mars!"""
Verify that the ValidateAccount trigger exists by executing the following command:
$ mupip trigger -select="^Acct*"Output file:
Respond with an empty string (Press Enter). Confirm that the trigger summary report contains an entry like the following:
;trigger name: ValidateAccount# cycle: 3
+^Acct("ID") -name=ValidateAccount -commands=S -xecute="Write ""Hello Mars!"""
Now, execute a command like the following:
$ mupip trigger -triggerfile=trigger_rename.trg
This command displays an output like the following:
=========================================
0 triggers added
0 triggers deleted
0 trigger file entries not changed
1 triggers modified
=========================================
You have successfully changed the trigger name ValidateAccount to ValidateAcct.
UPGRADE
MUPIP UPGRADE is not supported in YottaDB r2.00. YottaDB intends to support it in a future release. YottaDB r2.00 is fully functional with r1.x database files. If an application needs the larger maximum database file size of r2.00, create a new database file with r2.00 and MUPIP LOAD to load an extract created by MUPIP EXTRACT. In a replicated environment, remember to set the Region Seqno of the new database file to that of the old one (see DSE CHANGE FILEHEADER REG_SEQNO.
MUPIP Command Summary
Command |
Objects |
Main Qualifier |
Standalone Access |
|---|---|---|---|
B[ACKUP] |
region-name file-name |
|
N N N N N N N N N N N N N N N |
CR[EATE] |
- |
|
N.A. |
DO[WNGRADE] |
file-name |
|
Y |
DU[MPFHEAD] |
file-name or region-list |
|
N |
EN[DIANCVT] |
file-name |
|
Y Y |
EXI[T] |
- |
|
N.A. |
EXTE[ND] |
region-name |
|
N |
EXTR[ACT] |
- |
|
N N N N N N N N |
F[REEZE] |
region-list |
|
N N N |
FT[OK] |
File-name |
|
N N N |
H[ELP] |
command-option |
|
N |
I[NTEG] |
File-name or region-list |
|
N N N N N Y N N N N N N N Y N |
IN[TRPT] |
process id |
N.A. |
|
J[OURNAL] |
file-name |
|
N N N N N |
L[OAD] |
file-name |
|
N N N N N N N |
REO[RG] |
- |
|
N N N N N N N N N N N N N N |
REP[LICATE] |
file-name |
|
N N N N N |
RE[STORE] |
file-name or file-list |
|
Y |
RU[NDOWN] |
file-name or region-name |
|
Y N N N |
SE[T] |
file-name or region-name |
|
N N Y Y Y N N Y N N Y Y N N Y Y Y Y Y Y Y Y Y Y N Y Y N N Y Y Y N Y |
SI[ZE] |
global-name-list region-list |
|
N N N |
ST[OP] |
process-id |
|
N.A. |
TRIGGER |
- |
|
N N N N |
UP[GRADE] |
file-name |
|
Y |
The following table summarizes the qualifiers.
Main Qualifier |
MUPIP Command |
Options/Qualifiers |
|---|---|---|
EDITINSTANCE |
|
|
FENCES=<fence-options-list> |
-RECOVER -ROLLBACK |
|
OFF |
|
|
ON |
|
|
INSTANCE_CREATE |
|
|
JOURNAL=<journal-options-list> |
|
|
LOOKBACK_LIMIT=lookback-option-list |
-RECOVER -ROLLBACK |
|
RECEIVER |
|
|
RECOVER |
|
|
EXTRACT |
|
|
ROLLBACK |
|
|
SHOW=<show-option-list> |
|
|
SINCE |
|
|
SO[URCE] |
|
|
VERSION={V4|V5} |
|
