ctys-vdbgen

November, 2010

NAME

ctys-vdbgen - generates databases of VMs and PMs

SYNTAX

ctys-vdbgen


   [--append]
   [--base=<base-path-list>]
   [--cacheDB=<output-db-directory-path>]
   [--filecontext]
   [--no-splitted]
   [--progress]
   [--progressall]
   [--replace]
   [--scan-all-states]
   [--splitted]
   [--splitted-keep-files]
   [--stdio]
   [--threads=<#max-number-of-parallel-scans>]

   [-b]
   [-c]
   [-C]
   [-d <debug-level>]
   [-h]
   [-H <help-options>]
   [-t]
   [-T]
   [-V]
   [-X]
   -- 
   <target-list>

DESCRIPTION

ctys-vdbgen generates databases containing inventory data and static generic address mapping for VMs and PMs. The data is collectred from configuration files with additional data from prorietary vendor interfaces and stored for usage by ctys-vhosts (Address-Resolution). The collection of the inventory data is perfomed by scanning local and remote machines. The scan could be applied with by pre-defined constraints for scope and performance variation such as the operational states of the VMs and the specific assigned hypervisors, or restrictions on the search scope for filesystem locations limited to specific partitions or directory-subtrees. The scan once started is executed fully automated, just an operational installation of the UnifiedSessionsManager and access privileges for the scan-operator has to be pre-assured. The scanned database could be shared and utilized by numerous runtime components, e.g. virtual nameservices and graphical starters. The facilities scale up to thousands of entries even by the usage of a spreadsheet compatible file database.

Dynamic entities without persistent data are not scanned into the database and are handled by their dynamic data only. Dynamic entities are for now CLI, RDP, VNC, and X11 instances.

The whole automated initial creation of the cacheDB takes some Minutes for scanning filesystems and presents than the actually available stock of VMs and PMs. The contained entities could be addressed from than on by various attributes. The call interfaces of all components provide for the option '-p' for the selection of alternative databases, e.g. for specific views or pre-validation of test-scans without influence on the production environment. The common database is shared by the whole set of utilities including commandline tools and graphical applications.

Database creation and application

The database contains the hypervisors KVM, QEMU, VBOX, VMW, and XEN, additionally PMs. For each entry also default login desktops or commandline applications are stored for automation of console interconnection as well as native logins - based on CLI, X11, VNC, and RDP.

ctys-vdbgen itself is a wrapper for ctys -a ENUMERATE=machine....

The output of the ENUMERATE action for a given list is stored into one file contained in the given <output-db-directory-path>, or "$HOME/.ctys/db/default" respectively.

The mapping contains available VMs for a given list of PMs as well as the PMs. Multiple databases could be generated as specific access-groups which could be selected by the standard parameter -p for all ctys utilities with "cacheDB" access.

The scanned entities rely either on proprietary vendor files or interfaces, and/or on ctys specific configuration files containing extended GuestOS information. These documented ctys configuration files could be created by the utility ctys-createConfVM automatic, or interactive. Alternatively manual creation could be proceeded. These stored configuration could be in general imported into the UnifiedSessionsManager by the ENUMERATE action for dynamic runtime purposes as well as for storage within the database.

REMARKS:

Multiple-Entries of IP, and/or MAC addresses
All entries are evaluated and checked for matching indexes. This means for a the index of an "IP0" an matching index of "eth0" is searched, if missing a warning is generated. Whereas "eth0" without an IP address will be accepted.
The second specific is the generation of one seperate entry for each of the resolved IP-MAC pair. This is due to a network-centric view on the interfaces, which together found a network and represent their hosts to the outside.
Other Multiple-Entries
The common behaviour in case of multiple entries is to use the FIRST-ONLY. Even though some tools might present more than one, it must not be relied on!
This can be varied by option for some utilities.
Whitespaces
Whitespaces are generally not supported. When required within suboptions the '%' sign has to be used for padding field-separators. Anyhow, please avoid them, at least for now!

Due to the defaults described in the following subsection for options, the call

  ctys-vdbgen <host1> <host2> ...

leads to the default call

  ${HOME}/bin/ctys -a enumerate=machine,b:\$HOME >\
    ${HOME}/.ctys/db/default/enum.fdb <host1> <host2> ...

Where the PM plugin by default additionaly checks for the PM configuration which is located in

  /etc/ctys.d/pm.conf

For additional information refer to GENMCONF.

A special append-mode is supported for addition of data. This mode does a pure concatenation only, redundancy of added data is not tested. Therefore the user should be aware, when calling append mode, whether he already updated the data of that node.

The deletion of data has to be performed manually for now, a simple ASC-II editor, MS-Excel, or the spreadsheet application of OpenOffice could be used for this task. The data format is described in the appendix of the ctys manual.

ATTENTION:

When debian is accessed with an automounted filesystem the internal call of 'find' command does not force the mount including recognition of the complete contained filesystem, even worse, does this different call-by-call! So the work-around for now is either a fixed mount, or a 'ls -R' before the scan.

OPTIONS

ATTENTION:

The following options are case sensitive, thus are required literally. If not printed properly, word-options have 2 leading hyphens '-'+'-'="--".

<same as "ctys -a enumerate=machine,b:<srcpathlst>" With specific environment variables for modification of the default search path DEFAULT_ENUM_BASE.
```
  <srcpathlst>=
     "${BASEPATHLST:-$DEFAULT_ENUM_BASE}\
          ${BASEAPPENDLST:+%$BASEAPPENDLST}"
```
which expands by default to
```
  "ctys -a enumerate=machine,b:$DEFAULT_ENUM_BASE ..."
```
All context options and suboptions will be passed through, so apply same as for ENUMERATE action.

--append: This activates the append mode, just appending data to present fdb or creating a new if missing. The modes append, replace, and stdio are exclusive and could not be combined.
This is foreseen for addition of minor changes and avoiding a long running recollection of each data set from each target.

-b: Background mode, refer to common options -b
--base=<base-path-list>: A <base-path-list> to be scanned exclusively. This can be combined with multiple targets and the "--append" mode for quick addition of specific new VMs. The path could be the contianing directory for a single VM only.
-c: Nameservice caching mode, refer to common options -c

-C

Result data caching mode, refer to common options -C

--cacheDB=<output-db-directory-path>

Pathname for directory containing DB file to be created. This file stores the mapping records generated from ctys-ENUMERATE literally.

The evaluation order priority and predefined default values for the directory is defined as follows:

"--cacheDB=<output-db-directory-path>"

DEFAULT_VDBGEN_DB=$HOME/.ctys/db/default

DEFAULT_DBPATHLST=$HOME/.ctys/db/default

"--cacheDB=<output-db-directory-path>"

DEFAULT_VDBGEN_DB=<install-path>/conf/db/default

DEFAULT_DBPATHLST=<install-path>/conf/db/default

The name for the filedb itself is hard-coded as "enum.fdb". Additional files are be stored within the cacheDB directory.

REMARK: Only ONE path could be provided here for the PATHLST.

--filecontext: This switches off several defaults and assumes that a configuration file with a complete CONTEXT is provided.
The explicit suppression of defaults is required, due to the addition of chained context options for dialogue based entries from right to left. The evaluation of superposing options is proceeded from-left-to-right. Thus the last wins, so the last dialogue entries have the highest priority.

-h: Print help, refer to "-H" for additional information.

-H <help-option>: The extended help option is based on system interfaces for display of manpages, PDF and HTML documents. This comprises the man pages and installed manuals.
For additional help refer to the documents or type ctys -H help.

--no-splitted: This deactivates the splitted mode and passes the whole parallel operations control to ctys controlled by the parameter -b

--progress

This activates the formatted table output, therefore some traces are filtered and displayed in a compact overview, indicating the progress and required processing time.

For the application of the "--progress" option additionally the appropriate trace switches for the target has to be set. This is due to the required activation of the appropriate trace output for post-filtering on local machine. In case of usage of MACROS or GROUPS these could be stored permanenetly as required.

E.g. the following -d context options for the target of enumeration required to be present.

  
  ctys-vdbgen \
  --cacheDB=/home/tstusr/.ctys/db/tmp \
  --append \
  --base=qemu \
  lab00'(-d 2,s:16,w:0,p)'

When "progress" choosen the data for each displayed instance is a post-result, calculated as a sumup after finished processing. For continuous display refer to "progressall".

--progressall

This activates the formatted table output with continuous display, where the data is filtered as well, but any relevant entity with an intermidiate result is displayed. The final sum-up data is additionally displayed as soon as the somplete set of results for the current entity is available.

--replace: This activates the explicit replace mode, which is required when a "enum.fdb" alread exists. Choosing this will delete the present "enum.fdb", when the data has to be kept, make a backup before activating this option. The modes append, replace, and stdio are exclusive and could not be combined.

--scan-all-states: This option ignores the actual avaiability of an appropriate hypervisor and scans all present configurations. This is particularly helpful for test environments, where the hypervisor on a machine might be changed frequently.

--splitted: This provides for the by-target-and-by-session splitted parallel operations. This option utilizes the -b seq,ser option and establishes it's own background and parallel operations control. The number of parallel tasks is controlled by the environment variable CTYS_VDBGEN_PARTARGETS which is set to DEFAULT=10. In addition for each task a temporary cache file is created within the cacheDB directory and kept for later analysis. Thus e.g. in case of machine and plugin specific bug-tracing the specific results are availabel for fast analysis.

--splitted-keep-files: Keeps temporary files when in 'splitted' mode. These have to be removed manually once kept.

--stdio: Writes it's output to STDOUT only. The modes append, replace, and stdio are exclusive and could not be combined.

--threads=<#max-number-of-parallel-scans>: Number of tasks to be scanned in parallel, applicable for 'splitted' mode only. Default is CTYS_VDBGEN_PARTARGETS=10.

-t: Session type, refer to common options -t .

-T: Preload of session type plugins, refer to common options -T
-V: Version.
-X: Terse output format, effects "-V" when set left-of.

ARGUMENTS

Any target to be enumerated, could be hosts and nested groups. Additional context arguments are supported, but should be added carefully, e.g. a non-applicable action like '-a CREATE' will lead to misfunction.

EXIT-VALUES

0: OK:: Result is valid.
1: NOK:: Erroneous parameters.
2: NOK:: Missing an environment element like files or databases.

AUTHOR

Written and maintained by Arno-Can Uestuensoez:

Maintenance:	<<acue_sf1 (a) sourceforge net>>
Homepage:	<https://arnocan.wordpress.com>
Sourceforge.net:	<http://sourceforge.net/projects/ctys>
Project moved from Berlios.de to OSDN.net:	<https://osdn.net/projects/ctys>
Commercial:	<https://arnocan.wordpress.com>

COPYRIGHT

For BASE package following licenses apply,

for software see GPL3 for license conditions,
for documents see GFDL-1.3 with invariant sections for license conditions,

This document is part of the DOC package,

for documents and contents from DOC package see
'Creative-Common-Licence-3.0 - Attrib: Non-Commercial, Non-Deriv'

with optional extensions for license conditions.

For additional information refer to enclosed Releasenotes and License files.