ctys-common-addresssyntax

September, 2011

.

NAME

ctys-common-addresssyntax - Definition of the Generic Address Superset


General

This document describes the common generic address syntax for the command line interface. Additional data interfaces e.g. for LDAP and SNMP exist and may support additional features.

This specification describes the common generic address syntax for the management of single machines and groups of entities. This suffices all supported systems and may for some plugins applicable as a subset only.

The current version provides almost only the <machine-address> and the GROUPS objects, thus the remaining definitions were required for the design of an extendable overal concept.

Basic Syntax Elements

The addressing facility including the namebinding is splitted into a logical description as a general view and it's conrete adaptions which could be implemented by multiple presentations. The foreseen and implemented syntax scanners are designed to allow implementation in a straight-forward manner allowing an simple implementation of hierarchical structured syntax definitions by nested loops.

The following characters are reservered syntax elements, the full set and description is given in the chapter "Options Scanners - Reserved Characters".

'=' Option/Attribute Assignment
Seperator for an option and it's suboptions.

E.g. the CREATE action may be proceeded for a new client to be connected to a running server, here the default type '-t VNC'.
  ctys -a create=CONNECT ... myAccount@myHost

',' Option/Attribute Seperator
Seperator for suboptions belonging to one set of suboptions.

E.g. a hanging machine may be rebooted, where the hypervisor is utilized to a forced reboot of the locked guest os.
  ctys -t QEMU -a cancel=REBOOT,FORCE ... myAccount@myHost

':' Option/Attribute Value Assignment
Seperator for a suboption key and it's arguments.

The string representation of the identifier for a session.
  ctys -a create=LABEL:UserString ... myAccount@myHost

'%'Option/Attribute Multiple-Value Seperator
Seperator for suboption argument values.

E.g. VMware 2.x a credential is required, which could be either suppressed and later provided by the openning dialogue mask, or provided in clear on commandline. A secure alternative may be the application of Kerberos.
  ctys -t VMW \
    -a create=LABEL:MyMachine,USER:myAccountA%mySecretPasswd \
    myAccountB@myTargetHost

'()' Option/Attribute Sets
Grouping target specific context-options belonging to a common target a.k.a. host.

E.g. the previous example could be written as:
  ctys myHostAccount@myTargetHost'( \
        -t VMW \
        -a create=LABEL:MyMachine,USER:myVmAccount%mySecretPasswd\
       )'

This particularly enables the superposition of specific attributes for each of multiple targets:
  ctys -t VMW \
     myAccount@myTargetHost00'( \
        -a create=LABEL:MyMachine,USER:myAccount00%mySecretPasswd00 \
     )' \
     myAccount@myTargetHost01'( \
        -a create=LABEL:MyMachine,USER:myAccount01%mySecretPasswd01 \
     )'



The following arguments handle in general groups of syntax elements defining a set of elements as a group. This comprises the definition of a set of elements on the same level - as siblings, and the definition of a vertical structure of a path - parent-child relations. The contained elements could be targeted instances as well as sets of attributes for a specific instance or a group of instances.

',' Path-Operator
Seperator for sets of attributes and targets on the same level.

The following entities belong to a common level.
  ctys -t create=LABEL:myTest host00,host01,host02

Due to supported common legacy syntax of host addressing the previous example could also be written as
  ctys -t create=LABEL:myTest host00 host01 host02

'.' Set-Operator
Seperator for paths of nested attributes and targets.

The address of VM running on a specific physical machine could be written as:
  ctys -t create=LABEL:myTest myAccount@host00'(-z)'.vm01'(-t VMW -a create=LABEL:...)'

'{}' Group Operator
This ist the common set operator for grouping related syntax elements. Thus braces could be applied to attributes as and to containing instances. This may be particularly required to resolv ambiguity when third-party tools with independent call syntax may be utilized.

  1. Attribute Groups
    The following application shows the application for grouping of syntax-element when a MACRO is applied.
      ctys '{macro:tst-subgroups-01}(-d 99999)'
    
    Where the macro is defined as:
      tst-subgroups-01 =  -a list {host1 host2}
    
    The final expanded syntax is:
      ctys -a list host1'(-d 99999)' host2'(-d 99999)'
    

  2. Target Groups
    Grouping arguments for multiple targets including their specific options belonging to a common high-level-target a.k.a. SUBTASK.

    The following entity defines a group of entities as a new instance, which is a new instance consisting of the set of it's memebers including their context specific suboptions.
      myGroup00:={
         myTargetHost00'(-t VMW -a create=LABEL:MyMachine00a,USER:myAccount00%mySecretPasswd00)', \
         myTargetHost00'(-t VMW -a create=LABEL:MyMachine00b,USER:myAccount00%mySecretPasswd00)', \
         myTargetHost01'(-t QEMU -a create=LABEL:MyMachine01a)'
      }
    

    The new group instance could be applied as:
      ctys myGroup00
    



The destination address of each target could be named by any element of it's <machine-address> as described in the following chapter. In some cases a single attribute may be ambiguous within a distributed multi-user environment. This could for example be true for the hostname of a single virtual machine. Thus multiple attributes may be be required for a unique address.

'[]' Pre-Attribute-To-Address Conversion
Grouping multiple attributes to a unique attribute set for unambiguous resolution to target addresses.

The following entity defines a host unambiguously. This could be applied for example to test machines, which may be a simple backup, or redundant due to intention.

In case of the UnifiedSessionsManager the automated inventory scanner registers by default each present machine without redundancy checks for specific attributes. Thus an identical copy running in physical different private subnets behind NAT routers with identical IP ranges may occur redundant for a variaty of attributes. These targets could be addressed e.g. in the following way.
  1. Check execution state
      ctys [ LABEL:myMultiHost,BASEPATH:/mntn/myTestPool01/myHost007 ]
      
    
  2. Check run state
      ctys -a show [ LABEL:myMultiHost,BASEPATH:/mntn/myTestPool01/myHost007 ]
      
    
  3. Check static information
      ctys -a info [ LABEL:myMultiHost,BASEPATH:/mntn/myTestPool01/myHost007 ]
      
    
  4. Display inventory data
      ctys-host [ LABEL:myMultiHost,BASEPATH:/mntn/myTestPool01/myHost007 ]
      
    
    The brackets pre-resolv the attribute set to an appropriate target address.



The current syntax description may not yet formally be absolutely correct nor complete, but may cover the intended grade of open description and required understanding for it's application. Some modifications are still under development.

Syntax Examples

The previous elements provide for flexible and simplified addressing of hosts and contained applications. This could be as simple as

  ctys-host -a create=LABEL:myDesktop

which starts a VNC desktop on '${USER}@localhost' and assigns the symbolic name 'myDesktop' to it. Same for a remote host '${USER}@myHost'

  ctys-host -a create=LABEL:myDesktop myHost

which starts a VNC desktop(the configured default) on the host 'myHost' and assigns the symbolic name 'myDesktop' to it. The so called label 'myDesktop' could be used as a full scale address id for all further calls.

When required also a some more sophisticated call of a VM stack could be performed too:

  
  ctys \
    [ UUID:3f95fea7-ee51-445b-95ee-4e432e6e4187 ]\
    .{ \
       [ LABEL:myMultiHost,BASEPATH:/mntn/myTestPool01/myHost007 ]'( \
           -t VMW -a create=LABEL:MyMachine00a,USER:myAccount00%myPasswd00\
       )'\
       ,\
       myTargetHost00'( \
           -t VMW -a create=LABEL:MyMachine00b,USER:myAccount00%myPasswd00\
       )', \
    }'(-d 99999)' \
    ,\
    myTargetHost01'( \
      -t QEMU -a create=LABEL:MyMachine01a\
    )'
  



Address Syntax Elements

The following namebinding defines the superset of addressing attributes, which supports explicit addressing of targets as well as generic addressing of single and multiple targets by using search paths and content attributes in analogy to wildcards, a.k.a. keywords or attribute value assertions. The given sub-options are defined by default not to be order dependent, but some may influence the remaining. The keywords are case-insensitive.

The contained parenthesis, angle, and square brackets in the following figures are syntactic helpers. When they are part of the syntax, they will be quoted with single quotation marks.

The addressed top-level entity is the APPLICATION, thus here the <target-application-entity>. This contains in analogy to the OSI model the machine as well as the access point.

  
      <target-application-entity>:=<tae>
      <tae>:=[<access-point>]<application>
  
      <access-point>:={
         <physical-access-point>
         | <virtual-access-point>
         | <dialogue-access-point>
      }
  
  
      <application>:=<host-execution-frame><application-entity>
  
  
      <physical-access-point>:=<machine-address>[:<PM-access-port>]
      <virtual-access-point>:=<machine-address>[:<VM-access-port>]
      <dialogue-access-point>:=<machine-address>[:<HOST-access-port>]
  
  


The machine is addressed by the <machine-address>, which represents physical and virtual machines as well as login-sessions provided by the HOSTs plugin. The specific plugins may suppport a subset of the full scope, but the attributes ID and LABEL are supported in any case. The ID attribuet is here either a persistent identifier, in case of a VM a configuration file, or a dynamic identifier in case of the HOSTs plugin, e.g. for VNC the DISPLAY number excluding the port-offset. Whereas it is defined for X11 as the PID.

  
      <machine-address>:={
        (
           [(ID|I|PATHNAME|PNAME|P):<mconf-filename-path>][,]
           |
           [(ID|I):<id>][,]
        )
        [(BASEPATH|BASE|B):<base-path>[%<basepath>]{0,n}
        [(LABEL|L):<label>][,]
        [(FILENAME|FNAME|F):<mconf-filename>][,]
        [(UUID|U):<uuid>][,]
        [(MAC|M):<MAC-address>][,]
        [(TCP|T):<TCP/IP-address>][,]
      }
  


The type of access varies for the different <access-point>. Therefore the <access-port> has to be provided in several variants which are specific for the various products. The following figure depicts the major items, additional may be required, which are described within the specific subsystem.

  
      <PM-access-port> := (CLIcon|RDPcon|VNCcon|X11con)
      <VMaccess>       := (QEMUcon|VBOXcon|VMWcon|XENcon)
      <HOSTaccess>     := (CLIcon|RDPcon|VNCcon|X11con)
  

These items represent the various plugins, particularly the HOSTs plugin as a set of major desktop and console plugins commonly used for dialogue and batch access to the VMs and PMs.

Each of the listed access types provides several additional options related to the protocol access point and additional feature parameters. These paramteres are described within the specific plugins documents.

  
      CLIcon  := (SHELL)
      QEMUcon := (QEMU-SDL|CLI|VNC|X11)
      RDPcon  := (RDESKTOP)
      VBOXcon := (VirtualBox|VBoxSDL|RDP)
      VMWcon  := (VMware|vmware-rc|firefox|VNC)
      VNCcon  := (VNCVIEWER)
      X11con  := (XTERM|GTERM|EMACS|EMACSM|EMACSA|EMACSAM)
      XENcon  := (CLI|VNC|X11)
  


<access-point>
The complete path to the execution environment.

<access-port>
The port to be used on the access-point.

<application>
The application itself, which has to be frequently used in combination with a given service as runtime environment.

<application-entity>
The executable target entity of the addresses application, which could be an ordinary shell script to be executed by a starter instance, or an selfcontained executable, which operates standalone within the containing entity. E.g. this could be a shared object or an executable.

(basepath|base|b):<base-path>{1,n}
Basepath could be a list of prefix-paths for usage by UNIX "find" command. When omitted, the current working directory of execution is used by default.

(filename|fname|f):<mconf-filename>
A relative pathname, with a relative path-prefix to be used for down-tree-searches within the given list of <base-path>.

So far the theory. The actual behaviour is slightly different, as though as a simple pattern match against a full absolute pathname is performed. Thus also parts of the fullpathname may match, which could be an "inner part". This is perfectly all right, as far as the match leads to unique results.

More to say, it is a feature. Though a common standardname, where the containing directory of a VM has the same name as the file of the contained VM could be written less redundant, when just dropping the repetitive trailing part of the name.

<host-execution-frame>
The starter entity of addressed container, which frequently supports a sub-command-call or the interactive dialog-access of users to the target system.

(id|i):<mconf-filename-path>
The <id> is used for a variety of tasks just as a neutral matching-pattern of bytes, an in some cases as a uniqe VM identifier within the scope of single machine. The semantics of the data is handled holomporphic due to the variety of utilized subsystems, representing various identifiers with different semantics. Thus the ID is defined to be an abstract sequence of bytes to be passed to a specific application a.k.a. plugin, which is aware of it's actual nature.

The advantage of this is the possibility of a unified handling of IDs for subsystems such as VNC, Xen, QEMU and VMware. Where it spans semantics from beeing a DISPLAY number and offset of a base-port, to a configuration file-path for a DomU-IDs, or a PID of a "master process".

This eases the implementation of cross-over function like LIST, because otherwise e.g. appropriate access-rights to the file are required, which is normally located in a protected subdirectory. These has to be permitted, even though it might not be required by the actual performed function.

(LABEL|L):<label>
  <label>={[a-zA-Z-_0-9]{1,n} (n<30, if possible)}
User defined alias, which should be unique. Could be used for any addressing means. MAC|M.

(MAC|M):<MAC-address>
The MAC address, which has basically similar semantically meaning due to uniqueness as the UUID.

Within the scope of ctys, it is widely assumed - even though not really prerequired - that the UUIDs and MAC-Addresses are manual assigned statically, this could be algorithmic too. The dynamic assignment by VMs would lead to partial difficulties when static caches are used.

<mconf-filename>
The filename of the configuration file without the path-prefix.

<mconf-filename-path>
The complete filepathname of the configuration file.

<mconf-path>
The pathname prefix of the configuration file.

(PATHNAME|PNAME|P):<mconf-path>
When a VM has to be started, the <pathname> to it's configuration file has to be known. Therefore the <pathname> is defined. The pathname is the full qualified name within the callers namescope. SO in case of UNIX it requires a leading '/'.

<physical-access-point>:=<machine-address>[:<access-port>]
The physical termination point as the lowest element of the execution stack. This is the first entity to be contacted from the caller's site, normally by simple network access.

<target-application-entity>
The full path of the stacked execution stack, addressing the execution path from the caller's machine to the terminating entity to be executed. This particularly includes any involved PM, and VM, as well as the final executable. Thus the full scope of actions to be performed in order to start the "On-The-Top" executable is contained.

(TCP|T):<tcp/ip-address>
The TCP/IP address is assumed by ctys to assigned in fixed relation to a unique MAC-Address.

(UUID|U):<uuid>
The well known UUID, which should be unique. But might not, at least due to inline backups, sharing same UUID as the original. Therefore the parameter FIRST, LAST, ALL is supported, due to the fact, that backup files frequently will be assigned a name extension, which places them in alphabetical search-order behind the original. So, when using UUID as unique identifier, a backup will be ignored when FIRST is used.

Anyhow, cross-over ambiguity for different VMs has to be managed by the user.

<virtual-access-point>:=<machine-address>[:<access-port>]
The virtual termination point as an element of the execution stack. The stack-level is at least one above the bottom This stack element could be accessed either by it's operating hypervisor, or by native access to the hosted OS.



Stack Addresses

The stack address is a logical collection of VMs, including an eventually basic founding PM, which are in a vertical dependency. The dependency results from the inherent nested physical execution dependency of each upper-peer from it's close underlying peer. Therefore the stack addresses are syntactically close to GROUPS with additional specific constraints, controlling execution dependency and locality. Particularly the addressing of a VM within an upper layer of a stack could be smartly described by several means of existing path addresses schemas. Within the UnifiedSessionsManager a canonical form is defined for internal processing( StacksAsVerticalSubgroups ), which is available at the user interface too. Additional specific syntactical views are implemented in order to ease an intuitive usage. The following section depicts a formal meta-syntax as a preview of the final ASN.1 based definition. A stack address has the syntax as depicted in Figure Stack-Address.

  
      <stack-address>:=<access-point-list>
  
      <access-point-list>:={
             <physical-access-point>
             |<virtual-access-point-path-list>
      }
  
      <virtual-access-point-path-list>:={
             | <empty>
             | <virtual-access-point>
                 ['.'<virtual-access-point-path-list>]
             | [[',']<virtual-access-point-path-list>]
      }
  


A stack can basically contain wildcards and simple regexpr for the various levels, groups of entities within one level could be provided basically to. And of course any MACRO based string-replacement is applicable. But for the following reasons the following features are shifted to a later version:

Wildcards:
An erroneous user-provided wildcard could easily expand to several hundred VMs, which might be not the original intention. Even more worst, due to the detached background operation on remote machines, this can not easily be stopped, almost just by reboot of the execution target and probably any additional involved machine. Which, yes, might take some time, due to the booting VMs.

Level-Groups/Sets:
Due to several higher priorities this version supports explicitly addressed entries only.



Group Resolution

The GROUPS objects are a concatination of <machine-addresses> and nested GROUPS including specific context options.

  
      <group-address>:= (
                 [ <machine-addresses>['(' <machine-options> ')'[',']]{0,n}]
                 [ <group-address>['('     <group-options>   ')'[',']]{0,n}]
      )['('<group-options>')']
  


Groups are valid replacements of any addressed object, such as a HOSTs. Groups can contain in addition to a simple set of hostnames a list of entities with context specific parameters and include other groups in a nested manner. Each set of superposed options is permutated with the new included set.

The resolution of group names is processed by a search path algorithm based on the variable CTYS_GROUPS_PATH ,which has the same syntax as the PATH variable. The search algorithm is a first-wins filename match of a preconfigured set. Nested includes are resolved with a first-win algorithm beginning at the current position.

In addition to simple names an absolute or relative pathname for a group file could be used. For common addressing as the path-seperator a '.' could be used. In case of a '.' the user has to be aware of possible ambiguity, when file extensions are used. The address resolution works on first-match-wins base.

This allows for example the definition of arbitrary categories, such as server, client, desktop, db, and scan. Here are some examples for free definitions of categories based on simple subdirectories to search paths. The level of structuring into subdirectories is not limited.

server/*
A list of single servers with stored specific call parameters. Server is used here as a synonym for a backend process. Which could be either a PM or a VM, the characteristics is the inclusion of the backend process only.

client/*
A list of single clients with stored specific call parameters. This is meant as the user front end only, which could be a CONNECTIONFORWARDING. The user can define this category also as a complete client machine including the backend and frontend, which is a complete client for a service.

desktop/*
A composition of combined clients and servers for specific tasks. This could be specific desktops for office-applications, systems administration, software-development, industrial applications, test environments. Either new entries could be created, or existing groups could be combined by inclusion.

db/*
Multiple sets of lists of targets to be scanned into specific caching databases. This could be used for a working set as well as for different views of sets of machines.

scan/*
A list of items to be scanned by tools for access validation and check of operational mode. Therefore this entities should contain basic parameters onyl, such as machine specific remote access permissions type.

REMARK: The group feature requires a configured SSO, either by SSH-Keys or Kerberos when the parallel or async mode is choosen, which is the default mode. This is required due to the parallel-intermixed password request, which fails frequently.

For additional information on groups refer to GroupTargets and ctys-groups .


The GROUPS objects are a concatination of <machine-addresses> and nested GROUPS including specific context options. The end of the command with it's specific option should be marked by the common standard with a double column '--'.

  
      ctys -a <action> -- '(<glob-opts>)' <group>'('<group-opts>')'
  
      => The expansion of contained hosts results to:
  
         ... 
         <host0>'(<host-opts> <glob-opts> <group-opts>')' 
         <host1>'(<host-opts> <glob-opts> <group-opts>')' 
         ...
  
      => The expansion of contained nested groups results to:
  
         ... 
         <group-member0>'(<glob-opts>)'('<group-opts>')' 
         <group-member1>'(<glob-opts>)'('<group-opts>')' 
         ...
  


The context options are applied succesively, thus are 'no-real-context' options, much more a successive superposition. More worst, the GROUP is a set, thus the members of a group are reordered for display and execution purposes frequently. So the context options are - in most practical cases - a required minimum for the attached entity.




SEE ALSO

UserManual , HowTo

ctys(1) , ctys-vhost(1)




AUTHOR

Arno-Can Uestuensoez <https://arnocan.wordpress.com/>
<https://unifiedsessionsmanager.sourceforge.io/>
<https://github.com/unifiedsessionsmanager>




COPYRIGHT

Copyright (C) 2008, 2009, 2010, 2011, 2020 Ingenieurbuero Arno-Can Uestuensoez

For BASE package following licenses apply,

This document is part of the DOC package,

For additional information refer to enclosed Releasenotes and License files.