:man source:   crm
:man version:  1.1
:man manual:   Pacemaker documentation

crm(8)
======


NAME
----
crm - Pacemaker command line interface for configuration and management


SYNOPSIS
--------
*crm* [-D output_type] [-f file] [-hFRDw] [--version] [args]


DESCRIPTION
-----------
Pacemaker configuration is stored in a CIB file (Cluster
Information Base). The CIB is a set of instructions coded in XML.
Editing the CIB is a challenge, not only due to its complexity
and a wide variety of options, but also because XML is more
computer than user friendly. The `crm` shell alleviates this
issue significantly by introducing small and simple configuration
language. The CIB is translated into this language on the fly.

`crm` is also a management tool. For management tasks it relies
almost exclusively on other command line tools, such as
`crm_resource(8)` or `crm_attribute(8)`.  Use of these programs
is, however, plagued by the notorious weakness common to all UNIX
tools: a multitude of options, necessary for operation and yet
very hard to remember. `crm` tries to present a consistent
interface to the user and to hide the arcane detail.

It may be used either as an interactive shell or for single
commands directly on the shell's command line. It is also
possible to feed it a set of commands from standard input or a
file, thus turning it into a scripting tool. Templates with ready
made configurations may help newbies learn about the cluster
configuration or facilitate testing procedures.

The `crm` shell is line oriented: every command must start and
finish on the same line. It is possible to use a continuation
character (`\`) to write one command in two or more lines. The
continuation character is commonly used when displaying
configurations.

OPTIONS
-------
*-f, --file*='FILE'::
	Load commands from the given file. If the file is `-` then
    use terminal `stdin`.

*-D, --display=*'OUTPUT_TYPE'::
	Choose one of the output options: `plain`, `color`, or
    `uppercase`. The default is `color` if the terminal emulation
    supports colors. Otherwise, `plain` is used.

*-F, --force*::
    Make `crm` proceed with doing changes even though it would
    normally ask user to confirm some of them. Mostly useful in
    scripts.

*-w, --wait*::
    Make `crm` wait for the transition to finish. Applicable only
    for commands such as "resource start."

*-h, --help*::
    Print help page.

*--version*::
    Print Pacemaker version and build information (Mercurial Hg
    changeset hash).

*-R, --regression-tests*::
    Run in the regression test mode. Used mainly by the
    regression testing suite.

*-d, --debug*::
    Print some debug information. Used by developers. [Not yet
    refined enough to print useful information for other users.]

== Introduction to the user interface

Arguably the most important aspect of such a program is the user
interface. We begin with an informal introduction so that the
reader may get acquainted with it and get a general feeling of
the tool. It is probably best just to give some examples:

1. Command line (one-shot) use:

        # crm resource stop www_app

2. Interactive use:

        # crm
        crm(live)# resource
        crm(live)resource# unmanage tetris_1
        crm(live)resource# end
        crm(live)# node standby node4

3. Cluster configuration:

    # crm<<EOF
    configure
      erase
      #
      # resources
      #
      primitive disk0 iscsi \
        params portal=192.168.2.108:3260 target=iqn.2008-07.com.suse:disk0
      primitive fs0 Filesystem \
        params device=/dev/disk/by-label/disk0 directory=/disk0 fstype=ext3
      primitive internal_ip IPaddr params ip=192.168.1.101
      primitive apache apache \
        params configfile=/disk0/etc/apache2/site0.conf
      primitive apcfence stonith:apcsmart \
        params ttydev=/dev/ttyS0 hostlist="node1 node2" \
        op start timeout=60s
      primitive pingd pingd \
        params name=pingd dampen=5s multiplier=100 host_list="r1 r2"
      #
      # monitor apache and the UPS
      #
      monitor apache 60s:30s
      monitor apcfence 120m:60s
      #
      # cluster layout
      #
      group internal_www \
        disk0 fs0 internal_ip apache
      clone fence apcfence \
        meta globally-unique=false clone-max=2 clone-node-max=1
      clone conn pingd \
        meta globally-unique=false clone-max=2 clone-node-max=1
      location node_pref internal_www \
        rule 50: #uname eq node1 \
        rule pingd: defined pingd
      #
      # cluster properties
      #
      property stonith-enabled=true
      commit
    EOF

If you've ever done a CRM style configuration, you should be able
to understand the above examples without much difficulties.  The
shell should provide a means to manage the cluster efficiently or
put together a configuration in a concise manner.

The `(live)` string in the prompt signifies that the current CIB
in use is the cluster live configuration. It is also possible to
work with the so-called shadow CIBs, i.e. configurations which
are stored in files and aren't active, but may be applied at any
time to the cluster.

Since the CIB is hierarchical such is the interface too. There
are several levels and entering each of them enables the user to
use a certain set of commands.

=== Shadow CIB usage

Shadow CIB is a normal cluster configuration stored in a file.
They may be manipulated in the same way like the _live_ CIB, but
these changes have no effect on the cluster resources. The
administrator may choose to apply any of them to the cluster,
thus replacing the running configuration with the one which is in
the shadow CIB. The `crm` prompt always contains the name of the
configuration which is currently in use or string _live_ if we
are using the current cluster configuration.

At the configure level no changes take place before the `commit`
command. Sometimes though, the administrator may start working
with the running configuration, but change mind and instead of
committing the changes to the cluster save them to a shadow CIB.
This short `configure` session excerpt shows how:
...............
    crm(live)configure# cib new test-2
    INFO: test-2 shadow CIB created
    crm(test-2)configure# commit
...............

=== Templates

Templates are ready made configurations created by cluster
experts. They are designed in such a way so that users may
generate valid cluster configurations with minimum effort.
If you are new to Pacemaker, templates may be the best way to
start.

We will show here how to create a simple yet functional Apache
configuration:
...............
    # crm configure
    crm(live)configure# template
    crm(live)configure template# list templates
    apache       filesystem   virtual-ip   
    crm(live)configure template# new web <TAB><TAB>
    apache       filesystem   virtual-ip   
    crm(live)configure template# new web apache
    INFO: pulling in template apache
    INFO: pulling in template virtual-ip
    crm(live)configure template# list
    web2-d       web2         vip2         web3         vip          web
...............

We enter the `template` level from `configure`. Use the `list`
command to show templates available on the system. The `new`
command creates a configuration from the `apache` template. You
can use tab completion to pick templates. Note that the apache
template depends on a virtual IP address which is automatically
pulled along. The `list` command shows the just created `web`
configuration, among other configurations (I hope that you,
unlike me, will use more sensible and descriptive names).

The `show` command, which displays the resulting configuration,
may be used to get an idea about the minimum required changes
which have to be done. All `ERROR` messages show the line numbers
in which the respective parameters are to be defined:
...............
    crm(live)configure template# show
    ERROR: 23: required parameter ip not set
    ERROR: 61: required parameter id not set
    ERROR: 65: required parameter configfile not set
    crm(live)configure template# edit
...............

The `edit` command invokes the preferred text editor with the
`web` configuration. At the top of the file, the user is advised
how to make changes. A good template should require from the user
to specify only parameters. For example, the `web` configuration
we created above has the following required and optional
parameters (all parameter lines start with `%%`):
...............
    $ grep -n ^%% ~/.crmconf/web
    23:%% ip 
    31:%% netmask 
    35:%% lvs_support 
    61:%% id 
    65:%% configfile 
    71:%% options 
    76:%% envfiles 
...............

These lines are the only ones that should be modified. Simply
append the parameter value at the end of the line. For instance,
after editing this template, the result could look like this (we
used tabs instead of spaces to make the values stand out):
...............
    $ grep -n ^%% ~/.crmconf/web
    23:%% ip                192.168.1.101
    31:%% netmask 
    35:%% lvs_support 
    61:%% id                websvc
    65:%% configfile        /etc/apache2/httpd.conf
    71:%% options 
    76:%% envfiles 
...............

As you can see, the parameter line format is very simple:
...............
    %% <name> <value>
...............

After editing the file, use `show` again to display the
configuration:
...............
    crm(live)configure template# show
    primitive virtual-ip ocf:heartbeat:IPaddr \
            params ip="192.168.1.101"
    primitive apache ocf:heartbeat:apache \
            params configfile="/etc/apache2/httpd.conf"
    monitor apache 120s:60s
    group websvc \
            apache virtual-ip
...............

The target resource of the apache template is a group which we
named `websvc` in this sample session.

This configuration looks exactly as you could type it at the
`configure` level. The point of templates is to save you some
typing. It is important, however, to understand the configuration
produced.

Finally, the configuration may be applied to the current
crm configuration (note how the configuration changed slightly,
though it is still equivalent, after being digested at the
`configure` level):
...............
    crm(live)configure template# apply 
    crm(live)configure template# cd ..
    crm(live)configure# show
    node xen-b
    node xen-c
    primitive apache ocf:heartbeat:apache \
        params configfile="/etc/apache2/httpd.conf" \
        op monitor interval="120s" timeout="60s"
    primitive virtual-ip ocf:heartbeat:IPaddr \
        params ip="192.168.1.101"
    group websvc apache virtual-ip
...............

Note that this still does not commit the configuration to the CIB
which is used in the shell, either the running one (`live`) or
some shadow CIB. For that you still need to execute the `commit`
command.

To complete our example, we should also define the preferred node
to run the service:
...............
    crm(live)configure# location websvc-pref websvc 100: xen-b
...............

If you are not happy with some resource names which are provided
by default, you can rename them now:
...............
    crm(live)configure# rename virtual-ip intranet-ip
    crm(live)configure# show
    node xen-b
    node xen-c
    primitive apache ocf:heartbeat:apache \
            params configfile="/etc/apache2/httpd.conf" \
            op monitor interval="120s" timeout="60s"
    primitive intranet-ip ocf:heartbeat:IPaddr \
            params ip="192.168.1.101"
    group websvc apache intranet-ip
    location websvc-pref websvc 100: xen-b
...............

To summarize, working with templates typically consists of the
following steps:

- `new`: create a new configuration from templates
- `edit`: define parameters, at least the required ones
- `show`: see if the configuration is valid
- `apply`: apply the configuration to the `configure` level

=== Tab completion

The `crm` makes extensive use of the tab completion of
`readline`. The completion is both static (i.e. for `crm`
commands) and dynamic. The latter takes into account the current
status of the cluster or information from installed resource
agents. Sometimes, completion may also be used to get short help
on resource parameters. Here a few examples:
...............
    crm(live)# resource
    crm(live)resource# <TAB><TAB>
    bye           failcount     move          restart       unmigrate 
    cd            help          param         show          unmove 
    cleanup       list          promote       start         up 
    demote        manage        quit          status utilization 
    end           meta          refresh       stop          
    exit          migrate       reprobe       unmanage      
    crm(live)resource# end
    crm(live)# configure 
    crm(live)configure# primitive fence-1 <TAB><TAB>
    heartbeat:  lsb:        ocf:        stonith:    
    crm(live)configure# primitive fence-1 stonith:<TAB><TAB>
    apcmaster                external/ippower9258     fence_legacy 
    apcmastersnmp            external/kdumpcheck      ibmhmc 
    apcsmart                 external/libvirt         ipmilan 
    baytech                  external/nut             meatware 
    bladehpi                 external/rackpdu         null 
    cyclades                 external/riloe           nw_rpc100s 
    drac3                    external/sbd             rcd_serial 
    external/drac5           external/ssh             rps10 
    external/dracmc-telnet   external/ssh-bad         ssh 
    external/hmchttp         external/ssh-slow        suicide 
    external/ibmrsa          external/vmware          wti_mpc 
    external/ibmrsa-telnet   external/xen0            wti_nps 
    external/ipmi            external/xen0-ha         
    crm(live)configure# primitive fence-1 stonith:ipmilan params <TAB><TAB>
    auth=      hostname=  ipaddr=    login=     password=  port=      priv=
    crm(live)configure# primitive fence-1 stonith:ipmilan params auth=<TAB><TAB>
    auth* (string)
        The authorization type of the IPMI session ("none", "straight", "md2", or "md5")
    crm(live)configure# primitive fence-1 stonith:ipmilan params auth=
...............

[[SEMCHK]]
=== Configuration semantic checks

Resource definitions may be checked against the meta-data
provided with the resource agents. These checks are currently
carried out:

- are required parameters set
- existence of defined parameters
- timeout values for operations

The parameter checks are obvious and need no further explanation.
Failures in these checks are treated as configuration errors.

The timeouts for operations should be at least as long as those
recommended in the meta-data. Too short timeout values are a
common mistake in cluster configurations and, even worse, they
often slip through if cluster testing was not thorough. Though
operation timeouts issues are treated as warnings, make sure that
the timeouts are usable in your environment. Note also that the
values given are just _advisory minimum_---your resources may
require longer timeouts.

User may tune the frequency of checks and the treatment of errors
by the <<cmdhelp_options_check-frequency,`check-frequency`>> and
<<cmdhelp_options_check-mode,`check-mode`>> preferences.

Note that if the `check-frequency` is set to `always` and the
`check-mode` to `strict`, errors are not tolerated and such
configuration cannot be saved.

=== Access Control Lists (ACL)

It is desirable to allow users fine grained access to the cluster
and not just ``all or nothing:'' all being the two ultimate power
ids `root` and `hacluster`.

Access control lists consist of an ordered set of access rules.
Each rule allows read or write access or denies access
completely. Rules are typically combined to produce a specific
role. Then, users may be assigned a role.

For instance, this is a role which defines a set of rules
allowing management of a single resource:

...............
    role bigdb_admin \
        write meta:bigdb:target-role \
        write meta:bigdb:is-managed \
        write location:bigdb \
        read ref:bigdb
...............

The first two rules allow modifying the `target-role` and
`is-managed` meta attributes which effectively enables users in
this role to stop/start and manage/unmanage the resource. The
constraints write access rule allows moving the resource around.
Finally, the user is granted read access to the resource
definition.

For proper operation of all Pacemaker programs, it is advisable
to add the following role to all users:

...............
    role read_all \
        read cib
...............

For finer grained read access try with the rules listed in the
following role:

...............
    role basic_read \
        read node attribute:uname \
        read node attribute:type \
        read property \
        read status
...............

It is however possible that some Pacemaker programs (e.g.
`ptest`) may not function correctly if the whole CIB is not
readable.

Some of the ACL rules in the examples above are expanded by the
shell to XPath specifications. For instance,
`meta:bigdb:target-role` is a shortcut for
`//primitive[@id='bigdb']/meta_attributes/nvpair[@name='target-role']`.
You can see the expansion by showing XML:

...............
    crm(live) configure# show xml bigdb_admin
    ...
    <acls>
      <acl_role id="bigdb_admin">
          <write id="bigdb_admin-write"
          xpath="//primitive[@id='bigdb']/meta_attributes/nvpair[@name='target-role']"/>
...............

Many different XPath expressions can have equal meaning. For
instance, the following two are equal, but only the first one is
going to be recognized as shortcut:

...............
      //primitive[@id='bigdb']/meta_attributes/nvpair[@name='target-role']
      //resources/primitive[@id='bigdb']/meta_attributes/nvpair[@name='target-role']
...............

XPath is a powerful language, but you should try to keep your ACL
xpaths simple and the builtin shortcuts should be used whenever
possible.

== Reference

We define a small and simple language. Most commands consist of
just a list of simple tokens. The only complex constructs are
found at the `configure` level.

The syntax is described in a somewhat informal manner: `<>`
denotes a string, `[]` means that the construct is optional, the
ellipsis (`...`) signifies that the previous construct may be
repeated, `|` means pick one of many, and the rest are literals
(strings, `:`, `=`).

=== `status`

Show cluster status. The status is displayed by `crm_mon`. Supply
additional arguments for more information or different format.
See `crm_mon(8)` for more details.

Usage:
...............
        status [<option> ...]

        option :: bynode | inactive | ops | timing | failcounts
...............

[[cmdhelp_cib,CIB shadow management]]
=== `cib` (shadow CIBs)

This level is for management of shadow CIBs. It is available both
at the top level and the `configure` level.

All the commands are implemented using `cib_shadow(8)` and the
`CIB_shadow` environment variable. The user prompt always
includes the name of the currently active shadow or the live CIB.

[[cmdhelp_cib_new,create a new shadow CIB]]
==== `new`

Create a new shadow CIB. The live cluster configuration and
status is copied to the shadow CIB. Specify `withstatus` if you
want to edit the status section of the shadow CIB (see the
<<cmdhelp_cibstatus,cibstatus section>>). Add `force` to force overwriting the
existing shadow CIB.

To start with an empty configuration that is not copied from the live
CIB, specify the `empty` keyword. (This also allows a shadow CIB to be
created in case no cluster is running.)

Usage:
...............
        new <cib> [withstatus] [force] [empty]
...............

[[cmdhelp_cib_delete,delete a shadow CIB]]
==== `delete`

Delete an existing shadow CIB.

Usage:
...............
        delete <cib>
...............

[[cmdhelp_cib_reset,copy live cib to a shadow CIB]]
==== `reset`

Copy the current cluster configuration into the shadow CIB.

Usage:
...............
        reset <cib>
...............

[[cmdhelp_cib_commit,copy a shadow CIB to the cluster]]
==== `commit`

Apply a shadow CIB to the cluster.

Usage:
...............
        commit <cib>
...............

[[cmdhelp_cib_use,change working CIB]]
==== `use`

Choose a CIB source. If you want to edit the status from the
shadow CIB specify `withstatus` (see <<cmdhelp_cibstatus,`cibstatus`>>).
Leave out the CIB name to switch to the running CIB.

Usage:
...............
        use [<cib>] [withstatus]
...............

[[cmdhelp_cib_diff,diff between the shadow CIB and the live CIB]]
==== `diff`

Print differences between the current cluster configuration and
the active shadow CIB.

Usage:
...............
        diff
...............

[[cmdhelp_cib_list,list all shadow CIBs]]
==== `list`

List existing shadow CIBs.

Usage:
...............
        list
...............

[[cmdhelp_cib_import,import a CIB or PE input file to a shadow]]
==== `import`

At times it may be useful to create a shadow file from the
existing CIB. The CIB may be specified as file or as a PE input
file number. The shell will look up files in the local directory
first and then in the PE directory (typically `/var/lib/pengine`).
Once the CIB file is found, it is copied to a shadow and this
shadow is immediately available for use at both `configure` and
`cibstatus` levels.

If the shadow name is omitted then the target shadow is named
after the input CIB file.

Note that there are often more than one PE input file, so you may
need to specify the full name.

Usage:
...............
        import {<file>|<number>} [<shadow>]
...............
Examples:
...............
        import pe-warn-2222
        import 2289 issue2
...............

[[cmdhelp_cib_cibstatus,CIB status management and editing]]
==== `cibstatus`

Enter edit and manage the CIB status section level. See the
<<cmdhelp_cibstatus,CIB status management section>>.

[[cmdhelp_ra,Resource Agents (RA) lists and documentation]]
=== `ra`

This level contains commands which show various information about
the installed resource agents. It is available both at the top
level and at the `configure` level.

[[cmdhelp_ra_classes,list classes and providers]]
==== `classes`

Print all resource agents' classes and, where appropriate, a list
of available providers.

Usage:
...............
        classes
...............

[[cmdhelp_ra_list,list RA for a class (and provider)]]
==== `list`

List available resource agents for the given class. If the class
is `ocf`, supply a provider to get agents which are available
only from that provider.

Usage:
...............
        list <class> [<provider>]
...............
Example:
...............
        list ocf pacemaker
...............

[[cmdhelp_ra_meta,show meta data for a RA]]
==== `meta` (`info`)

Show the meta-data of a resource agent type. This is where users
can find information on how to use a resource agent.

Usage:
...............
        meta [<class>:[<provider>:]]<type>
        meta <type> <class> [<provider>] (obsolete)
...............
Example:
...............
        meta apache
        meta ocf:pacemaker:Dummy
        meta stonith:ipmilan
...............

[[cmdhelp_ra_providers,show providers for a RA and a class]]
==== `providers`

List providers for a resource agent type. The class parameter
defaults to `ocf`.

Usage:
...............
        providers <type> [<class>]
...............
Example:
...............
        providers apache
...............

[[cmdhelp_resource,Resource management]]
=== `resource`

At this level resources may be managed.

All (or almost all) commands are implemented with the CRM tools
such as `crm_resource(8)`.

[[cmdhelp_resource_status,show status of resources]]
==== `status` (`show`, `list`)

Print resource status. If the resource parameter is left out
status of all resources is printed.

Usage:
...............
        status [<rsc>]
...............

[[cmdhelp_resource_start,start a resource]]
==== `start`

Start a resource by setting the `target-role` attribute. If there
are multiple meta attributes sets, the attribute is set in all of
them. If the resource is a group or a clone, all `target-role`
attributes are removed from the children resources.

Usage:
...............
        start <rsc>
...............

[[cmdhelp_resource_stop,stop a resource]]
==== `stop`

Stop a resource using the `target-role` attribute. If there
are multiple meta attributes sets, the attribute is set in all of
them. If the resource is a group or a clone, all `target-role`
attributes are removed from the children resources.

Usage:
...............
        stop <rsc>
...............

[[cmdhelp_resource_restart,restart a resource]]
==== `restart`

Restart a resource. This is essentially a shortcut for resource
stop followed by a start. The shell is first going to wait for
the stop to finish, that is for all resources to really stop, and
only then to order the start action. Due to this command
entailing a whole set of operations, informational messages are
printed to let the user see some progress.

Usage:
...............
        restart <rsc>
...............
Example:
...............
        # crm resource restart g_webserver
        INFO: ordering g_webserver to stop
        waiting for stop to finish .... done
        INFO: ordering g_webserver to start
        # 
...............

[[cmdhelp_resource_promote,promote a master-slave resource]]
==== `promote`

Promote a master-slave resource using the `target-role`
attribute.

Usage:
...............
        promote <rsc>
...............

[[cmdhelp_resource_demote,demote a master-slave resource]]
==== `demote`

Demote a master-slave resource using the `target-role`
attribute.

Usage:
...............
        demote <rsc>
...............

[[cmdhelp_resource_manage,put a resource into managed mode]]
==== `manage`

Manage a resource using the `is-managed` attribute. If there
are multiple meta attributes sets, the attribute is set in all of
them. If the resource is a group or a clone, all `is-managed`
attributes are removed from the children resources.

Usage:
...............
        manage <rsc>
...............

[[cmdhelp_resource_unmanage,put a resource into unmanaged mode]]
==== `unmanage`

Unmanage a resource using the `is-managed` attribute. If there
are multiple meta attributes sets, the attribute is set in all of
them. If the resource is a group or a clone, all `is-managed`
attributes are removed from the children resources.

Usage:
...............
        unmanage <rsc>
...............

[[cmdhelp_resource_migrate,migrate a resource to another node]]
==== `migrate` (`move`)

Migrate a resource to a different node. If node is left out, the
resource is migrated by creating a constraint which prevents it from
running on the current node. Additionally, you may specify a
lifetime for the constraint---once it expires, the location
constraint will no longer be active.

Usage:
...............
        migrate <rsc> [<node>] [<lifetime>] [force]
...............

[[cmdhelp_resource_unmigrate,unmigrate a resource to another node]]
==== `unmigrate` (`unmove`)

Remove the constraint generated by the previous migrate command.

Usage:
...............
        unmigrate <rsc>
...............

[[cmdhelp_resource_param,manage a parameter of a resource]]
==== `param`

Show/edit/delete a parameter of a resource.

Usage:
...............
        param <rsc> set <param> <value>
        param <rsc> delete <param>
        param <rsc> show <param>
...............
Example:
...............
        param ip_0 show ip
...............

[[cmdhelp_resource_meta,manage a meta attribute]]
==== `meta`

Show/edit/delete a meta attribute of a resource. Currently, all
meta attributes of a resource may be managed with other commands
such as `resource stop`.

Usage:
...............
        meta <rsc> set <attr> <value>
        meta <rsc> delete <attr>
        meta <rsc> show <attr>
...............
Example:
...............
        meta ip_0 set target-role stopped
...............

[[cmdhelp_resource_utilization,manage a utilization attribute]]
==== `utilization`

Show/edit/delete a utilization attribute of a resource. These
attributes describe hardware requirements. By setting the
`placement-strategy` cluster property appropriately, it is
possible then to distribute resources based on resource
requirements and node size. See also <<cmdhelp_node_utilization,node utilization attributes>>.

Usage:
...............
        utilization <rsc> set <attr> <value>
        utilization <rsc> delete <attr>
        utilization <rsc> show <attr>
...............
Example:
...............
        utilization xen1 set memory 4096
...............

[[cmdhelp_resource_failcount,manage failcounts]]
==== `failcount`

Show/edit/delete the failcount of a resource.

Usage:
...............
        failcount <rsc> set <node> <value>
        failcount <rsc> delete <node>
        failcount <rsc> show <node>
...............
Example:
...............
        failcount fs_0 delete node2
...............

[[cmdhelp_resource_cleanup,cleanup resource status]]
==== `cleanup`

Cleanup resource status. Typically done after the resource has
temporarily failed. If a node is omitted, cleanup on all nodes.
If there are many nodes, the command may take a while.

Usage:
...............
        cleanup <rsc> [<node>]
...............

[[cmdhelp_resource_refresh,refresh CIB from the LRM status]]
==== `refresh`

Refresh CIB from the LRM status.

Usage:
...............
        refresh [<node>]
...............

[[cmdhelp_resource_reprobe,probe for resources not started by the CRM]]
==== `reprobe`

Probe for resources not started by the CRM.

Usage:
...............
        reprobe [<node>]
...............

[[cmdhelp_node,Nodes management]]
=== `node`

Node management and status commands.

[[cmdhelp_node_status,show nodes' status]]
==== `status`

Show nodes' status. If the node parameter is omitted then all
nodes are shown.

Usage:
...............
        status [<node>]
...............

[[cmdhelp_node_show,show node]]
==== `show`

Show a node definition. If the node parameter is omitted then all
nodes are shown.

Usage:
...............
        show [<node>]
...............

[[cmdhelp_node_standby,put node into standby]]
==== `standby`

Set a node to standby status. The node parameter defaults to the
node where the command is run. Additionally, you may specify a
lifetime for the standby---if set to `reboot`, the node will be
back online once it reboots. `forever` will keep the node in
standby after reboot.

Usage:
...............
        standby [<node>] [<lifetime>]

        lifetime :: reboot | forever
...............

[[cmdhelp_node_online,set node online]]
==== `online`

Set a node to online status. The node parameter
defaults to the node where the command is run.

Usage:
...............
        online [<node>]
...............

[[cmdhelp_node_fence,fence node]]
==== `fence`

Make CRM fence a node. This functionality depends on stonith
resources capable of fencing the specified node. No such stonith
resources, no fencing will happen.

Usage:
...............
        fence <node>
...............

[[cmdhelp_node_clearstate,Clear node state]]
==== `clearnodestate`

Resets and clears the state of the specified node. This node is
afterwards assumed clean and offline. This command can be used to
manually confirm that a node has been fenced (e.g., powered off).

Be careful! This can cause data corruption if you confirm that a node is
down that is, in fact, not cleanly down - the cluster will proceed as if
the fence had succeeded, possibly starting resources multiple times.

Usage:
...............
        clearstate <node>
...............

[[cmdhelp_node_delete,delete node]]
==== `delete`

Delete a node. This command will remove the node from the CIB
and, in case the heartbeat stack is running, run hb_delnode too.

Usage:
...............
        delete <node>
...............

[[cmdhelp_node_attribute,manage attributes]]
==== `attribute`

Edit node attributes. This kind of attribute should refer to
relatively static properties, such as memory size.

Usage:
...............
        attribute <node> set <attr> <value>
        attribute <node> delete <attr>
        attribute <node> show <attr>
...............
Example:
...............
        attribute node_1 set memory_size 4096
...............

[[cmdhelp_node_utilization,manage utilization attributes]]
==== `utilization`

Edit node utilization attributes. These attributes describe
hardware characteristics as integer numbers such as memory size
or the number of CPUs. By setting the `placement-strategy`
cluster property appropriately, it is possible then to distribute
resources based on resource requirements and node size. See also
<<cmdhelp_resource_utilization,resource utilization attributes>>.

Usage:
...............
        utilization <node> set <attr> <value>
        utilization <node> delete <attr>
        utilization <node> show <attr>
...............
Examples:
...............
        utilization node_1 set memory 16384
        utilization node_1 show cpu
...............

[[cmdhelp_node_status-attr,manage status attributes]]
==== `status-attr`

Edit node attributes which are in the CIB status section, i.e.
attributes which hold properties of a more volatile nature. One
typical example is attribute generated by the `pingd` utility.

Usage:
...............
        status-attr <node> set <attr> <value>
        status-attr <node> delete <attr>
        status-attr <node> show <attr>
...............
Example:
...............
        status-attr node_1 show pingd
...............

[[cmdhelp_options,user preferences]]
=== `options`

The user may set various options for the crm shell itself.

[[cmdhelp_options_skill-level,set skill level]]
==== `skill-level`

Based on the skill-level setting, the user is allowed to use only
a subset of commands. There are three levels: operator,
administrator, and expert. The operator level allows only
commands at the `resource` and `node` levels, but not editing
or deleting resources. The administrator may do that and may also
configure the cluster at the `configure` level and manage the
shadow CIBs. The expert may do all.

Usage:
...............
        skill-level <level>
        
        level :: operator | administrator | expert
...............

[[cmdhelp_options_user,set the cluster user]]
==== `user`

Sufficient privileges are necessary in order to manage a
cluster: programs such as `crm_verify` or `crm_resource` and,
ultimately, `cibadmin` have to be run either as `root` or as the
CRM owner user (typically `hacluster`). You don't have to worry
about that if you run `crm` as `root`. A more secure way is to
run the program with your usual privileges, set this option to
the appropriate user (such as `hacluster`), and setup the
`sudoers` file.

Usage:
...............
        user system-user
...............
Example:
...............
        user hacluster
...............

[[cmdhelp_options_editor,set preferred editor program]]
==== `editor`

The `edit` command invokes an editor. Use this to specify your
preferred editor program. If not set, it will default to either
the value of the `EDITOR` environment variable or to one of the
standard UNIX editors (`vi`,`emacs`,`nano`).

Usage:
...............
        editor program
...............
Example:
...............
        editor vim
...............

[[cmdhelp_options_pager,set preferred pager program]]
==== `pager`

The `view` command displays text through a pager. Use this to
specify your preferred pager program. If not set, it will default
to either the value of the `PAGER` environment variable or to one
of the standard UNIX system pagers (`less`,`more`,`pg`).

[[cmdhelp_options_sort-elements,sort CIB elements]]
==== `sort-elements`

`crm` by default sorts CIB elements. If you want them appear in
the order they were created, set this option to `no`.

Usage:
...............
        sort-elements {yes|no}
...............
Example:
...............
        sort-elements no
...............

[[cmdhelp_options_wait,synchronous operation]]
==== `wait`

In normal operation, `crm` runs a command and gets back
immediately to process other commands or get input from the user.
With this option set to `yes` it will wait for the started
transition to finish. In interactive mode dots are printed to
indicate progress.

Usage:
...............
        wait {yes|no}
...............
Example:
...............
        wait yes
...............

[[cmdhelp_options_output,set output type]]
==== `output`

`crm` can adorn configurations in two ways: in color (similar to
for instance the `ls --color` command) and by showing keywords in
upper case. Possible values are `plain`, `color`, and
'uppercase'. It is possible to combine the latter two in order to
get an upper case xmass tree. Just set this option to
`color,uppercase`.

[[cmdhelp_options_colorscheme,set colors for output]]
==== `colorscheme`

With `output` set to `color`, a comma separated list of colors
from this option are used to emphasize:

- keywords
- object ids
- attribute names
- attribute values
- scores
- resource references

`crm` can show colors only if there is curses support for python
installed (usually provided by the `python-curses` package). The
colors are whatever is available in your terminal. Use `normal`
if you want to keep the default foreground color.

This user preference defaults to
`yellow,normal,cyan,red,green,magenta` which is good for
terminals with dark background. You may want to change the color
scheme and save it in the preferences file for other color
setups.

Example:
...............
    colorscheme yellow,normal,blue,red,green,magenta
...............

[[cmdhelp_options_check-frequency,when to perform semantic check]]
==== `check-frequency`

Semantic check of the CIB or elements modified or created may be
done on every configuration change (`always`), when verifying
(`on-verify`) or `never`. It is by default set to `always`.
Experts may want to change the setting to `on-verify`.

The checks require that resource agents are present. If they are
not installed at the configuration time set this preference to
`never`.

See <<SEMCHK,Configuration semantic checks>> for more details.

[[cmdhelp_options_check-mode,how to treat semantic errors]]
==== `check-mode`

Semantic check of the CIB or elements modified or created may be
done in the `strict` mode or in the `relaxed` mode. In the former
certain problems are treated as configuration errors. In the
`relaxed` mode all are treated as warnings. The default is `strict`.

See <<SEMCHK,Configuration semantic checks>> for more details.

[[cmdhelp_options_show,show current user preference]]
==== `show`

Display all current settings.

[[cmdhelp_options_save,save the user preferences to the rc file]]
==== `save`

Save current settings to the rc file (`$HOME/.crm.rc`). On
further `crm` runs, the rc file is automatically read and parsed.

[[cmdhelp_configure,CIB configuration]]
=== `configure`

This level enables all CIB object definition commands.

The configuration may be logically divided into four parts:
nodes, resources, constraints, and (cluster) properties and
attributes.  Each of these commands support one or more basic CIB
objects.

Nodes and attributes describing nodes are managed using the
`node` command.

Commands for resources are:

- `primitive`
- `monitor`
- `group`
- `clone`
- `ms`/`master` (master-slave)

There are three types of constraints:

- `location`
- `colocation`
- `order`

Finally, there are the cluster properties, resource meta
attributes defaults, and operations defaults. All are just a set
of attributes. These attributes are managed by the following
commands:

- `property`
- `rsc_defaults`
- `op_defaults`

In addition to the cluster configuration, the Access Control
Lists (ACL) can be setup to allow access to parts of the CIB for
users other than `root` and `hacluster`. The following commands
manage ACL:

- `user`
- `role`

The changes are applied to the current CIB only on ending the
configuration session or using the `commit` command.

Comments start with `#` in the first line. The comments are tied
to the element which follows. If the element moves, its comments
will follow.

[[cmdhelp_configure_node,define a cluster node]]
==== `node`

The node command describes a cluster node. Nodes in the CIB are
commonly created automatically by the CRM. Hence, you should not
need to deal with nodes unless you also want to define node
attributes. Note that it is also possible to manage node
attributes at the `node` level.

Usage:
...............
        node <uname>[:<type>]
          [attributes <param>=<value> [<param>=<value>...]]
          [utilization <param>=<value> [<param>=<value>...]]

        type :: normal | member | ping
...............
Example:
...............
        node node1
        node big_node attributes memory=64
...............

[[cmdhelp_configure_primitive,define a resource]]
==== `primitive`

The primitive command describes a resource. It may be referenced
only once in group, clone, or master-slave objects. If it's not
referenced, then it is placed as a single resource in the CIB.

Operations may be specified in three ways. "Anonymous" as a
simple list of "op" specifications. Use that if you don't want to
reference the set of operations elsewhere. That's by far the most
common way to define operations. If reusing operation sets is
desired, use the "operations" keyword along with the id to give
the operations set a name and the id-ref to reference another set
of operations.

Operation's attributes which are not recognized are saved as
instance attributes of that operation. A typical example is
`OCF_CHECK_LEVEL`.

For multistate resources, roles are specified as `role=<role>`.

Usage:
...............
        primitive <rsc> [<class>:[<provider>:]]<type>
          [params attr_list]
          [meta attr_list]
          [utilization attr_list]
          [operations id_spec]
            [op op_type [<attribute>=<value>...] ...]

        attr_list :: [$id=<id>] <attr>=<val> [<attr>=<val>...] | $id-ref=<id>
        id_spec :: $id=<id> | $id-ref=<id>
        op_type :: start | stop | monitor
...............
Example:
...............
        primitive apcfence stonith:apcsmart \
          params ttydev=/dev/ttyS0 hostlist="node1 node2" \
          op start timeout=60s \
          op monitor interval=30m timeout=60s

        primitive www8 apache \
          params configfile=/etc/apache/www8.conf \
          operations $id-ref=apache_ops

        primitive db0 mysql \
          params config=/etc/mysql/db0.conf \
          op monitor interval=60s \
          op monitor interval=300s OCF_CHECK_LEVEL=10

        primitive r0 ocf:linbit:drbd \
          params drbd_resource=r0 \
          op monitor role=Master interval=60s \
          op monitor role=Slave interval=300s
...............

[[cmdhelp_configure_monitor,add monitor operation to a primitive]]
==== `monitor`

Monitor is by far the most common operation. It is possible to
add it without editing the whole resource. Also, long primitive
definitions may be a bit uncluttered. In order to make this
command as concise as possible, less common operation attributes
are not available. If you need them, then use the `op` part of
the `primitive` command.

Usage:
...............
        monitor <rsc>[:<role>] <interval>[:<timeout>]
...............
Example:
...............
        monitor apcfence 60m:60s
...............

Note that after executing the command, the monitor operation may
be shown as part of the primitive definition.

[[cmdhelp_configure_group,define a group]]
==== `group`

The `group` command creates a group of resources.

Usage:
...............
        group <name> <rsc> [<rsc>...]
          [meta attr_list]
          [params attr_list]

        attr_list :: [$id=<id>] <attr>=<val> [<attr>=<val>...] | $id-ref=<id>
...............
Example:
...............
        group internal_www disk0 fs0 internal_ip apache \
          meta target_role=stopped
...............

[[cmdhelp_configure_clone,define a clone]]
==== `clone`

The `clone` command creates a resource clone. It may contain a
single primitive resource or one group of resources.

Usage:
...............
        clone <name> <rsc>
          [meta attr_list]
          [params attr_list]

        attr_list :: [$id=<id>] <attr>=<val> [<attr>=<val>...] | $id-ref=<id>
...............
Example:
...............
        clone cl_fence apc_1 \
          meta clone-node-max=1 globally-unique=false
...............

[[cmdhelp_configure_ms,define a master-slave resource]]
==== `ms` (`master`)

The `ms` command creates a master/slave resource type. It may contain a
single primitive resource or one group of resources.

Usage:
...............
        ms <name> <rsc>
          [meta attr_list]
          [params attr_list]

        attr_list :: [$id=<id>] <attr>=<val> [<attr>=<val>...] | $id-ref=<id>
...............
Example:
...............
        ms disk1 drbd1 \
          meta notify=true globally-unique=false
...............

.Note on `id-ref` usage
****************************
Instance or meta attributes (`params` and `meta`) may contain
a reference to another set of attributes. In that case, no other
attributes are allowed. Since attribute sets' ids, though they do
exist, are not shown in the `crm`, it is also possible to
reference an object instead of an attribute set. `crm` will
automatically replace such a reference with the right id:

...............
    crm(live)configure# primitive a2 www-2 meta $id-ref=a1
    crm(live)configure# show a2
    primitive a2 ocf:heartbeat:apache \
        meta $id-ref="a1-meta_attributes"
        [...]
...............
It is advisable to give meaningful names to attribute sets which
are going to be referenced.
****************************

[[cmdhelp_configure_location,a location preference]]
==== `location`

`location` defines the preference of nodes for the given
resource. The location constraints consist of one or more rules
which specify a score to be awarded if the rule matches.

Usage:
...............
        location <id> <rsc> {node_pref|rules}

        node_pref :: <score>: <node>

        rules ::
          rule [id_spec] [$role=<role>] <score>: <expression>
          [rule [id_spec] [$role=<role>] <score>: <expression> ...]

        id_spec :: $id=<id> | $id-ref=<id>
        score :: <number> | <attribute> | [-]inf
        expression :: <simple_exp> [bool_op <simple_exp> ...]
        bool_op :: or | and
        simple_exp :: <attribute> [type:]<binary_op> <value>
                      | <unary_op> <attribute>
                      | date <date_expr>
        type :: string | version | number
        binary_op :: lt | gt | lte | gte | eq | ne
        unary_op :: defined | not_defined

        date_expr :: lt <end>
                     | gt <start>
                     | in_range start=<start> end=<end>
                     | in_range start=<start> <duration>
                     | date_spec <date_spec>
        duration|date_spec ::
                     hours=<value>
                     | monthdays=<value>
                     | weekdays=<value>
                     | yearsdays=<value>
                     | months=<value>
                     | weeks=<value>
                     | years=<value>
                     | weekyears=<value>
                     | moon=<value>
...............
Examples:
...............
        location conn_1 internal_www 100: node1

        location conn_1 internal_www \
          rule 50: #uname eq node1 \
          rule pingd: defined pingd

        location conn_2 dummy_float \
          rule -inf: not_defined pingd or pingd number:lte 0
...............

[[cmdhelp_configure_colocation,colocate resources]]
==== `colocation` (`collocation`)

This constraint expresses the placement relation between two
or more resources. If there are more than two resources, then the
constraint is called a resource set. Collocation resource sets have
an extra attribute to allow for sets of resources which don't depend
on each other in terms of state. The shell syntax for such sets is
to put resources in parentheses.

Usage:
...............
        colocation <id> <score>: <rsc>[:<role>] <rsc>[:<role>] ...
...............
Example:
...............
        colocation dummy_and_apache -inf: apache dummy
        colocation c1 inf: A ( B C )
...............

[[cmdhelp_configure_order,order resources]]
==== `order`

This constraint expresses the order of actions on two resources
or more resources. If there are more than two resources, then the
constraint is called a resource set. Ordered resource sets have an
extra attribute to allow for sets of resources whose actions may run
in parallel. The shell syntax for such sets is to put resources in
parentheses.

Usage:
...............
        order <id> score-type: <rsc>[:<action>] <rsc>[:<action>] ...
          [symmetrical=<bool>]

        score-type :: advisory | mandatory | <score>
...............
Example:
...............
        order c_apache_1 mandatory: apache:start ip_1
        order o1 inf: A ( B C )
...............

[[cmdhelp_configure_property,set a cluster property]]
==== `property`

Set the cluster (`crm_config`) options.

Usage:
...............
        property [$id=<set_id>] <option>=<value> [<option>=<value> ...]
...............
Example:
...............
        property stonith-enabled=true
...............

[[cmdhelp_configure_rsc_defaults,set resource defaults]]
==== `rsc_defaults`

Set defaults for the resource meta attributes.

Usage:
...............
        rsc_defaults [$id=<set_id>] <option>=<value> [<option>=<value> ...]
...............
Example:
...............
        rsc_defaults failure-timeout=3m
...............

[[cmdhelp_configure_role,define role access rights]]
==== `role`

An ACL role is a set of rules which describe access rights to
CIB. Rules consist of an access right `read`, `write`, or `deny`
and a specification denoting part of the configuration to which
the access right applies. The specification can be an XPath or a
combination of tag and id references. If an attribute is
appended, then the specification applies only to that attribute
of the matching element.

There is a number of shortcuts for XPath specifications. The
`meta,` `params`, and `utilization` shortcuts reference resource
meta attributes, parameters, and utilization respectively. The
`location` may be used to specify location constraints most of
the time to allow resource `move` and `unmove` commands. The
`property` references cluster properties. The `node` allows
reading node attributes. `nodeattr` and `nodeutil` reference node
attributes and node capacity (utilization). The `status` shortcut
references the whole status section of the CIB. Read access to
status is necessary for various monitoring tools such as
`crm_mon(8)` (aka `crm status`).

Usage:
...............
        role <role-id> rule [rule ...]

        rule :: acl-right cib-spec [attribute:<attribute>]

        acl-right :: read | write | deny

        cib-spec :: xpath-spec | tag-ref-spec
        xpath-spec :: xpath:<xpath> | shortcut
        tag-ref-spec :: tag:<tag> | ref:<id> | tag:<tag> ref:<id>

        shortcut :: meta:<rsc>[:<attr>]
                    params:<rsc>[:<attr>]
                    utilization:<rsc>
                    location:<rsc>
                    property[:<attr>]
                    node[:<node>]
                    nodeattr[:<attr>]
                    nodeutil[:<node>]
                    status
...............
Example:
...............
        role app1_admin \
            write meta:app1:target-role \
            write meta:app1:is-managed \
            write location:app1 \
            read ref:app1
...............

[[cmdhelp_configure_user,define user access rights]]
==== `user`

Users which normally cannot view or manage cluster configuration
can be allowed access to parts of the CIB. The access is defined
by a set of `read`, `write`, and `deny` rules as in role
definitions or by referencing roles. The latter is considered
best practice.

Usage:
...............
        user <uid> {roles|rules}

        roles :: role:<role-ref> [role:<role-ref> ...]
        rules :: rule [rule ...]
...............
Example:
...............
        user joe \
            role:app1_admin \
            role:read_all
...............

[[cmdhelp_configure_op_defaults,set resource operations defaults]]
==== `op_defaults`

Set defaults for the operations meta attributes.

Usage:
...............
        op_defaults [$id=<set_id>] <option>=<value> [<option>=<value> ...]
...............
Example:
...............
        op_defaults record-pending=true
...............

[[cmdhelp_configure_show,display CIB objects]]
==== `show`

The `show` command displays objects. It may display all objects
or a set of objects. The user may also choose to see only objects
which were changed.
Optionally, the XML code may be displayed instead of the CLI
representation.

Usage:
...............
        show [xml] [<id> ...]
        show [xml] changed
...............

[[cmdhelp_configure_edit,edit CIB objects]]
==== `edit`

This command invokes the editor with the object description. As
with the `show` command, the user may choose to edit all objects
or a set of objects.

If the user insists, he or she may edit the XML edition of the
object. If you do that, don't modify any id attributes.

Usage:
...............
        edit [xml] [<id> ...]
        edit [xml] changed
...............

[[cmdhelp_configure_filter,filter CIB objects]]
==== `filter`

This command filters the given CIB elements through an external
program. The program should accept input on `stdin` and send
output to `stdout` (the standard UNIX filter conventions). As
with the `show` command, the user may choose to filter all or
just a subset of elements.

It is possible to filter the XML representation of objects, but
probably not as useful as the configuration language. The
presentation is somewhat different from what would be displayed
by the `show` command---each element is shown on a single, i.e.
there are no backslashes and no other embelishments.

Don't forget to put quotes around the filter if it contains
spaces. Further, if you use this in a single-shot mode (from the
`sh` command line), protect all quotes with a backslash.
Otherwise, they won't reach the `crm` shell parser.

Usage:
...............
        filter <prog> [xml] [<id> ...]
        filter <prog> [xml] changed
...............
Examples:
...............
        filter "sed '/^primitive/s/target-role=[^ ]*//'"
...............

[[cmdhelp_configure_delete,delete CIB objects]]
==== `delete`

Delete one or more objects. If an object to be deleted belongs to
a container object, such as a group, and it is the only resource
in that container, then the container is deleted as well. Any
related constraints are removed as well.

Usage:
...............
        delete <id> [<id>...]
...............

[[cmdhelp_configure_default-timeouts,set timeouts for operations to minimums from the meta-data]]
==== `default-timeouts`

This command takes the timeouts from the actions section of the
resource agent meta-data and sets them for the operations of the
primitive.

Usage:
...............
        default-timeouts <id> [<id>...]
...............

.Note on `default-timeouts`
****************************
You may be happy using this, but your applications may not. And
it will tell you so at the worst possible moment. You have been
warned.
****************************

[[cmdhelp_configure_rename,rename a CIB object]]
==== `rename`

Rename an object. It is recommended to use this command to rename
a resource, because it will take care of updating all related
constraints and a parent resource. Changing ids with the edit
command won't have the same effect.

If you want to rename a resource, it must be in the stopped state.

Usage:
...............
        rename <old_id> <new_id>
...............

[[cmdhelp_configure_refresh,refresh from CIB]]
==== `refresh`

Refresh the internal structures from the CIB. All changes made
during this session are lost.

Usage:
...............
        refresh
...............

[[cmdhelp_configure_erase,erase the CIB]]
==== `erase`

The `erase` clears all configuration. Apart from nodes. To remove
nodes, you have to specify an additional keyword `nodes`.

Note that removing nodes from the live cluster may have some
strange/interesting/unwelcome effects.

Usage:
...............
        erase [nodes]
...............

[[cmdhelp_configure_ptest,show cluster actions if changes were committed]]
==== `ptest`

Show PE (Policy Engine) motions using `ptest(8)`.

A CIB is constructed using the current user edited configuration
and the status from the running CIB. The resulting CIB is run
through `ptest` to show changes which would happen if the
configuration is committed.

The status section may be loaded from another source and modified
using the <<cmdhelp_cibstatus,`cibstatus`>> level commands. In that case, the
`ptest` command will issue a message informing the user that the
Policy Engine graph is not calculated based on the current status
section and therefore won't show what would happen to the
running but some imaginary cluster.

If you have graphviz installed and X11 session, `dotty(1)` is run
to display the changes graphically.

Add a string of `v` characters to increase verbosity. `ptest`
can also show allocation scores. `utilization` turns on
information about the remaining capacity of nodes. With the
`actions` option, `ptest` will print all resource actions.

Usage:
...............
        ptest [nograph] [v...] [scores] [actions] [utilization]
...............
Examples:
...............
        ptest scores
        ptest vvvvv
...............

[[cmdhelp_configure_cib,CIB shadow management]]
=== `cib` (shadow CIBs)

This level is for management of shadow CIBs. It is available at
the `configure` level to enable saving intermediate changes to a
shadow CIB instead of to the live cluster. This short excerpt
shows how:
...............
    crm(live)configure# cib new test-2
    INFO: test-2 shadow CIB created
    crm(test-2)configure# commit
...............
Note how the current CIB in the prompt changed from `live` to
`test-2` after issuing the `cib new` command. See also the
<<cmdhelp_cib,CIB shadow management>> for more information.

[[cmdhelp_configure_cibstatus,CIB status management and editing]]
==== `cibstatus`

Enter edit and manage the CIB status section level. See the
<<cmdhelp_cibstatus,CIB status management section>>.

[[cmdhelp_configure_template,edit and import a configuration from a template]]
==== `template`

The specified template is loaded into the editor. It's up to the
user to make a good CRM configuration out of it. See also the
<<cmdhelp_template,template section>>.

Usage:
...............
        template [xml] url
...............
Example:
...............
        template two-apaches.txt
...............

[[cmdhelp_configure_commit,commit the changes to the CIB]]
==== `commit`

Commit the current configuration to the CIB in use. As noted
elsewhere, commands in a configure session don't have immediate
effect on the CIB. All changes are applied at one point in time,
either using `commit` or when the user leaves the configure
level. In case the CIB in use changed in the meantime, presumably
by somebody else, the crm shell will refuse to apply the changes.
If you know that it's fine to still apply them add `force`.

Usage:
...............
        commit [force]
...............

[[cmdhelp_configure_verify,verify the CIB with crm_verify]]
==== `verify`

Verify the contents of the CIB which would be committed.

Usage:
...............
        verify
...............

[[cmdhelp_configure_upgrade,upgrade the CIB to version 1.0]]
==== `upgrade`

If you get the `CIB not supported` error, which typically means
that the current CIB version is coming from the older release,
you may try to upgrade it to the latest revision. The command
to perform the upgrade is:
...............
    # cibadmin --upgrade --force
...............

If we don't recognize the current CIB as the old one, but you're
sure that it is, you may force the command.

Usage:
...............
        upgrade [force]
...............

[[cmdhelp_configure_save,save the CIB to a file]]
==== `save`

Save the configuration of the current level to a file.
Optionally, as XML.

Usage:
...............
        save [xml] <file>
...............
Example:
...............
        save myfirstcib.txt
...............

[[cmdhelp_configure_load,import the CIB from a file]]
==== `load`

Load a part of configuration (or all of it) from a local file or
a network URL. The `replace` method replaces the current
configuration with the one from the source. The `update` tries to
import the contents into the current configuration.
The file may be a CLI file or an XML file.

Usage:
...............
        load [xml] <method> URL

        method :: replace | update
...............
Example:
...............
        load xml update myfirstcib.xml
        load xml replace http://storage.big.com/cibs/bigcib.xml
...............

[[cmdhelp_configure_xml,raw xml]]
==== `xml`

Even though we promissed no xml, it may happen, but hopefully
very very seldom, that an element from the CIB cannot be rendered
in the configuration language. In that case, the element will be
shown as raw xml, prefixed by this command. That element can then
be edited like any other. If the shell finds out that after the
change it can digest it, then it is going to be converted into
the normal configuration language. Otherwise, there is no need to
use `xml` for configuration.

Usage:
...............
        xml <xml>
...............

[[cmdhelp_template,edit and import a configuration from a template]]
=== `template`

User may be assisted in the cluster configuration by templates
prepared in advance. Templates consist of a typical ready
configuration which may be edited to suit particular user needs.

This command enters a template level where additional commands
for configuration/template management are available.

[[cmdhelp_template_new,create a new configuration from templates]]
==== `new`

Create a new configuration from one or more templates. Note that
configurations and templates are kept in different places, so it
is possible to have a configuration name equal a template name.

If you already know which parameters are required, you can set
them directly on the command line.

The parameter name `id` is set by default to the name of the
configuration.

Usage:
...............
        new <config> <template> [<template> ...] [params name=value ...]"
...............
Examples:
...............
        new vip virtual-ip
        new bigfs ocfs2 params device=/dev/sdx8 directory=/bigfs
...............

[[cmdhelp_template_load,load a configuration]]
==== `load`

Load an existing configuration. Further `edit`, `show`, and
`apply` commands will refer to this configuration.

Usage:
...............
        load <config>
...............

[[cmdhelp_template_edit,edit a configuration]]
==== `edit`

Edit current or given configuration using your favourite editor.

Usage:
...............
        edit [<config>]
...............

[[cmdhelp_template_delete,delete a configuration]]
==== `delete`

Remove a configuration. The loaded (active) configuration may be
removed by force.

Usage:
...............
        delete <config> [force]
...............

[[cmdhelp_template_list,list configurations/templates]]
==== `list`

List existing configurations or templates.

Usage:
...............
        list [templates]
...............

[[cmdhelp_template_apply,process and apply the current configuration to the current CIB]]
==== `apply`

Copy the current or given configuration to the current CIB. By
default, the CIB is replaced, unless the method is set to
"update".

Usage:
...............
        apply [<method>] [<config>]

        method :: replace | update
...............

[[cmdhelp_template_show,show the processed configuration]]
==== `show`

Process the current or given configuration and display the result.

Usage:
...............
        show [<config>]
...............

[[cmdhelp_cibstatus,CIB status management and editing]]
=== `cibstatus`

The `status` section of the CIB keeps the current status of nodes
and resources. It is modified _only_ on events, i.e. when some
resource operation is run or node status changes. For obvious
reasons, the CRM has no user interface with which it is possible
to affect the status section. From the user's point of view, the
status section is essentially a read-only part of the CIB. The
current status is never even written to disk, though it is
available in the PE (Policy Engine) input files which represent
the history of cluster motions. The current status may be read
using the `cibadmin -Q` command.

It may sometimes be of interest to see how status changes would
affect the Policy Engine. The set of `cibstatus` level commands
allow the user to load status sections from various sources and
then insert or modify resource operations or change nodes' state.

The effect of those changes may then be observed by running the
<<cmdhelp_configure_ptest,`ptest`>> command at the `configure` level
or `simulate` and `run` commands at this level. The `ptest`
runs with the user edited CIB whereas the latter two commands
run with the CIB which was loaded along with the status section.

The `simulate` and `run` commands as well as all status
modification commands are implemented using `crm_simulate(8)`.

[[cmdhelp_cibstatus_load,load the CIB status section]]
==== `load`

Load a status section from a file, a shadow CIB, or the running
cluster. By default, the current (`live`) status section is
modified. Note that if the `live` status section is modified it
is not going to be updated if the cluster status changes, because
that would overwrite the user changes. To make `crm` drop changes
and resume use of the running cluster status, run `load live`.

All CIB shadow configurations contain the status section which is
a snapshot of the status section taken at the time the shadow was
created. Obviously, this status section doesn't have much to do
with the running cluster status, unless the shadow CIB has just
been created. Therefore, the `ptest` command by default uses the
running cluster status section.

Usage:
...............
        load {<file>|shadow:<cib>|live}
...............
Example:
...............
        load bug-12299.xml
        load shadow:test1
...............

[[cmdhelp_cibstatus_save,save the CIB status section]]
==== `save`

The current internal status section with whatever modifications
were performed can be saved to a file or shadow CIB.

If the file exists and contains a complete CIB, only the status
section is going to be replaced and the rest of the CIB will
remain intact. Otherwise, the current user edited configuration
is saved along with the status section.

Note that all modifications are saved in the source file as soon
as they are run.

Usage:
...............
        save [<file>|shadow:<cib>]
...............
Example:
...............
        save bug-12299.xml
...............

[[cmdhelp_cibstatus_origin,display origin of the CIB status section]]
==== `origin`

Show the origin of the status section currently in use. This
essentially shows the latest `load` argument.

Usage:
...............
        origin
...............

[[cmdhelp_cibstatus_show,show CIB status section]]
==== `show`

Show the current status section in the XML format. Brace yourself
for some unreadable output. Add `changed` option to get a human
readable output of all changes.

Usage:
...............
        show [changed]
...............

[[cmdhelp_cibstatus_node,change node status]]
==== `node`

Change the node status. It is possible to throw a node out of
the cluster, make it a member, or set its state to unclean.

`online`:: Set the `node_state` `crmd` attribute to `online`
and the `expected` and `join` attributes to `member`. The effect
is that the node becomes a cluster member.

`offline`:: Set the `node_state` `crmd` attribute to `offline`
and the `expected` attribute to empty. This makes the node
cleanly removed from the cluster.

`unclean`:: Set the `node_state` `crmd` attribute to `offline`
and the `expected` attribute to `member`. In this case the node
has unexpectedly disappeared.

Usage:
...............
        node <node> {online|offline|unclean}
...............
Example:
...............
        node xen-b unclean
...............

[[cmdhelp_cibstatus_op,edit outcome of a resource operation]]
==== `op`

Edit the outcome of a resource operation. This way you can
tell CRM that it ran an operation and that the resource agent
returned certain exit code. It is also possible to change the
operation's status. In case the operation status is set to
something other than `done`, the exit code is effectively
ignored.

Usage:
...............
        op <operation> <resource> <exit_code> [<op_status>] [<node>]

        operation :: probe | monitor[:<n>] | start | stop |
           promote | demote | notify | migrate_to | migrate_from
        exit_code :: <rc> | success | generic | args | 
           unimplemented | perm | installed | configured | not_running | 
           master | failed_master
        op_status :: pending | done | cancelled | timeout | notsupported | error

        n :: the monitor interval in seconds; if omitted, the first
           recurring operation is referenced
        rc :: numeric exit code in range 0..9
...............
Example:
...............
        op start d1 xen-b generic
        op start d1 xen-b 1
        op monitor d1 xen-b not_running
        op stop d1 xen-b 0 timeout
...............

[[cmdhelp_cibstatus_quorum,set the quorum]]
==== `quorum`

Set the quorum value.

Usage:
...............
        quorum <bool>
...............
Example:
...............
        quorum false
...............

[[cmdhelp_cibstatus_run,run policy engine]]
==== `run`

Run the policy engine with the edited status section.

Add a string of `v` characters to increase verbosity. Specify
`scores` to see allocation scores also. `utilization` turns on
information about the remaining capacity of nodes.

If you have graphviz installed and X11 session, `dotty(1)` is run
to display the changes graphically.

Usage:
...............
        run [nograph] [v...] [scores] [utilization]
...............
Example:
...............
        run
...............

[[cmdhelp_cibstatus_simulate,simulate cluster transition]]
==== `simulate`

Run the policy engine with the edited status section and simulate
the transition.

Add a string of `v` characters to increase verbosity. Specify
`scores` to see allocation scores also. `utilization` turns on
information about the remaining capacity of nodes.

If you have graphviz installed and X11 session, `dotty(1)` is run
to display the changes graphically.

Usage:
...............
        simulate [nograph] [v...] [scores] [utilization]
...............
Example:
...............
        simulate
...............

=== `end` (`cd`, `up`)

The `end` command ends the current level and the user moves to
the parent level. This command is available everywhere.

Usage:
...............
        end
...............

=== `help`

The `help` command prints help for the current level or for the
specified topic (command). This command is available everywhere.

Usage:
...............
        help [<topic>]
...............

=== `quit` (`exit`, `bye`)

Leave the program.

BUGS
----
Even though all sensible configurations (and most of those that
are not) are going to be supported by the crm shell, I suspect
that it may still happen that certain XML constructs may confuse
the tool. When that happens, please file a bug report.

The crm shell will not try to update the objects it does not
understand. Of course, it is always possible to edit such objects
in the XML format.

AUTHOR
------
Written by Dejan Muhamedagic, <dejan@suse.de>

SEE ALSO
--------
crm_resource(8), crm_attribute(8), crm_mon(8), cib_shadow(8),
ptest(8), dotty(1), crm_simulate(8), cibadmin(8)


COPYING
-------
Copyright \(C) 2008-2011 Dejan Muhamedagic. Free use of this
software is granted under the terms of the GNU General Public License (GPL).

//////////////////////
 vim:ts=4:sw=4:expandtab:
//////////////////////
