clockwork

NAME

clockwork - collection daemon for the Habitat suite  

SYNTAX

clockwork [-c <purl>] [-C <cfcmd>] [-e <fmt>] [-dDhsfv] [-j <stdjob> | -J <jobrt>]  

DESCRIPTION

Clockwork is the local collection agent for Habitat. It runs as a daemon process on each machine being monitored and is designed to carry out data collection, log file monitoring, data-driven actions and the distribution of collected data.

The default jobs are to collect system, network, storage, uptime and some busy process statistics on the local machine and make them available in a standard place. The collection of process data and file monitoring is available by configuring the jobs that drive clockwork. Configuration can be carried out at a local, regional and global level to allow deligation. One public and many private instances of clockwork can exist on a single machine, allowing individual users to carry out custom data collection Data is normally held in ring buffers or queues on the local machine's storage using datastores held to be self contained and scalable. Periodic replication of data rings to a repository is used for archiving and may be done in reverse for central data transmission.  

OPTIONS

-c <purl>

Append user configuration data from the route <purl>, rather than the default file ~/.habrc.

-C <cfcmd>

Append a list of configuration directives from <cfcmd>, separated by semicolons.

-d

Place clockwork in diagnostic mode, giving an additional level of logging and sending the text to stderr rather than the default or configured destinations. In daemon mode, will send output to the controlling terminal.

-D

Place clockwork in debug mode. As -d above but generating a great deal more information, designed to be used in conjunction with the source code. Also overrides normal outputs and will send the text to stderr. In daemon mode, will send output to the controlling terminal.

-e <fmt>

Change the logging output to one of eight preset alternative formats, some showing additional information. <fmt> must be 0-7. See LOGGING below.

-h

Print a help message to stdout and exit

-v

Print the version to stdout and exit

-j <stdjob>

Select from standard job tables, allowing different modes or behaviour to be selected easily from known good configurations. See COLLECTION MODES below for values and description of <stdjob>.

-J <jobrt>

Override standard job table with a private one provided by the route <jobrt>. Clockwork will not daemonise, run a data service or take an exclusive system lock (there can only be one public clockwork instance). Implies -s and alters the logging output to stderr, unless overridden with the range of elog configuration directives.

-f

Run in the foreground and don't daemonise

-s

Disable the public data service from being run, but will continue to save data as dictated by configuration.

 

COLLECTION MODES

default

Default mode. One minute samples recorded to disk, deriving three sets of averages: 4h@1m, 1d@5m, 7d@15m, 1mo@1h

 

DEFAULTS

When clockwork starts it reads $HAB/etc/habitat.conf and ~/.habrc for configuration data (see CONFIGURATION for more details). Unless overridden, clockwork will then look for its jobs inside the default public datastore for that machine, held in $HAB/var/<hostname>.grs (the route address is grs:$HAB/var/<hostname>.grs,jobs,0, see below for an explanation). If it does not find the jobs, clockwork bootstaps itself by copying a default job template from the file $HAB/lib/clockwork.jobs into the public datastore and then carries on using the datastore version.

The default jobs run system, network and storage data gathering probes every 60 seconds. It saves results to the public datastore using the template route grs:$HAB/var/<hostname>.grs,<jobname>,60 and errors to grs:$HAB/var/<hostname>.grs,err_<jobname>,60

All other errors are placed in grs:$HAB/var/<hostname>.grs,log,0  

ROUTES

To move data around in clockwork, an enhanced URL is used as a form of addressing and is called a 'route' (also known as a pseudo-url or p-url in documentation). The format is <driver>:<address>, where driver must be one of the following:-

file: fileov:

reads and write to paths on the filesystem. The format is file:<file path>, which will always append text to the file when writing. The fileov: driver will overwrite text when first writing and is suitable for configuration files or states.

http: https:

reads and writes using HTTP or HTTPS to a network address. The address is the server name and object name as a normal URL convention.

grs:

read and writes to a ring store, the primary local storage mechanism. Tabular data is stored in a time series in a queue or ring buffer structure. Multiple rings of data can be stored in a single ringstore file, using different names and durations.

sqlrs:

reads and writes tabular data to a remote repository service using the SQL Ringstore method, which is implemented over the HTTP protocol. Harvest provides repository services. Stores tabular data in a time series, addressed by host name, ring name and duration. Data is stored in a wueue or ring buffer storage.

 

CONFIGURATION

By default, clockwork will collect system, network and storage statistics for the system on which it runs. All the data is read and written from a local datastore, apart from configuration items which come from external sources. These external configuration sources govern the operation of all the habitat commands and applications.

Refer to the habconf(5) man page for more details.  

JOB DEFINITIONS

Jobs are defined in a multi columned text format, headed by the magic string 'job 1'. Comments may appear anywhere, starting with '#' and running to the end of the line.

Each job is defined on a single line containing 11 arguments, which in order are:-

1. start

when to start the job, in seconds from the starting of clockwork

2. period

how often to repeat the job, in seconds

3. phase

not yet implemented

4. count

how many times the job should be run, with 0 repeating forever

5. name

name of the job

6. requester

who requested the job, by convention the email address

7. results

the route where results should be sent

8. errors

the route where errors should be sent

9. nslots

the number of slots created in the 'results' and 'errors' routes, if applicable (applies to timestore and tablestore).

10.method

the job method

11.command

the arguments given to each method

See the habmeth(1) manpage for details of the possible methods that may be specified and the commands that can accept.  

DATA ORGANISATION

Data is stored in sequences of tabular information. All data has an ordered independently of time, allowing multiple separate samples that share the same time interval. This data is stored in a ringbuffer, which allows data to grow to a certain number of samples before the oldest are removed and their space recycled. Throughout the documentation, each collection of samples is known as a ring, and may be configured to be a simple queue, where data management is left up to administrators.

To limit the amount of storage used, data in a ring can be sampled periodically to form new summary data and stored in a new ring with a different period. In habitat, this is known as cascading and takes place on all the default collection rings. Several levels of cascading can take place over several new rings, This allows summaries at different frequencies to be collected and tuned to local requirements.

See the habmeth(1) man page for more information about the cascade method.  

DATA REPLICATION

Any ring of information can be sent to or from the repository at known intervals, allowing a deterministic way of updating both repository and collection agent.

This is implemented as a regular job which runs the replicate method. Data for the method is provided by configuration parameters which can be set and altered in the organisation. Thus the replication job does not normally need to be altered to change the behaviour.

See the habmeth(1) man page for the replicate method and the formation of the configuration data.  

LOGGING

Clockwork and the probes that provide data, also generate information and error messages. By convention, these are stored in the route specification ts:$hab/var/<host>.ts,log The convention for probes is to store their errors in ts:$HAB/var/<host>.ts,e.<jobname>.

To override the logging location, use the range of elog configuration directives, or rely on the options -d, -D, -j, which will alter the location to stderr as a side effect. See habconf(5) for details. Probe logging is configurable for each job in the job table.

The logging format can be customised using one of a set of configuration directives (see habconf(5)). For convenience, the -e flag specifies one of eight preconfigured text formats that will be sent to the configured location:-

0

all 17 possible log variables

1

severity character & text

2

severity & text

3

severity, text, file, function & line

4

long severity, short time, short program name, file, function, line & text

5

date time, severity, long program name, process id, file, function, line, origin, code & text

6

unix ctime, seconds since 1970, short program name, process id, thread id, file, function, line, origin, code & text

7

severity, file, line, origin, code, text

 

FILES

If run from a single directory $HAB:-
$HAB/bin/clockwork
$HAB/var/<hostname>.grs, $HAB/lib/clockwork.jobs
/tmp/clockwork.run
~/.habrc, $HAB/etc/habitat.conf

If run from installed Linux locations:-
/usr/bin/habitat
/var/lib/habitat/<hostname>.grs, /usr/lib/habitat/clockwork.jobs
/var/lib/habitat/clockwork.run
~/.habrc, /etc/habitat.conf  

ENVIRONMENT VARIABLES

 

EXAMPLES

Type the following to run clockwork in the standard way. This assumes it is providing public data using the standard job file, storing in a known place and using the standard network port for the data service.

clockwork

On a more secure system, you can prevent the data service from being started

clockwork -s

Alternatively you can run it in a private mode by specifying '-J' and a replacement job file.

clockwork -J "file:mywork.job"  

AUTHORS

Nigel Stuckey <[email protected]>  

SEE ALSO

killclock(8), statclock(8), habedit(8), habrep(8), habconf(5), myhabitat(1), habget(1), habput(1), habrs(1), habprobe(1), habmeth(1)

---

 

Index

NAME

SYNTAX

DESCRIPTION

OPTIONS

COLLECTION MODES

DEFAULTS

ROUTES

CONFIGURATION

JOB DEFINITIONS

DATA ORGANISATION

DATA REPLICATION

LOGGING

FILES

ENVIRONMENT VARIABLES

EXAMPLES

AUTHORS

SEE ALSO