Cleo Log Warehouse (CLW)

The Cleo Log Warehouse (CLW) is a data retention utility that can be used with a Clarify (and in some cases Harmony) server. It supports the offline storage, reporting, and analysis of log data, as well as the retention of large volumes of log data for long periods of time, which helps eliminate the impact of reporting activity on your production server.

Benefits include:

  • Operating and maintaining a leaner production server.
  • Reducing the required database and disk size.
  • It supports data retention requirements and good disaster recovery practices.
Note: CLW is a not a mirrored image of your production server environment; it is a separate server install that provides a holding area for processed data. CLW functionality includes batch processing scripts which facilitate the movement of data from you production server to the log warehouse server.

Typical Business Scenario

Your data retention policy requires you to keep a year’s worth of data, but you certainly don’t want to store that amount of data on your production server. A CLW implementation allows you to archive and store data in the separate CLW server and purge older data from the production server, thereby maintaining as lean an environment as possible.

You could schedule the data archiving and purging as part of a nightly process where data from 3 days ago is archived (one day is archived) while data older than 7 days is purged; this would allow a safety gap should a system issue or problem occur with the archive process. You can view and query the data in the CLW server using the Cleo Clarify Studio or Cleo Dashboards.

Process overview

Some initial setup and configuration steps are required before using the log warehouse. This includes a separate Clarify Server install, creation of directories to facilitate data archiving and un-archiving, and the installation of the CLW utility. Additional steps are required to schedule the archiving and un-archiving, as well as the setup of data purging from the Admin Console.

Note: If Harmony-specific database tables are included in the archive process (an reside within the Clarify Postgres database), then certain archive configuration steps may be required in Harmony. See the Harmony User Guide for archiving information.

Once set up and executed, the CLW transfers specific log record data and transactional data from a Clarify server to a CLW server. Specifically, CLW takes data from the production server, archives it into a .zip file, and places it into a directory; the .zip file must then be moved to the CLW Server and unarchived; any new records will be inserted into CLW server's database and data is placed in the storage manager directory.

The archive process should be executed on a schedule defined by the user. The archive and un-archive utilities are executed using batch files/shell scripts; the Business Process RunOsCommand task can be used to run these scripts.
Note: The archive process does not manipulate actual database tables on your production server; it only selects data from its tables. Only new records are inserted in the CLW server database during the un-archive process, i.e. no updates or deletes.

Log reports will be generated and made available for viewing. These logs show archived data (data copied from the production server) and unarchived data (data copied to the CLW server). Both the Clarify Studio/Admin Console perspective's Auditor view and the Cleo Dashboards can be used to inspect this data.