A standard practice that we all should be putting in place even before we go to production to ensure that we actually understand what needs to be backed up and how to put a process in place. There are two types here, the first being a full offline backup, offline here means the UCD server is not running, the second is an incremental backup. One key to how to design your backup plan is understanding your Service Level Agreement(SLA) and time to recover, what amount of data loss can you accept.
For a full backup, the process involves stopping the UrbanCode servers and taking a complete backup of both the database and the relevant filesystems. This provides us a simple restore point to recover from a catastrophic failure of the production system. The challenge here is that this requires an outage to complete, which is not something that can easily be taken for the servers once they are in full use. The expectation is that this is done on a regular basis, I would recommend that we plan on doing this in a monthly outage window, probably the same one you use to apply upgrades or do OS patching on the servers.
Incremental backups are another part of the solution here, we expect that we are able to do this process while the servers are online. All UrbanCode applications are database centric so while the filesystems are important we are most concerned with the consistency of the database, this is the brains of the operation if we lose a file in CodeStation or the logs we may have degraded functionality but the tool will remain functional, alternatively if our database is corrupt we will have to restore to an earlier point and lose deployment history/data. The incremental process for the database is standard practices as well, and depends specifically on the vendor but the concept is to use the full backup process to set regular full restore points then enable transaction logging on the database(s) to act as a play forward capability. So if we take a full backup on the 1st of the month, and we have transaction logs being captured all the time.
Our recovery solution for the database becomes a complete restore to the latest full backup, and a replay of the transaction logs to catch-up the database to the point in time we desire. The filesystem is similar, we recommend that you have a complete backup to match the database, and follow the same process to capture a incremental set of changes. This is no always as straight forward as the filesystem for logs and CodeStation change with both adds and deletes, based on your usage this could be a change of 100’s of Gigabytes of data over a week or two.
While the simplest case is to just match the full database and filesystem together, since storage is cheap this is not that bad of a choice while your logs and CodeStation files size is relatively small, say < 1T. But as your system grows and your concurrency increases, your files will start to exceed that 1-3T range and keeping several copies of this around is a less appealing option. One common practice is simplify the process and not necessarily keep an incremental but just keep the filesystem at the latest available.
Specific Backup procedure for the database
Use the vendor tools to do complete backups and move them to your offsite storage location. For incremental, we first enable transaction logging, the second part of this is to have a regular synchronization of these transaction logs offsite on a regular basis, every 15-30m.
Specific Backup procedure for files
This will be different for each product, but the list of folders to sync is pretty short.
For BluePrint Designer:
- /opt/ibm-ucd-patterns/conf ( per server )
- /opt/ibm-ucd-patterns/repositories ( shared )
- /opt/ibm-ucd-patterns/workspace ( shared )
- /opt/ibm-ucd/server/conf ( per server )
- /opt/ibm-ucd/server/appdata ( shared data )
- /opt/IBM/URelease/ucrelease ( per server )
Ensuring that you have a valid backup plan and process in place is key to being able to recover from a a disaster. One extremely useful side-effect here is that you know have the basis for creating staging and development environments for testing your systems for various scenarios like upgrades or training.