Back up data

This topic describes what you need to back up and provides guidelines for the backup process.

Database backup

When working with Connect, it is a best practice to perform a regular backup of the PostgreSQL database, the database that stores all of the connections and data source information, regularly.

To perform an effective backup, follow the instructions in the official PostgreSQL documentation, in the Backup and Restore section. This will ensure that your database is backed up correctly and that it can be restored in case of data loss or corruption.

Back to top

Configuration file backup

This section outlines the purpose and location of the configuration files that need to be backed up to maintain proper functionality and ensure smooth synchronizations.

File Description
Connect\AppData\data\database.properties Stores the connection details for the database, including credentials and connection details.
Connect\AppData\data\secrets-config.yaml

Contains configurations for secret encryption and decryption.

Caution: Losing this file will result in the inability to decrypt any secrets. For details, see Secret encryption.

Connect\AppData\data\node.properties Stores node-specific configurations.
Connect\AppData\osp-filter-configuration.properties Contains client OSP configuration settings.
Connect\AppData\osp-server-configuration directory

Contains server OSP configuration settings.
Connect\WebServer\conf directory Contains Tomcat server configurations.
Connect\jre\lib\security\cacerts Contains Java Runtime Environment (JRE) security information.

Ensure that these configuration files are backed up regularly to prevent data loss and facilitate quick recovery in case of system failures or configuration issues.

 

Back to top

Database backup example

For explicit details on how to perform a database backup, refer to the Backup and Restore section in the PostgreSQL documentation. The following is a sample syntax for the backing up the database.

Backup command

pg_dump -U <adminUsername> -F c -f "C:\backups\<databaseName>.backup" <databaseName>

Cross-Platform Compatibility

The pg_dump command works identically on both Linux and Windows systems. The only difference between the two platforms is the file path format used to specify backup files:

  • On Windows, use Windows-style paths, for example, C:\backups\db.backup
  • On Linux, use Unix-style paths, for example, /home/user/backups/db.backup

All command options and flags behave the same way across both operating systems, ensuring consistent backup and restore procedures regardless of the environment.

Back to top

Configuration file backup example

This section provides example scripts for backing up the Connect configuration files for both Windows and Linux.

Windows

Copy code 
@echo off
    setlocal

    REM Check if the Connect path and backup directory path are provided as arguments
    if "%~1"=="" (
    echo Usage: %~nx0 ^<connect_path^> ^<backup_directory^>
    exit /b 1
    )
    if "%~2"=="" (
        echo Usage: %~nx0 ^<connect_path^> ^<backup_directory^>
        exit /b 1
            )

    REM Set the Connect path and backup directory
    set "connect_path=%~1"
    set "backup_dir=%~2"

    if not exist "%connect_path%\" (
        echo Error: connect_path "%connect_path%" does not exist.
        exit /b 1
            )

    if not exist "%backup_dir%\" (
        echo Error: backup_dir "%backup_dir%" does not exist.
        exit /b 1
            )
    REM Get the current date and time
    for /f "tokens=2 delims==." %%I in ('wmic OS Get localdatetime /value') do set "datetime=%%I"
    set "year=%datetime:~0,4%"
    set "month=%datetime:~4,2%"
    set "day=%datetime:~6,2%"
    set "hour=%datetime:~8,2%"
    set "minute=%datetime:~10,2%"
    set "second=%datetime:~12,2%"

    REM Set the backup zip file name
    set "backup_zip=%backup_dir%\mfConnectBackup_%year%-%month%-%day%_%hour%-%minute%-%second%.zip"

    REM Create a temporary directory to preserve the file structure
    set "temp_dir=%backup_dir%\temp_backup"
    mkdir "%temp_dir%"

    REM Copy the files to the temporary directory preserving the structure
    xcopy "%connect_path%\AppData\data\database.properties" "%temp_dir%\AppData\data\" /Y
    xcopy "%connect_path%\AppData\data\secrets-config.yaml" "%temp_dir%\AppData\data\" /Y
    xcopy "%connect_path%\AppData\data\node.properties" "%temp_dir%\AppData\data\" /Y
    xcopy "%connect_path%\AppData\osp-filter-configuration.properties" "%temp_dir%\AppData\" /Y
    xcopy "%connect_path%\AppData\osp-server-configuration" "%temp_dir%\AppData\osp-server-configuration\" /E /Y
    xcopy "%connect_path%\WebServer\conf" "%temp_dir%\WebServer\conf\" /E /Y
    xcopy "%connect_path%\jre\lib\security\cacerts" "%temp_dir%\jre\lib\security\" /Y

    REM Create the backup zip file
    powershell -Command "Compress-Archive -Path '%temp_dir%\*' -DestinationPath '%backup_zip%'"

    REM Clean up the temporary directory
    rmdir /S /Q "%temp_dir%"

echo Backup created at %backup_zip%
    endlocal

Windows Usage Example

backup_script.bat <connect_path> <backup_target_directory>

Linux

Copy code 
#!/bin/bash
    # Check if the Connect path and backup directory path are provided as arguments
    if [ -z "$1" ] || [ -z "$2" ]; then
        echo "Usage: $0 <connect_path> <backup_directory>"
        exit 1
    fi

    # Set the Connect path and backup directory
    connect_path="$1"
    backup_dir="$2"

    if [ ! -d "$connect_path" ]; then
        echo "Error: Directory '$connect_path' does not exist."
        exit 1
    fi

    if [ ! -d "$backup_dir" ]; then
        echo "Error: Directory '$backup_dir' does not exist."
        exit 1
    fi

    # Get the current date and time
    current_date=$(date +"%Y-%m-%d_%H-%M-%S")

    # Set the backup tar.gz file name
    backup_tar="$backup_dir/mfConnectBackup-$current_date.tar.gz"

    # Create a temporary directory to preserve the file structure
    temp_dir=$(mktemp -d)

    # Create the necessary directories in the temporary directory
    mkdir -p "$temp_dir/AppData/data"
    mkdir -p "$temp_dir/AppData"
    mkdir -p "$temp_dir/AppData/osp-server-configuration"
    mkdir -p "$temp_dir/WebServer/conf"
    mkdir -p "$temp_dir/jre/lib/security"

    # Copy the files to the temporary directory preserving the structure
    cp "$connect_path/AppData/data/database.properties" "$temp_dir/AppData/data/"
    cp "$connect_path/AppData/data/secrets-config.yaml" "$temp_dir/AppData/data/"
    cp "$connect_path/AppData/data/node.properties" "$temp_dir/AppData/data/"
    cp "$connect_path/AppData/osp-filter-configuration.properties" "$temp_dir/AppData/"
    cp -r "$connect_path/AppData/osp-server-configuration" "$temp_dir/AppData/osp-server-configuration/"
    cp -r "$connect_path/WebServer/conf" "$temp_dir/WebServer/conf/"
    cp "$connect_path/jre/lib/security/cacerts" "$temp_dir/jre/lib/security/"

    # Create the backup tar.gz file
    tar -czf "$backup_tar" -C "$temp_dir" .

    # Clean up the temporary directory
    rm -rf "$temp_dir"

echo "Backup created at $backup_tar"        

Linux Usage Example

./backup_script.sh <connect_path> <backup_target_directory>

Back to top