Overview
The Replications panel lists all the replications created from the Source Tables Selection dialog box or added manually from the Replications panel.
The Replications panel displays the following columns:
Column Name | Description |
|---|---|
Target Name | Name of the target table configured in the replication. Click to view the replication overview, execution history, and logs for the recent replication refresh or execution. |
Target Schema | Schema of the selected target table. To modify the schema, click Edit, turn off the Use Default Schema toggle button, and specify the Target Schema. Then, click Save. |
Source Table | Name of the source table associated with the replication. |
Source Datastore | Name of the source datastore to which the source table belongs. |
Source Schema | Schema of the selected source table. |
Progress Status | Provides real-time status updates on the events that a replication job processes in a replication. Refer to the Replication Job Progress and Status section for more information. |
Replication Type | Displays the replication type.
|
Priority | Denotes the order of the replications to be executed for the selected source and target datastores. |
Record Count | Number of records moved to the target table. |
Volume (MB) | Displays the volume of data processed in megabytes (MB) during the last execution of the replication. |
Last Refreshed | Timestamp of the last execution of the replication. |
Duration | Time taken to complete the replication. |
Active | Turn on the toggle button to set a replication as Active for data replication. Turning off this toggle button disables all quick action buttons except Delete and Show Logs for this replication. |
Actions | Includes the quick action icons to execute a replication, schedule a replication job, view recent logs, and so on. Refer to the Quick Action Feature in the Replications Panel section for more information. |
The Replications panel displays the number of replications configured for the selected source datastore against a target datastore. If required, you can expand and collapse this panel when auditing or reviewing details in this panel.
Columns in grids throughout the Syniti Knowledge Platform (SKP) can be individually customized, sorted, and filtered. You can also search for names or values for in columns that have the Search feature enabled. The selected settings are remembered per user. Refer to SKP Grid Searching, Sorting, and Filtering for more information.
Quick Action Feature in the Replications Panel
You can perform the below replications or actions using the quick action icons in the Actions column of the Replications panel. Currently, the following quick actions are supported in the Replications panel:
Note
To view more quick actions available in the Replications panel, click the More Actions button in the Actions column.
Execute Replications: The Execute Replication icon allows you to run a replication job for the selected replication associated with the corresponding target and source datastores. Refer to Execute Replication Jobs for more information.
Schedule: The Schedule icon allows you to schedule the execution of a replication with customizable recurrence options, offering flexibility in running replication jobs at regular intervals as required. Refer to Schedule Replication Jobs for more information.
Field Mappings: The Field Mappings icon allows you to select the required columns in a source table, and to modify how the column attributes must be configured in a target table during a data snapshot. Refer to Perform Field Mapping for more information.
Replication Dependencies: The Replication Dependencies icon allows you to define dependent replications associated with a selected replication, such that the dependent replications are run before executing the selected replication. Refer to Configuring Replication Dependencies for more information.
Edit: Modify the replication configuration. For example, you can update the applied source filter or change the target table name.
Show Logs: View and debug logs for the most recent execution of the selected replication.
Delete: Remove a replication from the associated target and source datastores.
Bulk Actions in the Replications Panel
Use the multi-select checkboxes in the Replications panel to select the required replications and perform supported bulk actions, with or without column filters. Currently, the following bulk actions are supported:
Add Selected to Group: Add the selected replications to an existing group, a new group, or multiple groups.
Add All to Group: Add all available replications to a group.
Execute: Execute only the selected replications, with or without replication state filters applied on the Progress Status column.
Delete Selected: Delete the selected replications.
Delete All: Delete all replications.
Adding Manual Replications
If you want to create a replication for certain use cases (for example, move only a subset of the source table data to the target table) or specify different column attributes and names for your target table, you can add a replication manually or edit an existing replication.
To add a manual replication:
In the Sources panel, select the required source datastore associated with the intended target datastore and click the Add Replication (+) button in the Replications panel. The New Replication dialog box is displayed.
Note
Ensure that the selected source datastore and its corresponding target datastore is displayed correctly.
In the Source section, tables available in the source datastore is displayed. Select the required table.
If required, you can filter records from the source table using the Source Filter field. For example, you can specify a condition (WHERE clause) to ensure only relevant records are inserted into the target table.
Note
You don't need to explicitly include the WHERE keyword in the Source Filter field. For example, if you want to replicate only records where the COUNTRY field equals 'US', simply enter
COUNTRY = 'US'in the Source Filter field.In the Target section, the Target datastore and its database and schema values are displayed only when the datastore configuration contains their values. If no values are specified, then Source datastore’s database and schema values will be used.
In the Target Name field, enter the table name corresponding to the source table name.
In the Additional Properties section, complete the following steps:
If you want to specify higher preference for this replication over others, enter a priority value in the Priority field.
Note
The replications will be ordered and executed using the Priority value. By default, the value for each replication is set at 1000, which provides equal preferences for all replications, but ordered using the creation time of a replication.
If you want to replicate data without truncating data in the target table, turn on the Append Mode toggle button to append new data from the source table.
Select Pre-Execution and Post-Execution Workflows to run if this replication is integrated with other module’s tasks in an Orchestrate workflow.
Click Save.
A replication for the selected source datastore is added to the current workspace.
Adding an Orchestrate Workflow as a Replication
You can add an Orchestrate workflow to integrate with other module’s capabilities directly from Replication workspaces as part of your replication execution sequence, broadening data migration, quality, and replication use cases. For example, you may want to execute a data quality report, SQL command, and so on.
Complete the following steps to create an Orchestrate Workflow:
From the Targets and Sources panels, select the required Target and Source datastores. A list of replications configured for this target and source datastores combination is displayed in the Replications panel.
In the Replications panel, click More Actions > Create Orchestrate Replication. The New Orchestration Replication dialog box is displayed. Ensure that the selected the source datastore is displayed on the Source list.
From the Orchestrate Workflow list, select a workflow associated with the source datastore selected in the previous step.
If required, modify the Target Name field used as the replication name in the workspace. By default, Replicate Preview uses the Orchestrate Workflow’s Name for the replication name.
If you want to specify higher preference for this replication type over others, enter a priority value in the Priority field.
NoteThe replications will be ordered and executed using the Priority value. By default, the value for each replication is set at 1000, which provides equal preferences for all replications, but ordered using the creation time of a replication.
Click Save.
Adding File Replications
File Import Replications
Currently, you can import data from flat files using Syniti Drive as the source datastore and an SKP Catalog System as the target datastore.
To create file import replications:
In the Targets panel, select the required SKP Catalog System as the target datastore.
In the Sources panel, select the required Syniti Drive as the source datastore.
In the Replications panel, click the Add Replication (+) button. The New Replication dialog box is displayed.
In the Sources section, select the required file available from the Files list.
Note
The Files list is contingent upon the selected file format. Currently the following file formats are support for file import replications:
.csv
.json
.parquet
.xlsx
A maximum of 10,000 files can be searched when selecting files for replication.
For example, if you select a .csv file, enter the following details to parse data from the file:
Field Name
Description
Delimiter
Defines the character used to separate values in the file. For example, use
,for CSV files or|for pipe-separated files. Default character is comma (,).Encoding
Specifies the text encoding used in the file to ensure special characters are read properly. Default encoding format is UTF-8.
Header Row
Indicates which row (by index) in the file contains the column names. For example, set to 0 if the first row contains the headers.
Column Names
Specify the column names that match the destination columns using the following format:
["Column1", "Column2", "Column3"]. New columns will be created if the specified columns do not exist.In the Target section, use the Target Name field to modify the table name, if required.
(Optional) In the Additional Properties section, complete the following steps:
If you want to specify higher preference for this replication over others, enter a priority value in the Priority field.
Note
The replications will be ordered and executed using the Priority value. By default, the value for each replication is set at 1000, which provides equal preferences for all replications, but ordered using the creation time of a replication.
Select Pre-Execution and Post-Execution Workflows to run if this replication is integrated with other module’s tasks in an Orchestrate workflow.
Click Save.
File Export Replications
You can export data to a file in Syniti Drive by using it as the target datastore, with an SKP Catalog System as the source datastore. You can bulk create file export replications using the Source Table Selection feature or create manually.
To manually create file export replications:
In the Targets panel, select the required Syniti Drive as the target datastore.
In the Sources panel, select the required SKP Catalog System as the source datastore.
In the Replications panel, click the Add Replication (+) button. The New Replication dialog box is displayed.
In the Source section, complete the following steps:
Select the required table you want to export.
If required, you can filter out records from this table using the Source Filter field.
In the Target section, complete the following steps:
From the Export As list, select the required file format.
In the Delimiter field, specify the character that separates values in the file. Default character is comma (,). For example, enter comma (,) or pipe (|) if you are exporting data in .txt format.
In the Encoding field, specify the text encoding to be used in the file to ensure special characters are read properly. Default encoding format is UTF-8.
if required, use the Target Name field to modify the file name.
(Optional) In the Additional Properties section, complete the following steps:
If you want to specify higher preference for this replication over others, enter a priority value in the Priority field.
NoteThe replications will be ordered and executed using the Priority value. By default, the value for each replication is set at 1000, which provides equal preferences for all replications, but ordered using the creation time of a replication.
Select Pre-Execution and Post-Execution Workflows to run if this replication is integrated with other module’s tasks in an Orchestrate workflow.
Click Save.
Editing a Replication
To edit a replication:
In the Replications panel, select the required replication associated with the correct target and source datastores.
In the Actions column, select More Actions > Edit for the selected replication. The Edit Replication dialog box is displayed.
Modify the required fields that needs changes.
After finishing the required modifications, click Save.
Deleting a Replication
In the Replications panel, select the required replication associated with the correct target and source datastores.
In the Actions column, select More Actions > Delete for the selected replication. The Delete Replication Replication dialog box is displayed.
If you want to delete multiple replications, use the multi-select checkbox to select the required replications and click the Delete button at the top of the Replications panel. The Delete Selected Replications dialog box is displayed.
Click Confirm.
The selected replications and their associated workflows have been deleted successfully.