Snapshot Management Workspace

Note

• Existing customers can use both the Migrate > Snapshot Management page and Snapshot Management workspace for a limited time to help transition to the new user interface (UI). During this period, Migrate remains the source of truth for snapshot registrations and modifications. Refer to the Notes for Existing Customers section for more information.

• New customers will be provisioned with the Snapshot Management workspace and can immediately start using the enhanced functionalities available in Replicate Preview. For these users, Replicate Preview serves as the source of truth for all snapshot registrations and modifications.

Overview

The Snapshot Management workspace is a dedicated workspace in Replicate Preview, which replaces the legacy Snapshot Management UI within Migrate, allowing Migrate to use the capabilities offered by Replicate Preview instead of the Syniti Replicate desktop version. This change enhances the user experience by consolidating data replication capabilities within a unified interface while maintaining full compatibility with existing Migrate workflows, such as replications in ETL Jobs, target datastores on the Value Mapping page, and so on.

This workspace contains Snapshots (a point-in-time copy) of data from a source or target system being used for Mock or Live conversion.

• From the Source Datasource into the Snapshot Source Datasource (created with a prefix of SRC)

• From the Target Datasource to the Snapshot Target Datasource (created with a prefix of TGT)

Select Migrate > Snapshot Management from the Migrate menu to access the Snapshot Management workspace. This workspace provides access to the following additional functionalities that were not available in the legacy UI:

• Configuring replication dependencies

• Creating a replication group

• Running an Orchestrate workflow as a replication within Snapshot Management

Notes for Existing Customers

• If a target or source datastore is used in another workspace, Replicate Preview will automatically create the same datastore with a MIG suffix, which points to the same underlying database and schema. For example, if TGTBAR already exists in Workspace A, TGTBAR_MIG will be created using the same connection details in the Snapshot Management workspace.

• When the migration process in run again, any new registrations or changes made in the workspace will be synchronized with the state maintained on the Migrate > Snapshot Management page.

• Note that running the same replication in both the Snapshot Management page and the workspace will result in errors due to primary key conflicts. If no primary key is present, you may encounter duplicate snapshot records.

Key Changes

• Navigation: The Snapshot Management workspace is automatically provisioned for all new tenants in the Replicate Preview cloud environment. For existing tenants, all configuration resources are automatically migrated to the workspace, streamlining the migration process.

  Terminology in the legacy UI	Terminology in the workspace
  Snapshot Datasources	Target Datastores: Target datastores registered in the Syniti Knowledge Platform (SKP)’s Catalog Datastores prefixed, with SRC*** or TGT***. A target datastore registered in the Snapshot Management workspace creates a datasource in Migrate > Catalog > Datasources (with Datasource Purpose set to Snapshot), making the target datastore available for existing Migrate workflows.
  System Datasources	Source Datastores: Source datastores registered in both SKP Catalog Datastores and Catalog > Datasources within the Migrate module, with Datasource Purpose set to System.
  Snapshot Tables, System Snapshot Tables	Replications: Replication options: * Tables from source or target systems * Orchestrate workflows as replications

• Catalog > Datasources: Target datastores configured in Replicate Preview are displayed on the Datasources page with the Datasource Purpose set to Snapshot.

  Note

  You cannot delete datasources with a Snapshot or System Datasource Purpose while they are configured in the Snapshot Management workspace. Similarly, you cannot delete a target datastore while it is configured in Migrate.

• Do Not Import Toggle: This toggle was previously used to exclude columns from being imported during replication. This functionality has been replaced by the Field Mappings feature in the Replications panel. Refer to Performing Field Mappings for more information.

  To exclude columns:

  • 1. In the Replications panel, access the required replication and select Field Mappings > Map Explicit mapping rule.

    2. Deselect the columns that are no longer required and click Save.

• MIGRATE_APP Tables: This system datasource is now available as OData endpoints (automatically provisioned in SKP Catalog) containing a list of views prefixed with MIG_. These views are used to store data for Central Relevancy tables, System Reports, and Rulebook views. The Snapshot Management workspace retains the existing functionalities of MIGRATE_APP, and it cannot be edited or deleted from the Sources panel. However, it can be removed by deleting the associated target datastore.

• Job Queues: Logs generated during replication refresh (repopulate) are now managed within the Snapshot Management workspace using the Progress Status column in the Replications panel. Click the status of a replication to view its previous execution logs. Refer to Replication Job Progress and Status for more information.

• Target Exports: Target Export ETL tasks are now executed via Replicate Preview and exported to the configured folder path in Syniti Drive. Refer to Target Exports for more information.

Users & Roles

The Design Team is responsible for maintenance of the Snapshot Management workspace.

Tasks specific to Database Systems

The Syniti Migrate Application is stored in SQL Server, however, the data may reside in a different database on a client site. Refer to The Data Migration Process for details about working in a database that is not the Migration SQL Server database. In some cases, the project uses SAP HANA or a different client-held SQL Server for the working database. These sections of help describe how to use these different databases and provide more details about using Snapshot Datasources:

• Migration Using a Second SQL Server as the Database

• Migration using PostgreSQL as the Database

• Migration Using SAP HANA as the Database

Target Datastore xAppLTMC Used in the SAP S/4HANA LTMC Conversion Process

Refer to LTMC - SAP S/4HANA Migration Cockpit for details about using the LTMC Staging option for SAP S/4HANA system projects.

Configure the Snapshot Management Workspace

Prerequisites

Before you configure datastores in the Snapshot Management workspace, complete the following steps:

• Create datastores in the SKP Catalog module for both target and source systems. For example, create SRCECC and TGTBAR target datastores, as well as ECC and BAR system datastores. Refer to Datastores for more information.

• Ensure that the connection’s Usage for the required source and target datastores is set to Replication.

• Ensure that a scan has completed successfully for the datastores so metadata is ingested into the SKP.

• Ensure that the server for each datastore exists and that its connection is active.

• Configure the system datastores (for example, ECC and BAR) as System Datasources in Catalog > Datasources with the Datasource Purpose set to System. Refer to Datasources for more information.

Create a Target Datastore

1. Access Migrate > Snapshot Management and click Add Target (+) in the Targets panel. The Add Target Datastore dialog box is displayed.

2. From the Datastore list, select the target datastore configured in the Catalog module.

   Note

   • Only datastores prefixed with SRC*** and TGT*** must be selected as target datastores in the Snapshot Management workspace. Selecting other datastores will display an error.

   • The Default Database and Default Schema values for the target datastores are fetched from the datastore configuration. If the datastore configuration does not include database and schema values, you can manually enter the Default Database and Default Schema values for this target datastore.

3. If you want to specify higher preference for this target datastore over others, enter a priority value in the Priority field.

4. In the ZSource field, enter the source datastore name.

   Note

   • The ZSource field is automatically populated based on the selected datastore name minus the prefix as a default. If the target datastore is TGTECC, the ZSource is ECC. However, you can specify a different source datastore name.

   • You cannot update the ZSource name after the datastore is added.

5. Turn on the SRC Construct toggle to create MIGRATE_APP source datastore to import Central Relevancy tables, System Report views, and Rulebook views via OData endpoints.

   Note

   Once the MIGRATE_APP source datastore is added, it cannot be edited or deleted from the Sources panel.

6. Click Save.

7. In Migrate, access Catalog > Datasources, ensure that the newly created target datastore is displayed on the Datasources list with the Datasource Purpose set to Snapshot.

Refer to Creating Target Datastores and Working with the Targets Panel for more information on target datastores and its usage.

Create a Source Datastore

1. In Migrate, access Catalog > Datasources and create the required Datasources with the Datasource Purpose set to System.

2. Import the metadata for each table that must be included in the snapshot. Refer to Adding and Editing Datasources for more information.

3. Access Migrate > Snapshot Management and click Add Source (+) in the Sources panel. The Add Source Datastore dialog box is displayed.

4. From the Datastore list, select the source datastore configured from Step 1.

   Note

   Only datastores that are configured as Datasources with the Datasource Purpose set to System in Catalog > Datasources are available for selection as source datastores.

5. If you want to specify higher preference for this source datastore over others, enter a priority value in the Priority field.

6. Click Save.

Refer to Creating Source Datastores and Working with the Sources Panel for more information on source datastores and its usage.

Create a Replication

1. In Migrate > Snapshot Management > Sources panel, click Source Table Selection to import tables from the source datastore to the target datastore. This action creates the required replications in the Replications panel. Refer to Selecting Source Tables for more information.

   Note

   You can use the Import All button within the Soure Tables Selection dialog box to create replications and import all of the tables and views in the source datastore to the target datastore.

   Note

   Importing all tables is not recommended for SAP datastores, as they can contain a large number of tables.

2. To add manual replications, refer to the Adding Manual Replications section for more information.

3. If the ETL Tool is SAP Data Services, access that target datastore in Catalog > Datasources and run the BUILD XML to build out the XML scripting and create the tables in the database. Refer to Snapshot Datasource Build Import XML for more information.

   Note

   SAP Data Services ETL Tool is deprecated. Use Syniti Migrate as the preferred ETL tool.

Refer to Working with the Replications Panel for more information on replications and its usage.

Exclude Columns from Import

To mark columns that should not be imported:

1. In the Replications panel, access the required replication and select Field Mappings > Map Explicit mapping rule.

2. Deselect the columns that are no longer required.

3. Click Save.

Refer to Performing Field Mappings for more information.

Execute a Snapshot Replication

Clicking the Refresh icon at target-, source-, or replication-level executes a replication job to truncate the table and re-import the data rows (content) from the source datastore to the target datastore.

Modify a replication to use the Source Filter field to specify a condition (WHERE clause) such that only relevant records are inserted into the target table. If the replication exists prior to adding a value to this field, the next refresh attempts to add the Source Filter to the replication and save the change.

For SAP Datasources, data is populated into the target table from the source table based on the Where clause added and the Client field’s value. Refer to SAP Datasources filtered by Client for more information.

Note

Tables may be imported using a datastore (database) connection (the preferred method) or using an RFC connection, and sometimes both may be used. If both Database and RFC connections exist, all imports of data default to the Database connection unless the field TABLE TYPE is set as Pooled or Clustered. These are special SAP Tables that must be read from the application layer.

Note

Use the Progress Status column in the Replications panel to view the status of a replication and click the status to view its previous execution logs. Refer to Replication Job Progress and Status for more information.

Once refreshed, the Replications panel table shows the last refreshed date and active count of records pulled. The Duration is displayed in seconds.

Add an SRCCONSTRUCT Target Datastore

You must set up a static database for the Construct module on a project as it allows the resources the ability to continually maintain the Construct database tables while the team takes a snapshot and uses the static data for the testing phase.

The SRCCONSTRUCT database is stored in the database used for migration (see Working Database in Migrate) at the client site, and therefore the data in that database is visible for use by the conversion team. The CONSTRUCT database stores the active tables, views, and rules for Syniti Construct (SQL Server database).

For example, the SRCCONSTRUCT database can be configured in SQL Server in the Migration Working Server, but this configuration also works in SAP HANA or PostgreSQL. The SRCCONSTRUCT database stores all of the Construct page's tables as a copy from the CONSTRUCT database in the client site SQL Server.

The same process for assigning available source datastores (system datasource) tables to the target datastore (snapshot datasource) works for SRCCONSTRUCT.

Note

The SRCCONSTRUCT target datastore automatically includes the following three source datastores:

• CONSTRUCT to store the active Construct page tables

• MIGRATE to store the Value mappings tables

• MIGRATE_APP to store the Central Relevancy tables, System Report views, and the Rulebook views

Central Relevancy Snapshot Data Stored in SRCCONSTRUCT

The MIGRATE_APP source datastore is linked to the SRCCONSTRUCT target datastore to provide a list of views prefixed with MIG_ used to store data for System Reports. These views allow you to build reports against table data stored in the core, hidden database. The SRCCONSTRUCT database stores these views as tables that repopulate the rows of data.

MIGRATE_APP is not an actual database and it cannot be edited or deleted. It is assigned to SRCCONSTRUCT to support construction and reporting. Syniti Migrate creates this link between MIGRATE_APP and SRCCONSTRUCT on saving the SRCCONSTRUCT target datastore and then runs a job to import the views stored in the core database for use in reporting or metrics. Refreshing these views pulls in the records. Each of the views may be used as a System Report for viewing or exporting data.

The following MIG_ views are imported:

Value Mapping Snapshot Data Stored in SRCCONSTRUCT

The process for value mapping check table values from legacy to target is stored in the SRCCONSTRUCT database for use as a Snapshot Management process. The LIST_TARGET_VALUE and XTVALUEMAP tables are automatically added to:

• The SRCCONSTRUCT target datastore to store values for static use during mock conversion testing and

• The MIGRATE source datastore to store active values where changes are made in real time.

The LIST_TARGET_VALUE table stores the fields TARGET_VALUE and LOAD_VALUE so that in a situation where the Target Value is a concatenation of more than one field, the Load Value contains the actual value that loads to the target table. In the image below, for the records for the REGION_LIST check table, the TARGET_VALUE and LOAD_VALUE columns contain different values. For the records for the COUNTRY_LIST check table, these fields contain the same value.

The XTVALUEMAP table stores the data used in the value mappings for fields including the ZXREF_SOURCE, ZXREF_VALUE and TARGET_VALUE as examples. This table is used in the Source Xref rules for Update and Insert in ETL and when building the XML scripting for SAP Data Services for the Xref rules.

Edit Datastores in the SKP Catalog Module

In the Snapshot Management workspace, clicking a datastore in either the Targets or Sources panel opens the datastore’s details page in the SKP Catalog module. From the datastore’s details page, you can manage the datastore’s tables and columns, run scans, profile tables, and test the associated connection.

Refer to Datastores for more information.

Note

Running a scan ingests metadata into the SKP such as tables and their columns, along with the constraints for each table listed in the Tables list for this datastore. Refer to Scan a System Datastore for more information.

View Datasource Details in Migrate

The Datasources page in Catalog > Datasources lists all datasources with specific Datasource Purpose configured within Migrate using the system and datastore components from the SKP Catalog module. Refer to the Datasource Purpose section for more information.

To view details of the datasources, click on the Edit icon to display the Datasource Details page. The details of the datasource depend upon the type of datasource as a System, Migrate, Report, Snapshot, or Working type. Refer to the Edit a Datasource section for more information.

The server for all of the Snapshot purpose datasources is the database used for the conversion, for example, Migration, SAP HANA, or PostgreSQL. The Application is always set to Migration DEPLOY for Snapshot purpose datasources.

The System Datasource (the source datastore) must be created in the Application and Instance of the type System. The Snapshot is the static capture of data from the system source in the Migration SQL Server.

Import Metadata

For datasources with the System purpose, Migrate uses the scanned metadata from the SKP Catalog Datastore to build a job queue that imports columns and constraints for each table included in the import list. Filtering can be applied based on criteria set up in Administer > Advanced > Applications.

To import metadata:

1. Select Catalog > Datasources from the Migrate menu.

2. Click the Edit icon for a datasource.

3. Click Additional Options and select Import Metadata.

The single Job Queue contains one to many steps, one for each table that import metadata. Each step builds out the table's columns and constraints, and points to the specific debug log entries for that table.

View Debug Logs

To view Debug logs:

1. Select Catalog > Datasources from the Migrate menu.

2. Click the Edit icon for a datasource.

3. Click Additional Options and select View Debug Logs.

A list of Debug log entries for this Datasource's latest refresh of data displays. Refer to Debug Log for more information.

View History

You can view a history of all changes to a Datasource.

To view history:

1. Select Catalog > Datasources from the Migrate menu.

2. Click the Edit icon for a datasource.

3. Click Additional Options and select View History.

Edit a System Datasource in Migrate

To edit a System Datasource:

1. Select Catalog > Datasources in the Migrate menu.

2. Click the Edit icon for a System Datasource in the Datasources panel.

3. Select the System Datasource’s Instance from the Instance list box.

4. Update the datasource name in the Datasource Name field, if needed.

5. Enter a description of the System Datasource in the Description field.

6. If needed, toggle Active to off to make the System Datasource Inactive. Otherwise, retain the Active default setting.

7. Enter descriptive comments to support this System Datasource in the Comment field.

8. In the Syniti Catalog System section, select the source datastore. The System Datasource must be connected with an existing system in the SKP Catalog module.

9. Select the type of database used by the System Datasource from the Database Type list box.

   Note

   The Database Name and Database Schema fields display data for this datastore from by the SKP Catalog module.

10. Enter the SAP Client value in the Client field.

    Note

    This value is used for SAP replications, and the field only displays for SAP-based System Datasources.

11. Select the user responsible for this Datasource from the Owner list box.

12. Turn on the Import All toggle to import metadata for all tables of the Datasource.

    Note

    The Last Refresh Date is the date of the last import of metadata.

13. Enter the name of the Data Services Datastore from which jobs pull source data in the SAP Data Services field. This datastore is used in the Syniti Migrate automation XML.

    Note

    The SKP Asset ID field displays the Asset ID for the datasource provided by Syniti Catalog.

14. Click Save.

Delete a Datastore in the Workspace

You can delete a target or source datastore in the Snapshot Management workspace if it has no dependencies in both Replicate Preview and Migrate.

To delete a datastore:

1. In the Targets or Sources panel, select the required datastore.

2. In the Actions column, click the Delete icon for the selected datastore. The Delete Datastore dialog box is displayed.

3. Click Confirm.