Snapshot Management

A Snapshot is a static database of tables and columns pulled on a specific date from a system being used for Mock or Live Conversions.

Snapshot Management is the process of managing the data that is captured:

• from the Source Datasource into the Snapshot Source Datasource (created with a prefix of SRC) and

• from the Target Datasource to the Snapshot Target Datasource (created with a prefix of TGT).

The Snapshot Datasource pulls rows of data from the System Datasource into the table using Syniti Replicate and builds a job in Syniti Migrate that manages the job set in Syniti Replicate.

Access the Snapshot Management page by selecting Migrate > Snapshot Management from the Migrate menu.

Types of Snapshot tables that display on the Snapshot Management page are:

• Tables that are automatically refreshed and capture data from the Source Datasource to the Snapshot Source Datasource and from the Target Datasource to the Snapshot Target Datasource databases.These Snapshot Datasources are created with the prefix of either SRC or TGT to represent the direction of the data and are associated with a System Datasource. These types of tables are described in this topic.

• Tables that must be manually replicated from a text file or a system that is not supported by the Syniti Knowledge Platform. These tables do not have a System Datasource, but do have an associated Syniti Replicate replication. When you manually refresh tables of this type, the replication executes. Refer to Repopulate Using a Manual Replication for more information.

• Tables that exist in a Snapshot database that are not refreshed. Typically, these are views used for mapping. While these tables display on the Snapshot Management page, the Refresh icon is dimmed for them as they do not have a System table or an associated Syniti Replicate replication.

Prerequisites for Snapshot Management

Before you create a Snapshot Datasource:

• The Datasource Server must exist and the connection must be active.

• The System Datasource must exist on the Datasources page (Catalog > Datasources). Refer to Datasources for more information.

Users & Roles for Snapshot Management

The Design Team is responsible for maintenance of the Snapshot Datasources.

Tasks specific to Database Systems

The Syniti Migrate Application is stored in SQL Server, however, the data may reside in a different database on a client site. Refer to The Data Migration Process for details about working in a database that is not the Migration SQL Server database. In some cases, the project uses SAP HANA, Oracle or a different client-held SQL Server for the working database. These sections of help describe how to use these different databases and provide more details about using Snapshot Datasources:

• Migration Using Oracle as the Database

• Migration Using SAP HANA as the Database

• Migration Using a Second SQL Server as the Database

SNAPSHOT Datasource xAppLTMC Used in the SAP S/4HANA LTMC Conversion Process

Refer to LTMC - SAP S/4HANA Migration Cockpit for details about using the LTMC Staging option for SAP S/4HANA system projects.

Create and Populate the Snapshot Datasource

To create and populate the Snapshot Datasource with tables and rows of data, follow these steps.

1. Create the System Datasource and import the metadata for each table that must be included in the snapshot. Refer to Adding and Editing Datasources for more information.

2. Create the Snapshot Datasource, assigning it to a System Datasource from which the tables and row data is pulled on a scheduled basis. Refer to Add a Snapshot Datasource for details.

3. In the System Datasource, assign tables from the System Datasource to the Snapshot Datasource. Refer to the Add Tables to a Snapshot Datasource for more information.

4. In the Snapshot Datasource, run the BUILD XML to build out the XML scripting and create the tables in the database.

   Note

   This step creates the tables in the Snapshot Datasource if the ETL Tool is external (Data Services). Refer to Snapshot Datasource Build Import XML for more information.

5. Access the Snapshot Management page by selecting Migrate > Snapshot Management. from the Migrate menu.

6. Click the Repopulate icon to import table data (content) from the System Datasource into the Snapshot Datasource using Syniti Replicate system jobs. Refer to Repopulate Snapshot Datasource from System Datasource for details.  

The following details apply to the Snapshot Datasource. Details for the System Datasource that is attached to a Snapshot Datasource are provided in Adding & Editing Datasources.

The Snapshot Datasources panel displays the following fields:

Snapshot Tables Tab

When you select a row for a Snapshot Datasource, the Snapshot Tables tab displays the following fields:

Add a Snapshot Table

If the required table does not exist in the Catalog System Datasource, and is not available to select in the System Datasources tab, then add the table manually to the System Datasources database and then import it on the Snapshot Tables list.

To add a Snapshot table:

1. Select Migrate > Snapshot Management in the Migrate menu.

2. In the Snapshot Datasources panel, select the Snapshot Datasoruce to add the table to.

3. In the Snapshot Tables tab, click the Add  icon. The Import Table dialog box is displayed.

4. From the System Datasource list, select a Datasource to import a table manually.

5. Enter the System Datasource schema in the Schema field.

6. Enter the table name in the Table or View Name field.

   Note

   This field is case-sensitive and it validates that the table exists in the System Datasource database.

7. Click the Save icon.

System Datasources Tab

The System Datasources tab has two panels: System Datasources and System Snapshot Tables.

The System Datasources panel contains the following fields:

The System Snapshot Tables panel displays the following fields:

To add a System Snapshot table:

1. In the System Snapshot Tables panel, click the Add  icon. The Import Table dialog box is displayed.

2. Enter the System Datasource schema in the Schema field.

3. Enter the table name in the Table or View Name field.

   Note

   This field is case-sensitive and it validates that the table exists in the System Datasource database.

4. Click the Save icon.

Add a Snapshot Datasource

To add a Snapshot Datasource:

1. Select Migrate > Snapshot Management from the Migrate menu.

2. Click the Add  icon in the Snapshot Datasource panel.

3. Select the prefix from the Prefix list box. Options are:

   • SRC: Select this prefix if you are creating a Source Datasource.

   • TGT: Select this prefix if you are creating a Target Datasource.

   • No Prefix: This option is not allowed, you must select a prefix.

4. Enter the source system name in the ZSource field. This name is allowed a maximum of 12 characters.

   Note

   The Datasource Name field is automatically populated with the values entered in the Prefix and ZSource field.

5. Enter a description of the Datasource in the Description field.

6. Enter comments to support this Datasource in the Source Comments field.

7. Under System Datasources, click the Add button to select the System Datasource to link to this Snapshot Datasource.

8. Select the System Datasource from the System Datasource list box and click the Add button.

9. Under Connection, select the Connection Server name from the Server Id list boix. The values that display below this list box depend on the selected Server.

10. Enter the name of the database for this datasource in the Database Name field.

11. Enter the schema for the database for this datasource in the Default Schema field.

    Note

    You must use the Snapshot Datasource schema. The System Datasource schema cannot be used.

12. Enter the number of seconds this Datasource refreshes before it is stopped to prevent excessive clocking in the Timeout(seconds) field. The default with no value is set by the Job Queue and Job Type, but defaults to 30 seconds.

13. Enter the user name to access this datasource in the User Name field or leave blank and the Server access is used by default.

14. Enter the password to access this datasource in the Password field or leave blank and the Server access is used by default.

    Note

    The Datastore name in the Datastore field is copied from the ZSource field and is the SAP Data Services Datastore from which the jobs pull source data. This Datastore is used in the creation of the automation XML.

    Note

    The section for XML is visible on the Datasource details page if the ETL Tool is not Syniti Migrate (for example, if the tool is SAP Data Services). If the ETL Tool field in the Administer > Setup > Parameters page is set to Syniti Migrate, the XML section is hidden

    Note

    You cannot update the ZSource or Datastore name after the Datasource is added.

    Note

    The Connection section of the Create a Snapshot Datasource modal cannot be left empty. If it is, the following happens:

    • If the user does not provide any connection info, Migrate displays an error message after the user clicks the Save icon.

    • If the user provides server information (by completing the Server Id and Database Name fields) but does not provide the username and password, the system attempts to test the connection using the credentials from the server, and updates the status accordingly.

15. Click the Save icon to save the Snapshot Datasource.

    Note

    Snapshot databases are added to the Syniti Migration Application.

    Note

    A warning message displays to indicate that you cannot change the ZSource and or Datasource Name fields after you have saved the Snapshot Datasource.

16. Click Ok.

Issues During the Save

Snapshot Datasource needs to have prefix of TGT or SRC

An error message displays if you have not set the Prefix for the Datasource to either TGT or SRC.

To fix the issue, click the Cancel Save button and update the Prefix correctly.

Datasource already exists

The new Datasource has the same name as an existing Datasource. The Save fails. Click OK and add a unique Datasource name.

Snapshot Datasource Requires a Server

Snapshot Datasource requires a server ID to save the datasource. Click OK and complete the Server ID field to proceed.

Verify the Connection

After saving the Snapshot Datasource, you must verify the connection is active.

To verify the connection:

1. Select Migrate > Snapshot Management from the Migrate menu.

2. Select the Datasource in the Snapshot Datasources panel.

3. Click the Connection  Icon to initiate a check against the system.

   If the connection is successful a message displays: "Primary - Connection success."

If the connection is not successful, the tables and columns do not import and metadata does not process successfully. The Job Queue runs, but no outcome of data is present.

Repopulate a Snapshot Datasource from a System Datasource

Syniti Migrate allows you to repopulate a datasource to pull the rows of data from a source table to a target table using Syniti Replicate.

To repopulate the Snapshot Datasource from a System Datasource:

1. Select Migrate > Snapshot Management from the Migrate menu.

2. Click the Repopulate  icon for a Snapshot Datasource table or for an entire Snapshot Datasource to initiate the job queue refresh of data using a replication built within Syniti Replicate.

3. A confirmation message displays, click OK to proceed.

Note

This process connects Syniti Migrate to Syniti Replicate, builds a replication based on the data stored in the Snapshot Datasource and System Datasource, and updates the replication to contain a Where Clause filtering the data where a value exists in the field. This process is seamless: after you click the Repopulate icon, the replication runs. The data is populated into the snapshot table from the system table based on the Where clause added and the Client value (for SAP systems). Refer to SAP Datasources filtered by Client for more information.

Note

Tables may be imported using a Database connection (the preferred method) or using an RFC connection, and sometimes both may be used. If both Database and RFC connections exist, all imports of data default to the Database connection unless the field TABLE TYPE is set as Pooled or Clustered. These are special SAP Tables that must be read from the application layer.

While viewing the Snapshot Management page, select to highlight a snapshot datasource. This action updates the right side panel to display the tables linked between snapshot and system.

Refer to the Syniti Replicate help topic Managing Replications in the Management Center for details on maintenance for replications.

Snapshot Management: Snapshot Datasource Details

4. Select Monitor > Job Queues from the Migrate menu to view the progress of the Repopulate job running for the Datasource table. The replication process begins and completes showing the duration, record count, and date of run. Refer to Job Queues for more information.

The Debug log provides details of the run.

Once refreshed, the Snapshot Datasource table shows the Repopulated On date and active count of records pulled. The Duration is displayed in seconds.

In Syniti Replicate, the same job queue is stored as they are linked together.

Repopulate Using a Manual Replication

If a link from a source datasource to a target datasource requires a manual build through Syniti Replicate for the replication (for example, if the source is a text file, a system not registered in the Syniti Knowledge Platform or a system that cannot be scanned in the Syniti Knowledge Platform), you can perform a manual replication on a Syniti Migrate Snapshot Datasource table.

To perform a manual replication:

1. Select Migrate > Snapshot Management from the Migrate menu.

2. On the Snapshot Tables tab, click the Edit icon for the Snapshot table that must be manually replicated.

3. Enter the Where clause to pull data into the Snapshot table in the Snapshot System Where Clause field.

4. Click the Select SDR Replication icon  and select a replication from the active Replication jobs stored in Syniti Replicate.

   Note

   Refer to the Syniti Replicate help topic Managing Replications in the Management Center for details about creating manual replications.

1. Click the Save icon.

After the record is saved, the following occurs as part of the Repopulate process:

• This link overrides the Repopulate attempt to build a replication job as part of the Repopulate process,

• The Snapshot System Where Clause field is no longer visible or used for the replication.

• The Repopulate action runs the job assigned from Syniti Replicate as it is set.

By creating this manual replication, you can build replication jobs outside of the automation provided by Syniti Migrate and use them through the same Snapshot Management process as the normal repopulate steps.

Add a CONSTRUCT Snapshot Datasource

You must set up a static database for the Construct module on a project as it allows the resources the ability to continually maintain the Construct database tables while the team takes a snapshot and uses the static data for the testing phase.

The SRCCONSTRUCT database is stored in the database used for migration (SAP HANA, Oracle, or SQL Server) at the client site, and therefore the data in that database is visible for use by the conversion team.

The CONSTRUCT database stores the active tables, views and rules for Syniti Construct (SQL Server database).

In the image below, the SRCCONSTRUCT database is set up in SQL Server in the Migration Working Server, but this setup also works in SAP HANA or Oracle. The SRCCONSTRUCT database stores all of the Construct page's tables as a copy from the CONSTRUCT database in the client site SQL Server.

The same process for assigning available System Datasource tables to the Snapshot Datasource works for SRCCONSTRUCT. The SRCCONSTRUCT snapshot is linked to three System Datasources:

• CONSTRUCT to store the active Construct page tables

• MIGRATE to store the Value mappings tables

• MIGRATE_APP to store the Central Relevancy tables, System Report views, and the Rulebook views

Central Relevancy Snapshot Data Stored in SRCCONSTRUCT

The System Datasource MIGRATE_APP is linked to the SRCCONSTRUCT Snapshot Datasource to provide a list of views prefixed with MIG_ used to store data for System Reports. These views allow you to build reports against table data stored in the core, hidden database. The SRCCONSTRUCT database stores these views as tables that repopulate the rows of data.

MIGRATE_APP is not an actual database and it cannot be edited or deleted. It is assigned to the SRCCONSTRUCT Snapshot Datasource to support construction and reporting. Syniti Migrate creates this link between MIGRATE_APP and SRCCONSTRUCT on saving the SRCCONSTRUCT snapshot Datasource and then runs a job to import the views stored in the core database for use in reporting or metrics. Repopulating these views pulls in the records. Each of the views may be used as a System Report for viewing or exporting data.

The following MIG_ views are imported:

Value Mapping Snapshot Data Stored in SRCCONSTRUCT

The process for value mapping check table values from legacy to target is stored in the SRCCONSTRUCT database for use as a Snapshot Management process. The LIST_TARGET_VALUE and XTVALUEMAP tables are automatically added to:

• the SRCCONSTRUCT Snapshot Datasource to store values for static use during mock conversion testing and

• the MIGRATE System Datasource to store active values where changes are made in real time.

The LIST_TARGET_VALUE table stores the fields TARGET_VALUE and LOAD_VALUE so that in a situation where the Target Value is a concatenation of more than one field, the Load Value contains the actual value that loads to the target table. In the image below, for the records for the REGION_LIST check table, the TARGET_VALUE and LOAD_VALUE columns contain different values. For the records for the COUNTRY_LIST check table, these fields contain the same value.

The XTVALUEMAP table stores the data used in the value mappings for fields including the ZXREF_SOURCE, ZXREF_VALUE and TARGET_VALUE as examples. This table is used in the Source Xref rules for Update and Insert in ETL and when building the XML scripting for SAP Data Services for the Xref rules.

Edit a Snapshot Datasource

Snapshot Datasource names follow a naming convention:

• The Source Snapshot Datasource name contains the system name with a Prefix of SRC (For example, SRCJDE, SRCECC)

• The Target Snapshot Datasource name contains the system name with a Prefix of TGT (for example, TGTECC, TGTS4H)

The server for all of the Snapshot Datasources is the database used for the conversion (for example Migration, SAP HANA, or Oracle). The Application is always set to Migration DEPLOY for Snapshot Datasources.

Attach each Snapshot Datasource to a System Datasource of the same name. For example, attach a System Datasource of SRCJDE to a Snapshot Datasource named JDE.

The System Datasource (the source system connection) must be created in the Application and Instance of the type of system. These are the System Datasources (the actual source system connection). The Snapshot is the static capture of data from the system source in the Migration SQL Server.

To edit a Snapshot Datasource:

1. Select Migrate > Snapshot Management from the Migrate menu.

2. Click the Edit icon to display the Datasource Detailspage.

   Note

   So that the table data does not become fragmented and incorrect, once a System Datasource is linked to a Snapshot Datasource and vice versa, this link is permanent and cannot be removed. The value in the field is display only.

1. Select an Instance from the Instancelist box, if needed.

   Note

   A Snapshot Datasource is always created with a ZSource which is derived from the Datasource Name minus the prefix as a default. If the Snapshot Datasource is TGTECC, the ZSource is ECC. The ZSource cannot be edited.

2. Enter a description of the Snapshot Datasource in the Description field.

3. Toggle the Active toggle to off if the Snapshot Datasource is not active. Otherwise, leave the default.

4. Enter comments to support the Snapshot Datasource in the Comments field.

5. Select the Snapshot Datasource server from the Server Id list box. Refer to Datasource Servers for more information.

6. Enter the Snapshot Datasource database name in the Database Name field.

7. Enter the schema used by default for the Snapshot Datasource in the Default Schema field.

8. Enter the number of seconds to allow before a timeout when running this Snapshot Datasource refresh in the Timeout(seconds) field.

9. Enter the user name for the Snapshot Datasource in the Username field.

10. Enter the password for the Snapshot Datasource in the Passwordfield.

    Note

    Once the user name and password have been saved into a Datasource, only the Administrator role has access to view the password going forward. The Show/Hide icon for viewing the password is not available to anyone who is not provided the Role of Administrator.

11. The Owner field automatically populates with the user name of the user who created the record. The field value is configurable in Administer > Advanced > Stakeholders. Refer to Stakeholders for more information,.

12. Leave the Import Alloff, or toggle it to on, if needed.

    Note

    This activates an import process and is used on systems other than SAP. Refer to Import Metadata for more information.

13. If you are using a Data Services Datastore for the Snapshot Datasource, enter the name in the Datastore field. Refer to Building Datasources in Data Services for more information.

14. If the Snapshot Datasource is SAP Data Services, click the Create Snapshot Tables button to create all Snapshot Datasource tables listed in the working Database.

15. If the Snapshot Datasource is SAP Data Services, click the Build Import XML button to build the automation XML scripting for Datasource Import within the ETL. Refer to Snapshot Datasource Build Import XMLfor more information.

    Note

    The Build Import XML button displays on this page if the ETL Tool is not Syniti Migrate, for example if it is SAP Data Services. If the Snapshot ETL Tool list box on the Parameters page (Administer > Setup > Parameters) is set to Syniti Migrate, the XML section is hidden.

16. After the scripts are built, click the View Import XML button to view the automation XML scripting for Datasource Import in the ETL. Refer to Snapshot Datasources View Import XMLfor more information.

    Note

    The View Import XML button displays on this page if the ETL Tool is not Syniti Migrate, for example if it is SAP Data Services. If the Snapshot ETL Tool list box on the Parameters page (Administer > Setup > Parameters) is set to Syniti Migrate, the XML section is hidden.

17. If the Snapshot Datasource is not SAP Data Services, click the Start Data Refresh button to initiate the Job Queue to repopulate all active tables for this Snapshot Datasource.

18. As the purpose of this Datasource is Snapshot, leave the the Report Flag off to indicate that this datasource is does not have a Purpose of Report.

19. As the purpose of this Datasource is Snapshot, leave the Working DB Flagoff to indicate that this datasource is not set with a Datasource Purpose of Working.

    Note

    The Snapshot Flag is toggled to on by default for Snapshot Datasources.

20. Click the Save icon.

Edit a System Datasource

To edit a System Datasource:

1. Select Catalog > Datasources in the Migrate menu.

2. Click the Edit icon for a System Datasource in the Datasources panel.

3. Select the System Datasource’s Instance from the Instance list box.

4. Update the datasource name in the Datasource Name field, if needed.

5. Enter a description of the System Datasource in the Description field.

6. If needed, toggle Active to off to make the System Datasource Inactive. Otherwise, retain the Active default setting.

7. Enter descriptive comments to support this System Datasource in the Comment field.

8. Click the Select from Catalog button to open the Select Catalog Asset for Datasource modal, if this System Datasource must be connected with an existing system in the Syniti Knowledge Platform Catalog. .

   On this modal:

   1. Select the System from the Syniti Knowledge Platform Catalog to connect to this System Datasource.

   2. Click the Save icon.

   3. Close the modal..

9. Select the type of database used by the System Datasource from the Database Typelist box.

   Note

   The System Friendly Name field and the System Name field display data for this datasource provided by the Syniti Knowledge Platform’s Catalog.

   Note

   The Path field displays a path for the datasource provided by Syniti Catalog. This field only displays if this System Datasource has had a system from the Syniti Knowledge Platform connected to it.

10. Enter the SAP Client value in the Clientfield.

    Note

    This value is used for SAP replications, and the field only displays for SAP-based System Datasources.

11. Select the user responsible for this Datasource from the Owner list box.

12. Toggle Import Allto on to import metadata for all tables of the Datasource.

    Note

    The Last Refresh Date is the date of the last import of metadata.

13. Enter the name of the Data Services Datastore from which jobs pull source data in the Datastorefield. This Datastore is used in the Syniti Migrate automation XML.

    Note

    The SKP Asset ID field displays the Asset ID for the datasource provided by Syniti Catalog.

14. Click the Save icon.

Import Metadata on the Snapshot Datasources Page

To import metadata:

1. Select Migrate > Snapshot Management in the Migrate menu.

2. Click the Edit icon for a Snapshot Datasource.

3. Click the Additional Options  icon.

4. Select Import Metadata.

Syniti Migrate builds a job queue that imports columns and constraints for each table included in the import list for this datasource. Filtering is based on criteria set up in Administer > Advanced > Applications

The single Job Queue contains one to many steps, one for each table that import metadata as shown below.

Each step builds out the table's columns and constraints, and points to the specific debug log entries for that table.

View Debug Logs on the Snapshot Datasources Page

To view Debug logs:

1. Select Migrate > Snapshot Management in the Migrate menu.

2. Click the Edit icon for a Snapshot Datasource.

3. Click the Additional Options  icon.

4. Select View Debug Logs.

A list of Debug log entries for this Datasource's latest refresh of data displays. Refer to Debug Log for more information.

View History on the Snapshot Datasources Page

You can view a history of all changes to a Datasource.

To view history:

1. Select Migrate > Snapshot Management in the Migrate menu.

2. Click the Edit icon for a Snapshot Datasource.

3. Click the Additional Options  icon.

4. Select View History.

Delete a Snapshot Datasource

You can delete a Snapshot Datasource if it does not have any dependencies.

To delete a Snapshot Datasource:

1. Select Migrate > Snapshot Management in the Migrate menu.

2. Select that Snapshot Datasource, and click the Delete  icon.

3. Click OK to the confirmation message.

Add or Remove a Table from the Source Snapshot Datasource

On the System Datasources tab, on the Snapshot Management page, the System Datasources associated with the selected Snapshot Datasource display. The System Datasource tables included in the selected System Datasource display in the panel below.

To add or remove a table from a Source Snapshot Datasource:

1. Select Migrate > Snapshot Management in the Migrate menu.

2. On the System Datasource tab, click the Select Tables   icon for the System Datasource where the tables will be imported.

1. Toggle the Snapshot field to on or off for each table relevant to the project to and it add it to, or remove it from, the System Snapshot Tables panel on the Snapshot Management page.

2. Once all of the tables have been added, click the Cancel   icon to close the modal.

Add All Active System Tables to the Snapshot Datasource

You can import all of the tables in the System Datasource to the Snapshot Datasource.

To import all tables:

1. Select Migrate > Snapshot Management in the Migrate menu.

2. On the System Datasources tab, click the Select Tables   icon for the System Datasource where the tables will be imported.

3. Click the Import All button.

   Note

   This process imports every active table from the System Datasource into the Snapshot Datasource.

4. Click OK to the confirmation message.

Snapshot Datasource Tables Tab

On the right side of the Snapshot Datasources details page is the Datasource Tables tab, which provides a panel for Datasource tables and a panel for Datasource columns. Refer to Edit Snapshot Tables or Columns for more details.