Importing XML into Data Services

This topic is related to the following sections:

• SAP Data Services for Migrate

• Preparation of XML in Migrate

• Post Import Maintenance of Project

• Mapping Standards

• Export Building and Execution

• Report Building and Execution

• Cross Reference Building and Execution

Overview

After downloading the Syniti Migrate XML automation code, import the XML file to SAP Data Services. The order in which these files are imported is very important because the tables need to be active in the ETL prior to importing the Project shell and Source Mappings. Generally, the Datastores are built by the Admin IT Lead or Project Manager as they are determined to be in scope for the project. The tables are ultimately the responsibility of each developer as they build the migration of each object.

Import the XML scripts in this order:

1. XML Code for Datasources - the source systems, target systems, and work databases.

2. XML Code for the Dataset in Mapping - this will build out the project, jobs, workflows, dataflows, and queries of the target data.

3. XML Code for each Source in Mapping - this will build out the specific transform of each source mapping to include the XREF inserts, updates, from source to target.

Import XML Code for Datasource

Prerequisites for Importing XML Code for a Datasource

The prerequisite for this import will be to store the System Source and Database Source in the Local Repository.  Either import these from the Central Object Library or manually create the Datastores for the System Source and Database Source.  These datastores must be created manually within DataServices and then pushed to the Central Object Repository for all users to access.  Usually, the IT Lead or Project Manager will setup the initial datastores in DataServices so that the consultants will not need to build each.  Refer to section Building Datastores in Data Services for details of this effort.

Note

As part of the Data Services system that uses imported XML scripting from Syniti Migrate, Data Services must have an OSS note applied that will update the system for Case Sensitive.  Please refer to section DS Repository Setting for Case Sensitive for details of this update.

The import will create queries to pull the refresh of data from SYSSource tables into SRCSource Tables. The XML code for a Datasource will contain these types of scripting:

• Project

• Job for datasource as Source_010_AllTables

• Workflow for the source as Source_DEFAULT

• Dataflows for each of the ACTIVE tables of this source as Source_TableName_Import

• Queries for each of the ACTIVE tables of this source pulling data from the system source or sysSource.dbo to the database source or srcSource.dbo (These will need to be validated as running without errors)

While viewing the SAP Data Services Designer page, click Tools > Import From File…

On the Open Import File window, change the Files of Type drop-down to XML Files (*.xml) and select the desired file. Click Open to proceed.

Click Yes to proceed.  In the bottom left corner, the import data names will show as they are imported.

The Import Plan will display with details of the data for import. Click Import to proceed.

The system will display a Passphrase as shown below. Click Import as these files are not password protected.

The importing of XML scripting and the build of the Datasource project will run through to completion. The specific WF, DF, Tables will display at the bottom left corner as they are being added to the local repository. Example shown below:

Once the process is complete, the messaging will stop at bottom left corner of page. Click on the Local Object Library tab for Project, and select and double-click on that newly added Import Project to open it in the Project Area panel in upper left corner.

Make sure to view every one of the query built by the import to ensure that the tables from and to are active and joined to the query. In the case above, the Datastore for SYSADVWRK.dbo is not yet created in Data Services and so it shows with a RED X and is disconnected from the query to show that it will fail if run.

Once the missing Datastore is added, you will need to come back and import to these queries the active table from the newly added Datastore. Delete the Tables showing with the RED X as it will not work. This is why it is so important to build the Datastores manually PRIOR to any imports of Datasources to Data Services.

Continue this process for import of Datsource XML until complete.

Import from Central Object Library

The process of migrating data includes steps for Cross Reference, Reporting, Remediation of field values and these steps will incorporate tables stored within MIGRATE database.  The tables are assembled into a Datastore that will refresh these tables based upon the updates made throughout the conversion of the data.

xAppMigrate Datastore - This datastore contains 2 tables (XTVALUEMAP and LIST_TARGET_VALUE) from MIGRATE database within SQL Server.

xAppReport - This datastore contains all reports data as tables for a snapshot view of reporting.

xAppMigrate_WebService - This datastore is type REST and is used for WADL reporting functions.

As part of the initial setup for DataServices, these Datastores may be set up and stored in Central Object Library. The users can then import these and other Datastores for use in the conversion of data.

At this point, right click on the Datastore and select Get Latest Version > Object and dependents.  This action imports the latest version of the datastore and all associated tables to the Local Repository.

The resulting Datastore should look like the example below:

Import XML Code for Mapping Dataset

Prerequisites for Importing XML Code for a Mapping Dataset

• Create all Datastores for this conversion manually within the Local Repository.

• Within the Datastores, the user must already have setup and import the tables and fields for each of the tables needed in the object conversion - the source tables, the target checking and loading tables, and the working database tables.

The XML code for a Mapping Dataset contains the following:

• Project for Object Transform

• Jobs for each process in the conversion including Transform, Export, Load, TestRunner, Postload, and Target Readiness

• Workflows for each of the jobs

• Dataflows for each of the Workflows of each of the Jobs Queries for Inserts

• Target Tables and Reporting Tables

While viewing the SAP Data Services Designer page, click Tools > Import From File…

On the Open Import File window, change the Files of Type drop-down to XML Files (*.xml) and select the desired file.

Click Yes to proceed.  In the bottom left corner, the import data names will show as they are imported.

The Import Plan will display with details of the data for import. Click Import to proceed.

The system displays a Passphrase as shown below. Click Import button as these files are not password protected.

After the XML file has been imported, review the development object code by selecting the desire Work Flow, Data Flow, or other development object.

A successful import will include jobs for Transform, TargetReadiness, Export, Load, Postload, and TestRunner.  Within each job, there should be Workflows, Dataflows, and Queries as the base from which to build out the process for conversion.  This import is NOT expected to be a COMPLETE SOLUTION.  The user must supply the Rule update queries, any Export details, and validate all sections to ensure that it will run as expected.

Note

If Datastores were not added to the Local repository prior to this import, the queries that use these Datastores will be in an Error state and will require manual correction.

Note

At this point, the Project, Workflows, and Dataflows that are relevant to the target level information is imported.  Notice that there is no Source data Workflow in the Transform at this time. This is a separate import.

Import XML Code for Mapping Sources

Prerequisites for Importing XML Code for Mapping Sources

The prerequisite for this import will be to have already imported the Mapping Dataset XML scripting that will build the overall Project and target related structures. This import will specifically represent the Sources mapped to the targets. The XML code for a Mapping Dataset will contain these types of scripting:

• Source table definition

• Working table definition XREF Table setup

• Dataflow for the source table

• Workflows for the Dataflows of the source table Queries to Insert and Update the field values

While viewing the SAP Data Services Designer page, click Tools > Import From File…

On the Open Import File window, change the Files of Type drop-down to XML Files (*.xml) and select the desired file.

On the following dialog, click the Yes button to proceed.  In the bottom left corner, the import data names will show as they are imported.

The Import Plan will display with details of the data for import. Click Import to proceed.

The system will display a Passphrase as shown below. Click on the Import button as these files are not password protected.

After the XML file has been imported, review the development object code by selecting the desire Work Flow, Data Flow, or other development object.

Note

If Datastores were not added to the Local repository prior to this import, the queries that use these Datastores will be in an Error state and will require manual correction.

Note

At this point, Workflows, and Dataflows that are relevant to the source level information is imported to the Local Object Library.  The source is NOT attached to the project at this time.