This topic relates to the following sections:
This page contains the following sections:
There are two types of ETL within Syniti Migrate that include the SAP Data Services system and Syniti Migrate ETL. The Syniti Migrate ETL tool processes all conversion of migration data using the ETL Jobs tool stored within Migrate > Mappings section. The Syniti Migrate ETL tool is available for use in SQL Server, HANA database, or Oracle databases.
Syniti Migrate ETL Jobs perform the job steps that are normally performed within SAP Data Services Projects.
The ETL Jobs page provides a list of all Syniti Migrate ETL Jobs sorted by Project / Release / Dataset / Version.
Tasks unique to this process include:
The XML Builds and Views not relevant (hidden) for the Dataset, Working tables, or Snapshot Management as they are used within SAP Data Services.
All SQL Server tables are generated as part of the Import Dataset process.
The Developer completes the auto-generated views for all Rules, XREF Rules, etc.
The Developer completes the Stored Procedure that is automatically generated from the ADD RULES within each Rule in Mappings.
The Developer maintains the order the steps of the conversion within the ETL Jobs page.
The Developer executes the ETL Job to run the conversion through SQL Server to update the working table and target table values.
Step 1 - Parameters for ETL Tool
Within Administer > Setup > Parameters a default for ETL Tool is set at initial system setup. This defaults to a Project / Release prior to any direct assignment to the ETL Tool within each Release. At least one Release for ETL Tool must set to Syniti Migrate for the menu path and page for ETL Jobs to be visible. If the ETL Tool is set as SAP Data Services for all Releases in the Project, ETL pages are hidden from view (Not necessary). As shown below, the default in this project is set to Syniti Migrate for the ETL Tool.
NOTE: Datasources use the Parameters ETL Tool value to determine if the XML sections are visible. Should the project use both SAP Data Services and Syniti Migrate for ETL Tools, default the Parameters page to SAP Data Services so that the Datasources can produce the XML scripting needed for building datasources in SAP Data Services.
Administer: Parameters: ETL Tool
Step 2 - Project Setup - Release must assign ETL Tool
At least one Release within a Project must assign the ETL Tool as Syniti Migrate in order for the ETL icons and processing to be visible in the system. If all Releases are set to SAP Data Services, the ETL Jobs pages are not be visible for use.
NOTE: It is possible to work on a single project that includes both external and Syniti Migrate ETL tools.
Project Setup: Edit Release: ETL Tool
Step 3 - Mapping Fields to produce ETL Job Tasks
Once the field mapping is complete, click the ETL icon stored in the Mappings page to initiate the build process. Refer to section Migrate > Mappings > Mapping the fields of a Datasource > Mapping When Using Migrate ETL Tool for details of this process.
This process resides in a few sections of Syniti Migrate as shown below -
Parameters page - Designate Syniti Migrate as the ETL Tool
Mapping of the Datasource to target table - Map a field of fields as a Type = Rule to initiate the ETL Jobs Tool for the object. Setup one or more Rule Stored Procedures for the field. See Details for this process within section Migrate > Mappings > Mapping the Fields of a Datasource > Mapping When Using Migrate ETL Tool in section Mapping Action Details for a Rule using Migrate ETL Tool..
Mappings Dataset - Execute the ETL Job within Job Queue to run the conversion of data process. See Details for this process within section Migrate > Mappings > Mapping the Fields of a Datasource > Mapping When Using Migrate ETL Tool in section Execute the ETL Job queue
Mappings Dataset - View the ETL Job setup with a click on the ETL JOBS button. See Details for this process within section Migrate > Mappings > Mapping the Fields of a Datasource > Mapping When Using Migrate ETL Tool in section View ETL Job Sequence
The Data Migration Project Lead, Deployment IT lead, or the Migration Team Developers are responsible for managing ETL Jobs as part of the Syniti Migrate ETL Tool process.
The IT Lead or Team Administrator responsible for all Dataset Design initial setup
The user developing the ETL process in Syniti Migrate
Once the conversion has been executed within Migrate > Mappings, the user may view ETL Transform Job List by following pathway Migrate > ETL Jobs from the Syniti Migrate Homepage as shown below. This page allows for a view of all ETL jobs and overall management of the timing for runs. The grid for this page lists in ascending order each Project / Release / Dataset / Version. Within this page, the entire conversion process for a Version may be monitored.
ETL Jobs: List of all ETL Jobs
The ETL Job List panel displays the following fields that are all stored in other sections of the system:
|Project||The Project name stored in Project Setup.|
|Release||The Release name stored in Project Setup.|
|Dataset||The Dataset name stored in Dataset Design.|
|Version||The Version of the run. (Ex: DEV, MOCK1, FINAL)|
|Name||The Transform an Interface Name.|
|Description||The Job Step Description.|
|Executed On||The date of last execution of the job step.|
|Duration||The time in seconds for the job step to run.|
|Messages||Error messages produced from the run.|
|Developer||Logon ID for the Developer.|
|Working Datasource||Name of the Working Database used for this Dataset.|
|Prefix||Prefix value stored in Dataset Design Details page.|
|Interface RICEFW||RICEFW value stored in Mappings Dataset Details page. See note below.|
|Dataset RICEFW||RICEFW value stored in Dataset Design Dataset Details page. See note below.|
|Design Complete||Date stored in Mappings Dataset Details page|
|Develop Complete||Date stored in Mappings Dataset Details page|
|Test Complete||Date stored in Mappings Dataset Details page|
|Validation Status ID||Status of last run for the task.|
NOTE: RICEFW stands for Reports, Interface, Conversion, Enhancements, Forms, and Workflow. These are forms produced by SAP during processing. In this scenario, a ticket number is entered to represent the form.
View details for the ETL Transform Job List
From this page the user may select to view the ETL Job details by clicking the Edit icon for the row of the ETL Transform Job List. The Dataset Transform level of ETL Job displays and provides links to all levels of the ETL for this dataset. The Refer to section ETL Tasks at the Dataset Level for the details.
ETL Jobs: Dataset Transform