Syniti Migrate

ETL Jobs

This topic relates to the following sections:

This page contains the following sections:

About ETL Jobs

There are two types of ETL within Syniti Migrate that include the SAP Data Services system and Syniti Migrate ETL. The Syniti Migrate ETL tool processes all conversion of migration data using the ETL Jobs tool stored within Migrate > Mappings section. The Syniti Migrate ETL tool is available for use in SQL Server, HANA database, or Oracle databases.

Syniti Migrate ETL Jobs perform the job steps that are normally performed within SAP Data Services Projects.

The ETL Jobs page provides a list of all Syniti Migrate ETL Jobs sorted by Project / Release / Dataset / Version.

Tasks unique to this process include:

  • The XML Builds and Views not relevant (hidden) for the Dataset, Working tables, or Snapshot Management as they are used within SAP Data Services.

  • All SQL Server tables are generated as part of the Import Dataset process.

  • The Developer completes the auto-generated views for all Rules, XREF Rules, etc.

  • The Developer completes the Stored Procedure that is automatically generated from the ADD RULES within each Rule in Mappings.

  • The Developer maintains the order the steps of the conversion within the ETL Jobs page.

  • The Developer executes the ETL Job to run the conversion through SQL Server to update the working table and target table values.

Prerequisites to ETL Jobs

Step 1 - Parameters for ETL Tool

Within Administer > Setup > Parameters a default for ETL Tool is set at initial system setup. This defaults to a Project / Release prior to any direct assignment to the ETL Tool within each Release. At least one Release for ETL Tool must set to Syniti Migrate for the menu path and page for ETL Jobs to be visible. If the ETL Tool is set as SAP Data Services for all Releases in the Project, ETL pages are hidden from view (Not necessary). As shown below, the default in this project is set to Syniti Migrate for the ETL Tool.

NOTE:  Datasources use the Parameters ETL Tool value to determine if the XML sections are visible. Should the project use both SAP Data Services and Syniti Migrate for ETL Tools, default the Parameters page to SAP Data Services so that the Datasources can produce the XML scripting needed for building datasources in SAP Data Services.

Administer: Parameters: ETL Tool

Step 2 - Project Setup - Release must assign ETL Tool

At least one Release within a Project must assign the ETL Tool as Syniti Migrate in order for the ETL icons and processing to be visible in the system. If all Releases are set to SAP Data Services, the ETL Jobs pages are not be visible for use.

NOTE:  It is possible to work on a single project that includes both external and Syniti Migrate ETL tools.

Project Setup: Edit Release: ETL Tool

Step 3 - Mapping Fields to produce ETL Job Tasks

Once the field mapping is complete, click the ETL icon stored in the Mappings page to initiate the build process. Refer to section Migrate > Mappings > Mapping the fields of a Datasource > Mapping When Using Migrate ETL Tool for details of this process.

This process resides in a few sections of Syniti Migrate as shown below -

Users & Roles in ETL Jobs

The Data Migration Project Lead, Deployment IT lead, or the Migration Team Developers are responsible for managing ETL Jobs as part of the Syniti Migrate ETL Tool process.

Role

Definition

Admin

The IT Lead or Team Administrator responsible for all Dataset Design initial setup

Developer

The user developing the ETL process in Syniti Migrate

View ETL Transform Job List

Once the conversion has been executed within Migrate > Mappings, the user may view ETL Transform Job List by following pathway Migrate > ETL Jobs from the Syniti Migrate Homepage as shown below. This page allows for a view of all ETL jobs and overall management of the timing for runs. The grid for this page lists in ascending order each Project / Release / Dataset / Version. Within this page, the entire conversion process for a Version may be monitored.

ETL Jobs: List of all ETL Jobs

The ETL Job List panel displays the following fields that are all stored in other sections of the system:

Heading

Description
Project The Project name stored in Project Setup.
Release The Release name stored in Project Setup.
Dataset The Dataset name stored in Dataset Design.
Version The Version of the run. (Ex: DEV, MOCK1, FINAL)
Name The Transform an Interface Name.
Description The Job Step Description.
Executed On The date of last execution of the job step.
Duration The time in seconds for the job step to run.
Messages Error messages produced from the run.
Developer Logon ID for the Developer.
Working Datasource Name of the Working Database used for this Dataset.
Prefix Prefix value stored in Dataset Design Details page.
Interface RICEFW RICEFW value stored in Mappings Dataset Details page. See note below.
Dataset RICEFW RICEFW value stored in Dataset Design Dataset Details page. See note below.
Design Complete Date stored in Mappings Dataset Details page
Develop Complete Date stored in Mappings Dataset Details page
Test Complete Date stored in Mappings Dataset Details page
Validation Status ID Status of last run for the task.

NOTE:  RICEFW stands for Reports, Interface, Conversion, Enhancements, Forms, and Workflow. These are forms produced by SAP during processing. In this scenario, a ticket number is entered to represent the form.

View details for the ETL Transform Job List

From this page the user may select to view the ETL Job details by clicking the Edit icon for the row of the ETL Transform Job List. The Dataset Transform level of ETL Job displays and provides links to all levels of the ETL for this dataset. The Refer to section ETL Tasks at the Dataset Level for the details.

ETL Jobs: Dataset Transform