ETL Jobs

This topic relates to the following sections:

• Mapping When Using Migration ETL Tool

• ETL Jobs Details

• Execution of the ETL Jobs List

• ETL Version Selection

  • ETL Tasks at the Release level

  • ETL Tasks at the Dataset level

• ETL Tasks at the Target level

• ETL Tasks at the Source level

About ETL Jobs

There are two types of ETL within Syniti Migrate that include the SAP Data Services system and Syniti Migrate ETL. The Syniti Migrate ETL tool processes all conversion of migration data using the ETL Jobs tool stored within Migrate > Mappings section. The Syniti Migrate ETL tool is available for use in SQL Server, HANA database, or Oracle databases.

Syniti Migrate ETL Jobs perform the job steps that are normally performed within SAP Data Services Projects.

The ETL Jobs page provides a list of all Syniti Migrate ETL Jobs sorted by Project / Release / Dataset / Version.

Tasks unique to this process include:

• The XML Builds and Views not relevant (hidden) for the Dataset, Working tables, or Snapshot Management as they are used within SAP Data Services.

• All SQL Server tables are generated as part of the Import Dataset process.

• The Developer completes the auto-generated views for all Rules, XREF Rules, etc.

• The Developer completes the Stored Procedure that is automatically generated from the ADD RULES within each Rule in Mappings.

• The Developer maintains the order the steps of the conversion within the ETL Jobs page.

• The Developer executes the ETL Job to run the conversion through SQL Server to update the working table and target table values.

Prerequisites to ETL Jobs

Step 1 - Parameters for ETL Tool

Within Administer > Setup > Parameters a default for ETL Tool is set at initial system setup. This defaults to a Project / Release prior to any direct assignment to the ETL Tool within each Release. At least one Release for ETL Tool must set to Syniti Migrate for the menu path and page for ETL Jobs to be visible. If the ETL Tool is set as SAP Data Services for all Releases in the Project, ETL pages are hidden from view (Not necessary). As shown below, the default in this project is set to Syniti Migrate for the ETL Tool.

Note

Datasources use the Parameters ETL Tool value to determine if the XML sections are visible. Should the project use both SAP Data Services and Syniti Migrate for ETL Tools, default the Parameters page to SAP Data Services so that the Datasources can produce the XML scripting needed for building datasources in SAP Data Services.

Step 2 - Project Setup - Release must assign ETL Tool

At least one Release within a Project must assign the ETL Tool as Syniti Migrate in order for the ETL icons and processing to be visible in the system. If all Releases are set to SAP Data Services, the ETL Jobs pages are not be visible for use.

Note

It is possible to work on a single project that includes both external and Syniti Migrate ETL tools.

Project Setup: Edit Release: ETL Tool

Step 3 - Mapping Fields to produce ETL Job Tasks

Once the field mapping is complete, click the ETL icon stored in the Mappings page to initiate the build process. Refer to section Migrate > Mappings > Mapping the fields of a Datasource > Mapping When Using Migrate ETL Tool for details of this process.

This process resides in a few sections of Syniti Migrate as shown below -

• Parameters page - Designate Syniti Migrate as the ETL Tool

• Mapping of the Datasource to target table - Map a field of fields as a Type = Rule to initiate the ETL Jobs Tool for the object. Setup one or more Rule Stored Procedures for the field. See Details for this process within section Migrate > Mappings > Mapping the Fields of a Datasource > Mapping When Using Migrate ETL Tool in section Mapping Action Details for a Rule using Migrate ETL Tool..

• Mappings Dataset - Execute the ETL Job within Job Queue to run the conversion of data process. See Details for this process within section Migrate > Mappings > Mapping the Fields of a Datasource > Mapping When Using Migrate ETL Tool in section Execute the ETL Job queue

• Mappings Dataset - View the ETL Job setup with a click on the ETL JOBS button. See Details for this process within section Migrate > Mappings > Mapping the Fields of a Datasource > Mapping When Using Migrate ETL Tool in section View ETL Job Sequence

Users & Roles in ETL Jobs

The Data Migration Project Lead, Deployment IT lead, or the Migration Team Developers are responsible for managing ETL Jobs as part of the Syniti Migrate ETL Tool process.

View ETL Transform Job List

Once the conversion has been executed within Migrate > Mappings, the user may view ETL Transform Job List by following pathway Migrate > ETL Jobs from the Syniti Migrate Homepage as shown below. This page allows for a view of all ETL jobs and overall management of the timing for runs. The grid for this page lists in ascending order each Project / Release / Dataset / Version. Within this page, the entire conversion process for a Version may be monitored.

The ETL Job List panel displays the following fields that are all stored in other sections of the system:

Note

RICEFW stands for Reports, Interface, Conversion, Enhancements, Forms, and Workflow. These are forms produced by SAP during processing. In this scenario, a ticket number is entered to represent the form.

View details for the ETL Transform Job List

From this page the user may select to view the ETL Job details by clicking the Edit icon for the row of the ETL Transform Job List. The Dataset Transform level of ETL Job displays and provides links to all levels of the ETL for this dataset. The Refer to section ETL Tasks at the Dataset Level for the details.