ETL Version Selection

This topic relates to the following sections:

• Execution of the ETL Jobs

• ETL Tasks at the Release Level

• ETL Tasks at the Dataset Level

• ETL Tasks at the Target Level

• ETL Tasks at the Source Level

About ETL Versions

ETL Versions provide multiple Milestone versions of ETL packages that are operating against the same target objects so that the user may continue to make changes to the DEV Milestone version for a future mock while running a different Milestone Version that is locked in place. The version is combined with use of Reporting Environments to separate the ETL Job Tasks by Milestone. If changes are made to add new rules, or remove tasks within the DEV Milestone, the other Milestones are not be affected. It is only when the DEV Milestone is copied to one of the other Milestones that the ETL Rules will all be the same.

The benefit of ETL Version Selection is that once a Milestone has completed, the list of Job Tasks run with durations, counts of records, and timing is available for review. This data is all stored a Migration table identified by the Version ID. Also, the user may continue to run the ETL Job list against the DEV Milestone during a Milestone (Ex: Mock1, Integration Test) and test changes without affecting the work of the Milestone. This process allows for segregation of the information for the Developer.

Prerequisites to using ETL Versions

Project Setup at Milestone level

Within the Project Setup of the Release, each Milestone is required to select a Reporting Environment. As the image below shows, each Milestone is assigned to an Environment (Reporting Environment). In this project, there are two Reporting Environments DEV and LOAD. Refer to section Administer > Setup > Environments for details of the setup and use of Environments.

Promote Object Data from DEV Environment to LOAD Environment

The Dataset tables and views must be copied into the other Environments in order for the process to work successfully. Refer to section Migrate > Working DB Object Promotion for details of the database copy process.

Setting the Version for an ETL Job

The Version banner is located within ETL Job Tasks page as a header banner that will provide the current Milestone Version. There is always a DEV Version as it is the base for Development - The user should setup a Milestone named for this DEV version so that they can develop and run the ETL in that version during a Milestone. Each Milestone other than the DEV Milestone should be assigned to the LOAD Environment. In this way, the Milestone ETL Job and updates will be separate from the Development ETL Job and updates.

The DEV version is the default where all work will be completed. A copies of the remaining Milestones may be initiated through the Version Selection window. At this point the user may select another version by a click on the drop down, or they may create a new version (by copying from the "DEV" version to a new Milestone version.

The drop-down box on the dialog window will contain an option for "DEV", and items for all of the Release's Milestones.

Note

The versions yet to be created will be gray in color while the active versions are in black text.

Note

The color of each of the versions is inherited from the Milestone's Reporting Environment.

If a Version has not yet been created for the selected Milestone, the user will need to create a copy before they're able to switch to view the version.

The copy feature of this process will complete the following:

• If any component of DEV package is created, the entire DEV package will now be created.

• If the DEV package is copied to a Milestone Version, the entire DEV package will be copied, no matter from where the copy is triggered.

If a Version has already been created for the selected Milestone, the user will be able to Switch to the Version, or re-copy from Dev (which will overwrite).

To Select the version - Click on the SWITCH TO THIS VERSION button

To Copy and Renew - Click on the COPY DEV TO INTEGRATION TEST button to initiate the copy process.

The result will display the new version selected for the ETL Job Tasks as shown below:

When Job Tasks are viewed for the Source Transform, the Milestone name is added to each of the tasks and the Target and Datasource values reflect the Milestone's Reporting Environment working datasource (In this case WRKFINL for Load).

ETL Version Selection within Data Mapping Details

Within the Target Source Mapping details, each ETL Task setup for the field mapping will provide the version copies. Select a field mapping and then click on the ETL Tasks button to view ETL Tasks built for each Milestone. These all are built from the Copy feature from DEV to the Milestone.

As shown below, this Rule is part of a migration that currently has a DEV and Integration Test Version. Both versions of the same rule are displayed one additional version are created for the object.

ETL Version Selection as Part of the ETL Job List

From within the Migrate > ETL Jobs page, the user will see the version as part of the list of jobs running in ETL.