Mapping the Fields of a Datasource

This topic relates to the following sections:

• Mappings

• Mapping When Using Syniti Migrate ETL Tool

• Mapping When Using an Exterior ETL Tool

• Build XML Code for Source Data

• Download Individual XML Files

• Create System Views

• Encrypt Decrypt and Mask Fields

About Mapping Fields of a Datasource

Within the Mappings page of Syniti Migrate the mapping of source to target to translate the legacy to target data is produced within each Target Source. The Target Source (Sources) are added to each Target table as shown in the image below. Each of the active relevant fields of the target are mapped from the source table to the target table through the mappings process. This builds the framework for the rules to translate data.

View the following page by navigating to Migrate > Mappings on the Syniti Migrate Homepage. The Mapping page is linked through each Source table.

Prerequisites for Mapping

There are a number of prerequisites to actually mapping fields in a Target Source. Refer to section Migrate > Mappings for details of dependencies upon other sections of Syniti Migrate prior to executing the Mapping process.

Since the Release determines the ETL Tool for use in the object conversions, it is important to review the ETL Tool types and determine which set of processes are relevant to the current project. Refer to section Migrate > Project Setup > Creating & Maintaining Releases for details of ETL Tool determination and setup.

Mapping the Fields

Within the Sources panel of the Mappings page, mapping of legacy to target fields is completed. This is where the conversion of data is documented and eventually built within the Working Database. The details for this process vary depending upon which ETL Tool is selected for the Release of the Project. Both types of mapping are discussed in detail as they vary, and the general information is handled in this section.

From the multi-panel page, mappings are completed from within the Sources panel.

First, select the data source and click the Mapping   icon in the page above. A page containing all of the active for mapping fields display. A breadcrumb pathway is displayed to provide the Mapping team identity of the Release, Dataset, Target for this mapping.

This page is divided into three panels, the Mappings panel, the Data Element Details panel, and the Data Mapping Details panel.

Note

This list does not display the field ZSOURCE as a value as the value is defaulted from the datasource selected for this mapping. Also, field ZSOURCE is never counted within the number of fields mapped on the main Mapping page or within any Metrics reporting built from this effort. The ZSOURCE field is used for the conversion team's benefit and should not affect overall reporting and counts.

Mappings panel

The Mappings panel contains the list of fields from the Migration tab of the Dataset that were marked as Active for Mapping or Element Active. There are five Map Action icons that can immediately set the lower panel (Data Mapping Details) for that type of mapping action.

The Mappings columns are defined below:

View Mapping History

Provided below is an example of a Object History list. The user would click on the View Mapping History   icon for the row to open this page and provide details of the history of changes made to this Mapping.

Note

As with any page grid of data, on the right upper corner, click the Hamburger Details icon to view the options for show or hide column, filter of rows, and column width. This is extremely handy for situations where the data is fluid and changes or when there are many records that are hard to review in the small window provided for the list. Refer to section Page Grid Options for details.

Data Element Details

The fields in this panel are copied from the target table stored in the Dataset and they support the row highlighted in Mappings panel. If the Datasource table stores values for these fields then they will copy into this Mappings panel. To view the details of the Dataset that stores this Data Element, click on the Dataset Design link.

The Data Element Details fields are defined below:

Show Previous Mappings

The Show Previous Mappings button will provide a list of all mappings that use this same field/table. See section Show Previous Mappings below for details.

Data Mapping Details

The fields of the Data Mapping Details panel change depending upon the Mapping Action selected. The Mapping Actions options are available from a pick list, as depicted in the list below:

Note

Should the field require encryption for security purposes, refer to section Encryption, Decryption, and Mask Fields for details of this process.

These are a few examples of Mapping Actions used and the fields included:

Mapping Action Default

This mapping is the easiest as it contains one mapping field; a default value.

Mapping Action Details fields for mapping Default as listed:

Mapping Action Copy

This mapping copies the value from assigned Datasource / Table / Field for every record.

Mapping Action Details fields for mapping Copy as listed:

Mapping Action Rule

A Rule generally pulls together tables using joins and filtering data to produce a list relevant for update with a value.

Note

Should the field require encryption for security purposes, refer to section Encryption, Decryption, and Mask Fields for details of this process.

Mapping Action Details fields for mapping Rule as listed:

Mapping Action Xref

This simple Xref pulls the legacy value into a XTVALUEMAP table for users to compare and determine the target value from the Check Table assigned. Once the Check Tables are built in the database through section System Views, the field Value Mappings provides a count of Records Updated / Total Records and contain a link to the Value Mappings page where these values may be cross referenced.

Mapping Action Details fields for mapping a Value Mapping Xref as listed:

Mapping Action RuleXref

This mapping is designed to cover the situation where a value mapping is made up of multiple key fields, or possibly a view is required to build the list of records to update. This mapping allows for building a view in the database that work to update from legacy value to target value.

Mapping Action Details fields for mapping a Value Mapping Xref as listed:

Mapping Action Details when using an Exterior ETL Tool

If the ETL (Extract Transform Load) process is set to SAP Data Services, mapping involves the Build and View of XML files to import to those external ETL Tool sets. Refer to section Migrate > Mappings > Mapping the Fields of a Datasource > Mapping When Using an Exterior ETL Tool  for the details of mapping fields for the Actions of Default, Copy, XREF, Rule, Rule XREF, or ENRICH as example.

Mapping Action Details when using Syniti Migrate ETL Tool

The ETL (Extract Transform Load) process within Syniti MigrateApplication - Within Mappings, the ETL process is executed by using the ETL TASKS   button which provides for attaching the associated Stored Procedure (SP) to the Rule. This SP runs in the ETL process within Syniti Migrate as part of the conversion. Tasks are added to the rules using the ETL Tasks button which is not active until the Map Status shows as Complete.

Refer to section Migrate > Mappings > Mapping the Fields of a Datasource > Mapping When Using Migration ETL Tool for details of mapping Rules when the Syniti Migrate Application runs the ETL processing.

General Tools for Mappings pages

Table and Field Links

While viewing a mapping, should the user want to view the Table data, click Table Link   icon to view a window of the Top 50 rows of data as shown below.

Note

There are a few field formats that may cause issue with this link - the SpacialLocation and UUID should convert to NVARCHAR in the translation from system to snapshot table data, however, should this link fail with an error as provided below, then it would be wise to properly translate these fields as text (CHAR or NVARCHAR).

"exceptionMessage": "The 'ObjectContent`1' type failed to serialize the response body for content type 'application/json; charset=utf-8'.",

    "exceptionType": "System.InvalidOperationException",

To view the Distinct values of the Field selected, click Field Link    icon to view a list of distinct values for this field.

Value Mapping for Check Tables

The process for building the Value Mappings for each Check table is performed directly within the Migrate > Mappings page. Refer to section Migrate > Mappings > Value Mappings by Object for details of this process.

For Map Action of Xref, the Check Table Value Mappings stored within XTVALUEMAP table for this mapping is provided. Directly below the Check Table field is a count for Value Mappings:  Completed / Total as well as a link to the Value Mappings page.

Note

Within Administer > Value Mapping Config there is an Active for Value Mapping flag used when the table is LARGE and not worthy of the standard Value Translation process (This field is not directly tied to same field in Datasource table details). When this flag is set to not active, the result removes all value mapping for this table. So, within Mappings pages, the Value Translation link is hidden and this table is no longer be visible in the Migrate > Value Mappings page. This table is not a part of the XTVALUEMAP process, and the user must manually build a table that supports value mapping for this table. Refer to sections Migrate > Value Mappings and also refer to Administer > Value Mapping Config for details of these processes.

Show Previous Mappings

To provide more details for mapping, the system provides a button within the Mapping page to SHOW PREVIOUS MAPPINGS for fields that are mapped in multiple datasets. Click this button to view mappings for this data element in other Releases / Datasets. For Mappings where the Map Status is New, the user may copy a previous mapping into the current mapping.

Note

Only mappings where the Map Status is Complete is included as they are finalized.

Click to highlight the previous mapping, and then click COPY THIS MAPPING button to copy into the current mapping.

The system produces a message ' Copy was Successful. Please review before saving.'  The current mapping contains a Map Status is In_progress. Once review is complete, click Save   icon to complete this process.

Relevancy and Source Filtering information

Information regarding the relevancy of data to move to Target, or rules that would filter the data imported from Source to Working table may be stored in the Comments fields as well during a mapping session. Use the ZLEGACY column and the ZACTIVE column to store this detail into field Comments. The Comments field values are copied into the ETL tool, and displays on the Object Rulebook that is provided to the client during each Mock Conversion.

The relevancy for Target rules are built manually by the developer and added to the ETL tool as views for query mapping.

When finished with edits, click the Save  icon to complete.

Maintenance of Map Status and Build Status

The Mapper updates the Map Status during the mapping of fields from New to Design Required, In-Progress, or Complete. The Map Status of Complete initiates icons on the Build Status level as shown below:

The three icons next to Build Status represent - Mark In-Progress, Mark Complete, and Reject. If the user selects Mark In-Progress or Mark Complete, it updates the values to reflect same.

If the user selects Reject, a window displays providing the Owner's email address. The user must fill in the Rejection Comments. Once complete, click on the Mail   icon to send the email to the Owner.

The Rejection Comments History may be viewed by a click on the Rejection History   icon. The interaction discussion and voting are all possible within this window.

The BUILD status is automatically set to complete for Simple Mappings (Copy, Xref, Default, Not Used, Internal, etc) when you GENERATE the XML or when the Migrate ETL is executed. It is reset if the Mapper unlocks the mapping to make changes. In that case, the Developer manually changes it back to complete after they review the new mappings and make any manual updates to the ETL jobs.

Lock and Unlock a Mapping

Unlock Icons are hidden at entry of the mapping. Once the mapping has been successfully marked as Map Status = ' Complete', the Unlock  icon displays. This indicates that the data has been stored successfully, and it is now Locked for changes. The system changes to 'Lock' the data. It displays as all fields are Display Only.

Should there be changes for this mapping, the user clicks Unlock  icon to change the fields to active and allowed for change. A message displays to request approval for this change as shown below:

Click OK to proceed and unlock the field values, or click CANCEL button to exit without making changes.

The Unlock icon is hidden, the SAVE icon is inactive, and the fields display as active for changes within the Data Mapping Details panel. The user may now make changes to the fields and Save the data once more.

Note

As part of the automation of this change to unlock the mapping details, the system automatically updates the fields -

Map Status = 'In Progress'

Mapped By = current user

Mapped On = current date

Build Status = 'New'

Once the Edits have been completed, and the SAVE icon is again activated, click SAVE  to post the new changes for this Data Element.

Should the user want to cancel without saving their changes, they should simply leave this page, and it reverts to last saved entries again. The page issues a warning message for the user to make sure that they want to leave without saving the changes. The page returns to the main Mappings page, exiting the process.

Note

The status for the current mapping remains as In-Progress and the field remains as Unlocked. To revert to original - the user must edit and update these manually.

Removing or Adding Fields for Mapping

Should the situation arise where a field is set as part of the migration of data, but once mapping is being built that field is determined to not be relevant, the user may remove the field from the migration process.

Also, during the course of a project, a field might be added to a target table that is relevant to the migration and needs to be added. Refer to section Removing or Adding Fields for Mapping for details of this process that has updates in both the Dataset Design and in the Field Mappings pages.

Subsequent Steps

Once the mapping of the source table is completed, the next step is to Create System Views.