Create & Maintain Datasets
  • 21 Jun 2024
  • 18 Minutes to read
  • Contributors
  • Dark
    Light
  • PDF

Create & Maintain Datasets

  • Dark
    Light
  • PDF

Article summary

This topic contains the following sections:

About Datasets

Within the Dataset Design section of Syniti Migrate, the objects are defined by Subject Area and Dataset. This information is imported to Syniti Migrate from Knowledge Platform.

Prerequisites

  • The Datasources should be defined for the objects prior to creating a dataset. Datasource is a required field and it locks and does not allow for edit after the dataset has been imported into a Mapping. Refer to section Catalog > Datasources for details of defining Datasources.

  • The Subject Area must be created in order to create a Dataset. Many Datasets can be assigned to a Subject Area.

Dataset Objects

View the following page by navigating to Catalog > Dataset Design from Syniti Migrate Homepage.

On the multiple panel Dataset Design page, Subject Areas are stored in the left panel and Datasets are stored in the right panel. The Dataset panel displays a list of Datasets within the highlighted Subject Area.

Note

The count of rows for the panel is displayed within parenthesis next to the panel title.

Dataset Design: Datasets Panel

The Dataset grid displays the following fields:

Heading

Description

Index

Sequence of the Datasets. Value increases by factor of 10 for each new entry.

Prefix

Acronym or simple name of Dataset - This prefix is used to build out the XML and must be unique and limited in size. (See Note below)

Name

Name of the Dataset.

Description

The description for this Dataset.

Dataset Type

Select the Dataset Type. Refer to Data Types and Use below.

Datasource

The target system to which the migration data is loaded.

Tier

List the Tier of this Dataset from the drop-down list of options. (Ex: Foundational, Master Data, Conditional, or Transactional)

Security Classification

List the Data Classification of the Dataset from the drop-down list of options. (Ex: Internal Only, Public Data, or Sensitive).

Status

The current status of the Dataset.

Elements

A count of the Data Elements selected for the Dataset.

Note

If there are policies, a count displayd, otherwise blank.

Active

Toggle to set status of the Dataset for future-state design mapping and migration.

Note

Defaults as Active.

RICEFW

The identity of this Dataset to the project.

Geographic Scope

List the Geographic Scope of the Dataset from the drop-down list of options. (Ex: Global, International, North America) This field is hidden.

Complexity

List the Complexity of the Dataset from the drop-down list of options. (Ex: Low, Medium, High, or Very High)

Owner

User responsible for this Dataset.

Developer

Developer responsible for this Dataset.

Business SME

Business user subject matter expert responsible for this dataset.

Technical SME

Technical user subject matter expert responsible for this dataset.

Note

The Prefix is vital to the build of XML and build of the conversion as it shortens the name - it cannot contain special characters or spaces.

Sliding Divider Bar

Throughout the Migration system, pages provide the ability to move the divider bar between panels for a Vertical break or Horizontal break. As one example of this benefit, Dataset Design provides the user ability to move or slide the vertical page break from left to right and back. Refer to section Migration User Interface > Menu Structure for more details of this benefit.

Add a Dataset

On the Dataset panel, click the Add   icon to create a Dataset and launch the Edit Dataset Details window.

This is a multiple panel page where the left side panel stores the Dataset Detail information including but not limited to Name, Target System, Complexity, Attributes, Stakeholders, and Review Status. The right side of this multiple panel page contains associated Table and Field or Data Element details. The Dataset information is built within the Syniti Knowledge Platform and then imported into this section for use in the migration of data from legacy to target systems.

Within the Dataset Details panel, field values may be entered as shown below for the details of the Dataset. Fields within section Details remain static in nature. The remaining sections of Dataset Details panel alters based upon configuration of these sections including - Data Classification, Additional Details, Technical Details, Design Details, and Content Review. The section for Audit maintains field values automatically for use in metrics and reporting.

The field Dataset Type is a configurable field that determines the use of this Dataset (Ex: Report or Table) and the fields that are relevant with formatting. Once this field is updated, the remaining fields on the page updates to reflect the relevancy of the Dataset Type. The fields are set as Display, Optional, Required, or Hidden based upon this setting.

Note

The Dataset Type values are configured within Administer. View section for Adminster > Advanced > Dataset Types for more details.

Dataset Design: Adding a dataset

Field definitions for Dataset Details:

Fields marked with the asterisk (*) symbol are required fields.

Section

Field

Definition

Details

Subject Area*

This value is defaulted and for display only for the related Subject Area short name.

Name*

Name of Dataset. This is typically a business object, Ex: Article (Master), Purchase Order, or Business Partner.

Description*

The long, descriptive name or description of the object. (Ex: Article Master or Structured Articles)

Dataset Type*

Dataset Type provides a drop-down list of available types including Datalake, User Interface, and Transaction. For more details, navigate to Administer > Advanced > Dataset Types.

RICEFW

The identity of this Dataset to the project.

Datasource*

The Datasource from which the active objects are mapped. The system is defined in Deploy as a Datasource. (Ex: ECC)

Note

This field may only be changed prior to importing tables/fields. See Note below.

Active

Toggle to set status of the Dataset for future-state design mapping and migration.

Note

Defaults as Active.

Index*

Sequence of the Datasets. Value increases by factor of 10 for each new entry.

Review Status

Enter the current review status. Refer to Review Status section below for details and use.

Note

Defaults as Not Started.

Additional Notes

Text line to add notes for this dataset.

Data Migration

Is Migration Specific

Toggle to mark record as relevant for Migration for internal object types that should not be included in the project metrics. See NOTE below.

Note

Defaults as Inactive and accessible by Developers/Admin.

Migration Prefix*

Acronym or simple name of Dataset. (Ex: "AM" for Article Master. Used to identify the Dataset in all Views and Reports) See NOTE below.

 ** Fields in sections below are configurable **

Security Profile

Security Profile

Security Profiles in Migrate allow users to extend role-based security settings on an individual asset or component, such as Datasets, Datasources, and Releases. When a security profile is applied to an asset, only users associated with user groups and roles designated on the security profile have visibility or access to that asset. For example, if a security profile is applied to a release, then the subject areas, datasources, reports, mappings, ETL Jobs, Metrics, and all the granular details of a release are restricted from other users.

Refer to Security Profiles and Security Profiles in Migrate for more information on implementing and managing security profiles.

Data Classification

Tier

Tier level of the data as defined in the drop-down list including Foundational, Master Data, and Transactional.

Note

This field is List Box configurable within Administer > Advanced > List Boxes. See note below.

Complexity

Level of complexity from 1(Low) to 4 (Very High). This is used to determine future planning of resources and hours. For more details, navigate to Administer > Advanced > Dataset Complexities.

Security Classification*

Security Classification ranging from Public Data or Internal Only based on the security needs of the data in a drop-down list. Determines the accessibility to data by users and can classify data such as personally identifiable information (PII) to help establish safeguards around viewing the data.

Geographic Scope

Identifies if the field applies to the global solution or the local country as a drop-down list.

Note

This field is List Box configurable within Administer > Advanced > List Boxes.

Stakeholders

Owner*

The name of the business team user who is responsible for the data that is being migrated in a drop-down list or freely entered as text. See Stakeholder Details

Developer

The name of the Syniti team member who translates the needs of the business into the software capabilities and executes in a drop-down list or freely entered as text. See Stakeholder Details

Functional Lead

The name of the SI counterpart to this Dataset listed in a drop- down list or freely entered as text. See Stakeholder Details

Business SME

The name of the businessperson who is responsible for integrity of this Dataset listed in a drop-down list or freely entered as text. See Stakeholder Details

Technical SME

The name of the IT businessperson who is responsible for integrity of this Dataset listed in a drop-down list or freely entered as text. See Stakeholder Details

Additional Details

T Code

The transaction code used for the table.

Technical Details

Load Type

Drop-down selection for the type of load Ex: Batch, Manual, or ALE 

Refer to section The Data Migration Process > LTMC - SAP S/4HANA Migration Cockpit  for details of option - LTMC Staging

Note

This field is List Box configurable within Administer > Advanced > List Boxes.

Relevancy Rule

Information about how the data is filtered for selection before being presented in the Dataset. This often details the technical criteria for the "Active" records shown on a page, or in a report, or what records are affected by an Interface.

Note

Used for PMO Dashboard metrics

Load program

Text entry to provide load program details.

Note

Used for PMO Dashboard metrics

FUT Completion Date

Enter the date for completion of Functional Unit Testing.

Note

Used for PMO Dashboard metrics

TUT Completion Date

Enter the date for completion of Technical Unit Testing.

Note

Used for PMO Dashboard metrics

Attributes

Various Fields

This section is managed within Administer > Advanced > Attributes.

Design Details

Workshop Date

Dataset (target) Design Date on which the design is considered complete and ready for release for the mapping phase.

Workshop Notes

Any additional notes can go here such as the details of use.

Design Date

Date Design is started.

Design Complete

Date Design is completed.

Develop Date

Date Development is started.

Develop Complete

Date Development is completed.

Test Date

Date Testing is started.

Test Complete

Date Testing is completed.

Audit

Various fields

These are the system generated values for metrics reporting and auditing the work.

When finished with edits, click the Save  icon to complete.

The Is Migration Specific Toggle - Visible within the Subject Area Details, Dataset Details and Data Element Details of the Syniti Migrate system. These toggles mark the object type as active for internal Migration Project team use such as Central Wave or Relevancy Targets. They are toggled as active to be excluded from the overall Migrate metrics calculations and to appear as if they are not "In Scope" objects for the Project.

Note

The Migration Prefix is vital to the build of XML and build of the conversion as it shortens the name - it cannot contain special characters or spaces. The Migration Prefix must be unique throughout the system. If the user enters a duplicate value, an error message displays during the save as shown below. This field is accessible to Developers or Admin roles.

Edit a Dataset

Once a dataset has been added to a Subject Area, it may be edited. Click the Edit icon for the dataset from within the Dataset Design page to modify the fields. A few fields like the Datasource field are locked and cannot be altered once the Dataset is created. To add Tables or Columns to an existing dataset refer to section Maintain Tables and Fields. When finished with edits, click the Save  icon to complete.

Delete a Dataset

Although the Subject Area details provides a Delete button, it is important to only delete a table that has not yet been mapped. The Active toggle marks the table as Inactive and prevent users from adding to the Mappings. This way history of this table may be stored in the system.

However, in the event that a Dataset Table was added in error, it may be removed by clicking the Delete icon for the table in panel Tables in the Dataset Details page. A Validation Warning message displays to warn that interfaces are deleted. This indicates that all Mappings, all Reports, all ETL tasks are deleted from the system should the user click on CONTINUE DELETE button. This action can not be reversed, so it should be used only in situations where the Dataset should not have been added.

Dataset Design: Delete a Dataset that is mapped

Dataset Types and Use

Within the Dataset Design, Datasets are used to identify various different types of data. The Dataset Types are configurable within Administer > Advanced > Dataset Types. Examples are provided below:

  • A Datalake (centralized repository that allows you to store all your structured and unstructured data at any scale) stores data that may be part of the active client system.

  • A Report stores data elements both from the target system as well as derived data elements to document and build a report.

  • An Interface stores the data elements used to transfer information from one system to another.

  • A User Interface covers the process of migration of data from legacy to target systems.

  • A Procedure may be used to document the steps of a procedure used in the daily processing of transactions.

  • A Transaction documents the specific use and data within a specific transaction of business use.

Note

Dataset Types are configurable and have specific attributes documenting an interface - an API , map data into this.

Tier Field

See below for field definitions for the Tier Pick list:

Tier

Definition

0

Foundational - Data used to define, support or create other data. This includes configuration and reference data and metadata, which helps support the data quality and adherence to business processes in the target system

1

Master Data - Data that defines the principal business objects shared across the enterprise, such as business partner and material/article data

2

Conditional - Data that is dependent on and extends the Master Data; and is necessary to support business transactions, such as pricing conditions

3

Transactional - Data is produced or updated as the result of business process transactions, such as sale orders, purchases orders and inventory movement

4

Informational - Data that is ancillary to and descriptive of business transaction data. This data supports reporting, metrics, and business intelligence

** Datasource field

The system from which the active objects are mapped. The system is defined in Migration as a Datasource and maybe set as a Source or Target System on the Datasource Details page Ex: ECC, S4H, MDG. The Datasource is not be the Snapshot Datasource (tgtECC, tgtS4H), but instead the Source or Target Datasource (ECC, S4H).

Note

The Datasource field may only be changed prior to importing tables/fields. Once there are active tables and data elements, this field changes to Display only. Should the situation warrant changing the Datasource after the Dataset is imported to a Mapping Release, it would be wise to delete the entire Dataset and start over. Therefore, the user should determine a valid system and build that Datasource within Migration prior to any Mapping of source data.

When finished with edits, click the Save icon to complete.

At this point, the right-side panel becomes active and allows for adding new tables and fields to the dataset.

Central Wave Datasets

Central Wave depends on creating a Subject Area and Datasets specific to "Central Relevancy". These are Migration only objects similar to the Migration Data Elements that are mapped to determine the relevancy of objects like Sales Orders or Purchase Orders. These are not relevant to the Dataset Designer (if they are the SI who’s concern is for the objects loading data).

These Central Wave Datasets do not display in Dataset Design like the normal datasets. The Migration toggle assists with Security to make them hidden from Designers who do not need this information. These Migration toggled Datasets are also excluded from all Metric calculations.

Add Supportive Information for a Dataset

The left side of the Dataset Details page contains the Dataset Detail Fields. These fields are supportive of the Dataset Design, usage, classification, and review. Each section of this panel stores valuable information. At the header of this left side panel there are quick use icons as shown below:

Icon

Name

Use

Subscribe

This icon adds the user name to the Subscribe list for email notifications when there are changes made to this object.

Save

The Save icon saves the current entries.

Cancel

The Cancel icon leaves the current page without saving changed data.

Help

The Help icon links to the page in Syniti help text to provide detailed explanation of this section and use.

Options

This icon opens to provide user access to View History, Data Definition Report , Export Dataset, and Import Dataset- See individual options below for details of each.

Link

This icon opens to a page within Syniti Migrate that supports the details of the field value. The Stakeholder fields contain active or inactive links to the details for that Stakeholder.

Additional Options Icon

Within this icon of the Dataset there are four options:  Data Definition Report, View History, Export Dataset, and Import Dataset. This is a quick step process that allows the user to perform tasks without having to modify using the Edit process.

Data Definition Report - Provides a list of the active data elements by table and index. Refer to section Data Definition Report below for details.

View History - This option provides the history of changes for the dataset. Refer to section View History of Changes below for details.

Export Dataset - Allows for export to a local file all dataset details for use in development. Refer to Import / Export Dataset and Mappings below for details.

Import Dataset - Allows for import from a local file all dataset details for use in development. Refer to Import / Export Dataset and Mappings below for details.

Subscribing to a Dataset

At any time after the Dataset is added and saved, a user may Subscribe to the Dataset for notifications. The Subscribe icons add the user name to the Subscribe list for email notifications when there are changes made to this object. As the user clicks on the bell icon, it changes from grayed out to highlighted and a message displays "Subscription Added."

Icon

Name

Use

UN -Subscribed

This icon adds the user name to the Subscribe list for email notifications when there are changes made to this object.

Subscribed

This icon shows that the user is subscribed to this Dataset for email notifications of changes.

Note

Subscribing does not require that the Dataset be changed or SAVED. The action of clicking on the Bell updates the status to/from Subscribed.

As changes are made to the Dataset Details (Left Vertical list of fields) in the Object, notification emails forward to all subscribers. This process only takes place when the Dataset's Review Status has been set to either Complete or Re-Review.

Also, if the Dataset is deleted, independent of the Review Status, a notification is sent to all subscribers.

Note

Email notifications are sent from skp-no-reply@syniti.com. To prevent notifications from going to a spam folder, add skp-no-reply@syniti.com to the contact list.

Note

Emails are sent to the email address set up on the user's associated Employee record. Emails are queued, and then sent once per minute.

Dataset Design: Dataset: Subscribe to notifications

The email looks similar to the example shown below:

Catalog: Dataset: Notification Email sent to originator

Data Definition Report

Provided below is an example of a Data Definition Report. The user would click Data Definition Report  stored within the Options icon in the header panel to open this window and provide details of the changes made.

Note

This report may be downloaded as a CSV file to a local drive by a click on the Hamburger  icon to select " Export all data as csv". The Columns may be reset to default by a click on Reset Columns.

Dataset Design: Data Definition Report

View History of Changes

Provided below is an example of a Object History list. The user would click View History  stored within the Options icon in the header panel to open this page and provide details of the history of changes made to this Dataset.

Note

The Download icon will produce a file on the local drive with details of this table data. The Search icon will provide filtering of data.

Dataset Design: History

Note

As with any page grid of data, on the right upper corner, click on the Hamburger page Details icon to view the options for field in or out, sorting, and Export all data as csv. This is extremely handy for situations where the data is fluid and changes or when there are many records that are hard to review in the small window provided for the list.

page Options: Hamburger Icon

Import / Export Dataset and Mappings

During a multi-release project, a Dataset may be used multiple times within multiple Migration systems, and therefore it becomes necessary to export the data into a file and then import the same Dataset information for use in another Migration system. It exports only the setup of the Dataset to include the Tables, Fields, and the Data Element details. The import of Dataset data is inserted to an existing Subject Area within the new system or new release of same system. This process provides a shell for many large objects as well as providing a more accurate way to repeat an objects migration over many releases.

Export Dataset and Import Dataset reside within the Options icon in the header bar.

Refer to section Catalog > Dataset Design > Create and Maintain Datasets > Import / Export Dataset and Mappings for details of this process.

Data Element Priority using Drag and Drop

The Dataset Design team must take the time to properly order the data elements in the Dataset Design pages. Elements should be arranged in the same order they appear on the user Interface pages used to create or maintain the records in the Target System. The order of the elements defines the order in which the fields appear in Mapping workshops, in the Validation Reports, and across all the pages in Migration.

It can be difficult to Map and Validate a dataset which contains the fields in a random order. For example, the address fields should be grouped and prioritized so that an address is easy to validate. Having the address fields out of order and intermingled with all the other Customer fields makes it very difficult to work overtime. This is one of the most important aspects of a good Dataset Design, and extraordinary care should be spent to make it right from the beginning of the project.

Note

Easily edit the order of Data Elements in the multi-panel view by selecting(highlighting) the Data Element and dragging and dropping into the desired order. This automatically adjusts the index of the Data Element(s) reordered. If any sort order has been enabled for a column, the drag and drop functionality is disabled. Removing the sort order enables drag and drop functionality again.

Dataset: Data Elements: Drag & Drop

Dataset Objects 

On the right side of the Dataset Design Details page, there are three tabs to pages that add and maintain the supportive information of this object (Data Elements, Tables and Fields, Migration). These pages are used as follows:

Tab

Responsible

Description of Use

Data Elements

Dataset Designer

These elements may be migration data fields or may be other elements related to the overall governance of data.

Catalog > Dataset Design > Maintain Data Elements

Tables and Fields

Developer

The page contains 2 panels to store the table and field data for relevant use in both Data Elements and Migration pages.

Catalog > Dataset Design > Maintain Tables and Fields

Migration

Developer

This is the result of adding a table with fields to the Tables and Fields tab. The ZFields display on this tab.

Note

This tab only displays if the Dataset has been imported into a Mapping Release.

Catalog > Dataset Design > Maintain Migration

Depending upon the Security role of the user, one or more of these panel options are available for display or edit.


Was this article helpful?

ESC

Eddy, a generative AI, facilitating knowledge discovery through conversational intelligence