Adding & Editing Datasources

This topic relates to the following sections:

• Initial Setup - First Week

• Datasources

• Adding & Editing Applications

• Adding & Editing Instances

• Adding & Editing Tables or Columns

• Import Metadata

• Verify Migrate Database Upgrade

About Datasources

This section provides details for the maintenance of System Datasources. This section does not discuss the process for maintaining Working Datasources or Snapshot Datasources. A Datasource is defined as a connection to a system at a specific level to provide the means of transferring data between that datasource and the client database.

View the following page by navigating to Catalog > Datasources from the Syniti Migrate Homepage. On the Datasources multi-panel page, the data is filtered by Applications, Instances, and Datasources. The wizard provides guidance for building the datasource correctly and to prevent adding unnecessary field values or leaving valuable fields blank.

This section refers specifically to the System Datasources. The remaining datasources are built in the following sections of the system:

• Refer to section Administer > Setup > Development Areas > Maintain Working Datasources to Add or Edit a Working Datasource.

• Refer to section Administer > Setup > Environments > Adding & Editing Reporting Datasources to Add or Edit a Reporting Datasource.

• Refer to section Administer > Setup > Environments > Adding & Editing Migrate Datasources to Add or Edit the Migrate Datasources.

• Refer to section Migrate > Snapshot Management > Add a Snapshot Datasource to Add or Edit a Snapshot Datasource.

Prerequisites to Adding a New Datasource

• System Datasources must first be set up within the Syniti Knowledge Platform in the Data Catalog. Refer to section Initial Setup - First Week for details of creating the Connector and Connections.

• The remaining Datasources must also connect to a Datasource Server maintained within the Syniti Migrate system. This would include the Migrate, Reporting, and Construct datasources. Refer to section Initial Setup - First Week for details of the database preparation steps.

Datasource Purpose

Although there are five Datasource Purpose options, this section provides the steps for building System Datasources or Reporting Datasources. All other Datasource Purpose options have referred sections as provided above. As the user completes field values, supportive fields become available for entry. The fields vary by response of a prior field.

Datasource Purpose. The response alters the following tabs and required fields to meet the needs of the datasource purpose. There are seven options for selection and each option is discussed below:

• System - These are originating systems Ex: JDE, BPCS, Cloudera, ECC, SRM - Stored within Applications / Instances of the same naming structure. These Datasources pull from Syniti Data Catalog systems. Refer to section Add a System Datasource below for details

• Snapshot - These are the point in time captures of an originating system used as a Source or Target - Stored in Syniti Migration Application (Ex: SRCJDE, TGTS4H) Refer to section Migrate > Snapshot Management to add or edit a Snapshot Datasource.

• Construct Snapshot - This Datasource (SRCCONSTRUCT) stores the Snapshot data from the CONSTRUCT database within SQL Server for Migration - Stored in Syniti Migration Application. The connection is to the Database for migration. Ex: HANA DB, ORACLE, or SQL server. Refer to section Migrate > Snapshot Management to add or edit a Snapshot Datasource.

• Construct - The CONSTRUCT datasource is a system datasource - Stored in Syniti Construct Application (SQL Server connection). Refer to section Add a Construct Datasource below for details.

• Migrate - This Datasource (MIGRATE) is added by the system automatically - Stored in Syniti Migration Application. It stores the Value Mapping data for migration. Once it is modified to add the Migration Database Server, the system automatically pulls in the Tables and Functions of this Datasource. Refer to section Adding & Editing Migrate Datasources for details of this process.

  Note

  Datasource Purpose field locks once the Datasource is saved. The value for this field may be viewed within the grid of Datasources on the Datasources main page.

Add a System Datasource

All legacy system datasources must use the Knowledge Platform Data Catalog system to associate the System Datasource to an Agent and Connection. On the Datasources panel, click the Add   icon to view the list of possible connections for data connectivity and launch the appropriate Details window.

Icon	Description of Use
	Application - The system being used as a source or target.
	Instance - within the Application, the instance is the instance of the application. There may be multiple instances of the same application.
	Datasource - within the Instance of the Application, the Datasource contains the connection information for that datasource. The term datasource implies that one may be a source or target system.

While displaying the Hierarchy list of Datasources, the user should highlight the Instance in which this new Datasource resides. Click Add   icon and select Add New Datasource from the drop down list to display Create a new datasource window.

Once the required datasource is added, access Monitor > Debug Log to view the log messages to monitor the datasource update process. Refer to Verify Migrate Database Upgrade for more information.

Usage Tab

This is a step by step setup that includes adding data to the Usage, Details, Connection, and Attributes tabs of the Wizard process. In this example, the Datasource Purpose selected is System. System datasources import from Knowledge Platform Data Catalog System.

Fill in the Usage fields for this Datasource. 

Note

This datasource example for an ECC source system does not require adding a Snapshot Datasource. The SAP Datasources do not use the Import All field.

Click NEXT   button to proceed to the Data Catalog tab.

Data Catalog Tab

Within the Data Catalog tab connects the Datasource to an active Datasource within Syniti Data Catalog, the user selects the Database Type and clicks Select Value button within the Selected Catalog Asset panel. This produces a list of system Asset Types (databases, tables, or schemas) to select from.

Select to highlight the row of Asset based upon the Asset Name and Asset Type, and then either click Save icon or double click on that line to add it as the Datasource Connection.

The page fields are automatically filled in by the selected Asset.

The Data Catalog fields for this Datasource.

Click NEXT  button to proceed to the Details page tab.

Details Tab

The Details tab is standard for all Datasource Purpose types.

Fill in the Details fields for this Datasource.

Note

Once every required field has been entered, the NEXT button becomes active.

Click the NEXT  button to proceed to the Attributes page tab.

Attributes Tab

Fill in the Attributes fields for this Datasource. There may be Attributes and Stakeholders stored on this tab.

Select the value for Owner which is required, and click Finish button to Save and close the wizard dialog box.

The system clocks and if successful the new Datasource displays in the list as expected. Refer to the section below Verify the Connection for subsequent steps in this process.

Note

If the Import All toggle was not selected, then the user must update the Applications for the tables and possibly columns that need to be imported for this Datasource. Refer to section  Administer > Advanced > Applications for more details of filtering the Import of Metadata to the Datasource. The Reporting Datasource Types do not Import Metadata so this option is not present.

The new Datasource displays in the list for the Instance highlighted. 

Note

During the Save, if the user selects YES to import the Metadata for the new Datasource, there is no Job Queue or Debug Log postings for this effort. If the import is not successful, a message briefly displays stating 'Unable to import metadata due to invalid connection.'  At this point, check the server information and fix the connection.

Note

Once the system datasource is saved, the user may access in Edit mode to update for field Client. This field filters the data replicated into the snapshot table from the system datasource for the client value. See section SAP Datasources filtered by Client below for details of this filter.

Add a Construct Datasource

The CONSTRUCT Datasource is stored within the Syniti Migration Application / Instance as it is set up on the client's database. This datasource stores all of the construction tables and views used in the Syniti Construct system.

To add a construct datasource:

1. Select the Syniti Migration Instance and click Add   icon.

2. Select Add New Datasource from the list. The Create a New Datasource wizard is displayed.

It is a step-by-step setup that includes adding data to the Usage, Details, Connection, and Attributes tabs in the wizard, as described below.

Usage Tab

In the Usage tab, select Construct as the purpose from the Datasource Purpose list of the Datasource Usage section. The CONSTRUCT datasource stores tables that are actively maintained throughout the project, while the SRCCONTRUCT datasource stores snapshot copies of same tables for use during the testing periods. Click the NEXT   button to proceed to the Details page tab.

Note

Once a Construct datasource is created as a Construct Datasource Purpose, the options for the Datasource Purpose list hides the Construct value. Only one Construct Datasource Purpose (Enrich) can exist for a Syniti Migration instance.

Details Tab

The Details tab is standard for all Datasource Purpose types.

Fill in the Details fields for this Datasource.

Note

Once every Required field has been entered, the NEXT button becomes active.

Click NEXT  button to proceed to the Connection page tab.

Connection Tab

The connection information of the Datasource is entered on this tab of the Report Datasource.

Note

Fields on the Connection tab are contingent upon the Server selected and the ETL Tool set in Parameters, so if there were no connection for this Datasource, click Next immediately to advance to save this datasource. 

If the database selected is SQL Server or HANA DB, both Database Name and Default Schema are required.

If the server selected Oracle or Oracle_ODBC Servers, Database Name is hidden, and Default Schema is a required field.

Fill in the Details for this Datasource Connection.

Click the NEXT  button to proceed to the Attributes page tab.

Attributes Tab

Fill in the Attributes fields for this Datasource. There may be Attributes and Stakeholders stored on this tab.

Select the value for Owner which is required, and click Finish button to Save and close the wizard.

The system clocks and if successful the new Datasource displays in the list as expected. Refer to the section below Verify the Connection for subsequent steps in this process.

Verify the Connection

Upon saving the new Datasource, the User should verify that the connection is active. Select the Datasource and click the Connection  Icon to initiate a check against the system. If the connection is successful a message displays " Primary - Connection success."

If the Connection is not successful, the tables and columns do not import and metadata does not process successfully. The Job Queue runs, but no outcome of data is present. This verification is extremely important as well as the setup of the Datasource and Snapshot.

Edit a Datasource

To edit a datasource, click on the Edit   icon to display the Datasource Details page. The details of the datasource depend upon the type of datasource as a System, Report, Snapshot, or Working type.

Note

Once a Datasource is linked to a Snapshot Datasource, this link is permanent and cannot be removed. The value in the field is display only. This way the table data does not become fragmented and incorrect.

Note

Once the User Name and Password have been saved into a Datasource, only the Admin role has access to view the Password going forward. The Show/Hide  icon for viewing the password is not available to anyone who is not provided the Role of Admin. 

The Details page for Datasource is very similar to Instance and Application in that there is a left side panel with Details, and the tabbed section on the right side panel that contains associated object data. The list of fields below are for the System Datasources.

Note

If within the Syniti Catalog System section, the Database Type value is SAPNETWEAVER, then Default Schema must be left blank for the system to properly scan and refresh the table data for the datasource. For all other Database Types, the Default Schema value is necessary.

At the header of this panel are quick use icons as shown below:

Icon	Name	Use
	Save	The Save icon saves the current entries.
	Cancel	The Cancel icon leaves the current page without saving changed data.
	Help	The Help icon links to the page in Syniti help text to provide a detailed explanation of this section and use.
	Additional Options	This icon produces a list of options for the Datasource including -             Import Metadata      View Debug Logs      View History Refer to sections below that discuss these options and use.

SAP Datasources filtered by Client

The SAP System Datasources use the field Client to determine which client filter of data within a server is used. To provide a filter for all tables of a datasource, the field Client is provided within the Datasource Details page in section Syniti Catalog System as shown below. This field is then passed onto the Replicate system for the snapshot datasource linked to this system datasource. Individual tables within a datasource may also be assigned a Client, however, in the system datasource, the client applies to all tables of that datasource.

Note

When the Database Type is SAPNETWEAVER, leave the Default Schema field blank, otherwise the scanning process will fail.

When finished with edits, click the Save  icon to complete.

Edit the Tables or Columns

At this point, the user may add new Tables and Columns to the existing Datasource. They may also edit the details section to update field values. Refer to section Adding & Editing Tables or Columns for details.

Additional Options Icon for a Datasource

Within this icon of the Datasource there are four options:  Import Metadata, Import Metadata Delta, View Debug Logs, and View History. This is a quick step process that allows the user to perform tasks without having to modify using the Edit process.

Import Metadata - Refer to section Catalog > Datasources > Adding & Editing Datasources > Import Metadata for details of this process.

View Debug Logs - A click on this option opens the Debug Log and filter to job logs for this Datasource.

Note

The Download Debug Zip File icon will produce a file on the local drive with details of this table data.

View Debug Logs

View History - Click on this option to view the history of changes for this system datasource. A window displays rows of changes that may be downloaded to a local drive.

Note

The Download icon will produce a file on the local drive with details of this table data.

View History

Delete a Datasource

Should the System Datasource not be relevant to the project it may be removed from Syniti Migrate. Generally, once tables are imported to the Datasource it is considered a required datasource, however, if the tables are not set as references to Datasets, Mappings, Value Mappings, etc. it may be deleted from the system.

In the image below, the system datasource JDE contains many tables of which table F0004 contains References. In this case, the system datasource cannot be removed.

A warning message displays after a click on Delete icon. Click OK to proceed.

The Validation Error message indicates that the system datasource contains datasource tables with references and therefore cannot delete. The additional warning messages are ignored due to the Error message. Click OK to exit this action.

Datasources: System Datasource: Delete Validation Error Message

In the case where the System Datasource does not contain tables with references, it may be removed.

There are no references attached to the Datasource tables, and only warning messages display. Click on CONTINUE DELETE button to proceed to remove this system datasource table from the list. The warning messages indicate that the tables are removed and the snapshot relationships are removed with this action.

Datasource Objects

On the right side of this page is the Tab for Tables & Columns. The Administrator or IT Lead is responsible for this tab of data. This page provides two panels - one for Datasource Tables, and then one for Datasource Columns. Refer to section Catalog > Dataset Design > Maintain Tables & Fields for more details on this topic.

Subsequent Tasks

• Upon saving the Datasource, the User should attempt to Import metadata (import the tables and fields as they exist in the tables in the system identified). Refer to section Catalog > Datasources> Adding & Editing Datasources > Import metadata for details of this subsequent process in Syniti Migrate.

• In the case of a Snapshot Datasource, the User would IMPORT TABLES from the source Datasource to the Snapshot Datasource (all Active tables), and then Import metadata. Refer to section Migrate > Snapshot Management for details of this subsequent process in Syniti Migrate.