Metadata Scanner Filters
  • 21 Mar 2024
  • 3 Minutes to read
  • Contributors
  • Dark
    Light

Metadata Scanner Filters

  • Dark
    Light

Article Summary

Metadata scanners scan all available metadata in the target systems. In some cases, the amount of metadata ingested into the Catalog is overwhelming and the total scan and ingestion time is too long. Scan filters allow you to ingest a filtered set of metadata into the Catalog instead of all available metadata.

A filter is made up of 1 or more filter components; a filter component is made up of an object name and a value. Each scanner can support various object types, value styles and combinations of components. For example, to filter a HANA database to a single schema, specify a filter with a single component where the object is schema and the value is ABAP1. The HANA scanner then applies a SQL WHERE clause when querying the HANA metadata tables - where schema_name = ‘ABAP1’.

A filter is defined for a system but they are optional. A filter can only be defined if a connection exists that supports scanning (i.e., the Usage for the connection is Metadata Scanning) and the scanner for the connection type supports filtering (refer to Scanner-Specific Filters for details). If a filter is defined for a system, it is passed to the scanner when a scan is run. The filter is preserved between scans, although it can be changed and deleted.

Where a filter has been passed to a scanner in a scan request, the scan results show the details of the filter.

General Filter Rules

  1. Filter Name and Value allow spaces, slashes and single quotes.

  2. Name and Value case is passed to the scanner as-is

Scanner-Specific Filters

The following table outlines the scanners that support content filtering and specific behavior of filters, which are generally single-value filters, based on the scanner type.

Note

If two filter conditions are created with the same name, e.g., database, then the conditions are applied as an OR condition. If two filter conditions are created with different names, e.g., database and schema, then an AND condition is applied. Combinations of OR and AND beyond this are not supported.

Scanner

Supported Object Types

Multi-component Behavior

SAP NetWeaver

table

Refer to SAP NetWeaver Filters.

SAP HANA

schema

When multiple components are specified, they are combined into a logical OR.

For example, a filter with 2 components

  1. object = schema, value = uncle

  2. object = schema, value = bob

becomes schema = uncle or schema = bob

Oracle

schema

PostgreSQL

schema

IBM Db2 LUW & IBM Db2 OS/390

schema

SQL Server

  • database

  • schema

When multiple components are specified of the same object type, they are combined into a logical OR. When the object type differs, they are combined with a logical AND.

For example, a filter with 3 components

  1. object = database, value = PROD2

  2. object = schema, value = uncle

  3. object = schema, value = bob

becomes (database = PROD2) and (schema = uncle or schema = bob)

SAP NetWeaver Filters

By default, executing an SAP NetWeaver scan reads the metadata for all pool, cluster and transparent tables. To scan a subset of the tables, the SAP NetWeaver scanner supports a table filter, which can be defined in 2 ways:

  1. A user can define a table filter using filter conditions on the System Scans page. A condition can specify a single table or multiple tables separated by a comma. If multiple conditions are defined, they are combined. Refer to Add a Scan Filter for more information.

  2. An administrator can use a JRE system property called tableFilter.file to specify a file that contains a list of tables to scan. To set this property, use the jvmArguments value in the scanners control file /opt/apps/syniti-agent/scanners/scan.properties.json, e.g., "jvmArguments": "-DtableFilter.file=/opt/apps/syniti-agent/scanners/sap_tables.json".

Syniti supplies a file called sap_tables.json , which contains 2200 tables that cover most migration projects. An administrator can modify this file or create another file in the same structure. The structure required is a single JSON string array, e.g., ["MARA","LFA1","T000"].

If tables are defined using both mechanisms, they are combined. Syniti suggests that an administrator configures the scanner to use the supplied sap_tables.json file and for a user to add a small number of additional tables via the System Scans page, e.g., customer-specific Z tables.

Duplicate or incorrect table names are ignored by the scanner. Spaces in filter values are not supported on the System Scans page.


Was this article helpful?