Job Queues

This topic is related to the following section - Debug Log

About Job Queues

Jobs run when the user executes either a variety of tasks like building XML coding or refreshing target reports or importing table fields in a datasource as example.

Jobs can also be scheduled to run at a specific date and time repeatedly. 

Note

If the user who executes the job has been setup with a valid Email address within Users Management, they receive an email with information regarding the status of the job once it is completed.  

Each Job Queue is a process complete with one or more steps to complete the process. In some cases, only one step is required as when we refresh an individual table. However, if the user selects BUILD XML within a Dataset or one of the sources, then the job queue kicks off for the process with each table, dataflow, workflow, and etc. building code as a step.

View the following page by navigating to Monitor > Job Queues from the Syniti Migrate Homepage.  The page is setup as two separate processes.  Within the field in top left corner for Jobs, the user may select either Executable Jobs or Scheduled Jobs to display a list of the jobs available for that selection. 

Executable Jobs

In this example below, the Executable Jobs list displays all jobs with their status.  These jobs have been initiated within the sections of Migration for Datasources, Dataset Design, Mapping and etc. 

Most Executable jobs are created by initiating a process such as Importing Metadata or Building XML or Creating Reports as examples.  Executable jobs are not created manually in the Job Queue.  The Executable jobs may be set to repeat processing on a scheduled basis forming a Scheduled Job.  The most typical Executable jobs are listed below:

Notification of Job Complete

Each user in Syniti Migrate should be setup with a valid Email address as shown below. Access Profile > My Profile to view your profile information.

Should the user with a valid Email address initiate a Job Queue, the notification of completion and status is sent in an email once the job is complete. An example for the email notification is provided below.

Note

Email notifications are sent from skp-no-reply@syniti.com . To prevent notifications from going to your spam folder, add skp-no-reply@syniti.com to your contact list.

Re-Run an Executable Job

Should an Executable job fail, or should the user want to run another time, they have a chance to correct the issue and run that same job again successfully by a click on the Re-Run icon listed in the Jobs panel.  Only Executable Jobs that have status of COMPLETE or ERROR may be Re-Run.  Executable Jobs with Status of INITIALIZE or IN PROGRESS have not been completed and cannot be Re-Run. 

Note

If the job finishes with some of the steps showing step status of COMPLETE (successful) and some of the steps showing step status of ERROR (Failed), the Job Status overall shows as ERROR to provide indicator that the job did not complete successfully for all steps.

The user clicks OK to initiate the re-run of the job.  If the issues are resolved, this run should end successful. 

The Created Date updates to current date and time.  The Status changes to New, and the Steps reset to 0/X number as shown below.

As it runs, it displays Status of IN PROGRESS.  Once complete, it displays STATUS of COMPLETE or ERROR with the count of StepsPassed updated to XX/XX. 

Once the job is complete, it provides the count of Steps Passed, and Status updated to new values for this run.  A click on any of the Steps provides details for the Created On, Created By, Modified On, and Modified By values.  It is within the details that the user can see that the job is a Re-Run.

Note

If the Step Name produced exceeds 100 Characters, it is reduced to first 100 Characters and then ... is added at the end to show the name was reduced in length. 

Scheduled Jobs

Filtering the list of Jobs for Scheduled Jobs provides the same panels and information based upon jobs that repeat on a time table.  The Status for these jobs is SCHEDULE, and they may also have multiple steps.  The frequency of the run is displayed in the grid. 

Add a Scheduled Job

Once a Job Queue has been initiated and run (as an Executable Job), it can then be setup as a Scheduled Job for the future.  In the example shown below, the build / refresh of working tables ran successfully as a one time Executable Job.  Click on the Edit  icon to open the JobQueue Details window. 

Click on the Schedule  icon to initiate the scheduling.  A new window displays that provides fields for scheduling the job for a future repeatable time frame. 

Enter the details for fields for Frequency and timing. 

Note

The Date/Time for a scheduled job is stored in UTC time zone always.

Job Queue: Schedule JobQueue: Add timing for new job

When finished with edits, click the Save  icon to complete.

To view the new Scheduled Job, filter Jobs for Scheduled.  The new job displays in the list with a Created Date.

Edit a Scheduled Job

To view the timing for the scheduled job, first click on the Edit  icon.  Next, click on the Schedule  icon as show below.  The schedule displays within the Schedule JobQueue window and is editable. 

When finished with edits, click the Save  icon to complete.

Delete Jobs Prior to a Date

The user may choose to delete Executable jobs based upon a date if they click on the Delete   icon within the header Jobs title bar.  A window displays with entry for a Date to clear all jobs prior to the selected date.  Click on the Delete button to proceed. 

A Warning message displays "Are you sure you want to Clear?".  Click OK to proceed. 

All of the jobs prior to the date selected are removed. 

Delete a specific job

If the user would like to delete a specific job from either the Scheduled Jobs or Executable Jobs lists, they may click on the Delete  icon on the row of the Job as shown below.  A Warning message displays "Are you sure you want to Delete?"  Click OK to proceed.

Interpreting Steps, Parameters, and Values

The Steps are sequential, and some steps are set as a clean up step to another step.  For instance, if Step 130 is in ERROR, then the Clean Up Step (140) provides the action or result of Step 130 failing.  In this case, Step 140 is updating the STATUS_MESSAGE with details of what failed in Step 130.  Step 10 completed successfully, therefore Step 20 carries status of SKIPPED as it was not needed. 

In the process of debugging for failed jobs, the user may copy and paste SQL values into the SQL Server Query page and run that scripting to see what fails. 

Note

This example shown is running the Generate Reports Job Queue where multiple reports produce multiple independent steps.  Each Step has opportunity to run multiple times if they fail so that system issues do not fail a job (Ex: system timeout or System is Busy). 

In other cases, it may be necessary to combine running the job with using the Debug Log to determine the error messaging.  Refer to section Monitor > Debug Log for more details. 

Importing Table Data SQL Server

In the case where the job queue is importing Datasource table data, the job queue steps show a status for importing the Datasource and Snapshot Datasource

Importing Table Data to a Datasource

In the case of Import Metadata for a Datasource, the system runs a single line in the Job Queue. In this case, the Datasource is marked as Import All.

Click the Bug icon to view details for this job. The details show that 10 tables were imported as part of this job.

Job Status

Depending on the size of the job, it may be necessary to wait a few minutes and refresh periodically by clicking the browser Refresh button to monitor the status from "IN PROGRESS" to "COMPLETION" or "ERROR".

You may encounter the following Status messages during a run of a job:

1. NEW - Job is created

2. INITIALIZE - Job is prepping to run

3. IN PROGRESS - Job execution is in progress

4. COMPLETE - Job finished successfully

5. ERROR - Job did NOT finish and contains errors

The Search  icon on the top right allows you to search for a specific job.

Job Status Details

Generally, a job should INITIALIZE (new), IN PROGRESS (run), and either COMPLETE (run successfully) or ERROR (fail to complete).  However, sometimes there is messaging that requires additional review. 

Job Queue #1 - In the case where the Job Queue is showing as ERROR and the StepsPassed show as YY/XX (where XX is the total count of steps, and YY is the total count of steps that passed), the difference of the two counts equal the number of Job Steps that show ERROR for Status.  This job has Completed with Errors. 

Resolve:  The user should review the individual steps to determine the cause of ERROR(s) and then manually correct the situation within the database.