Job Queues

This topic is related to the following section - Debug Log

About Job Queues

Jobs run when the user executes either a variety of tasks like building XML coding or refreshing target reports or importing table fields in a datasource as example.

Jobs can also be scheduled to run at a specific date and time repeatedly. 

Note

If the user who executes the job has been setup with a valid Email address within Users Management, they receive an email with information regarding the status of the job once it is completed.  

Each Job Queue is a process complete with one or more steps to complete the process. In some cases, only one step is required as when we refresh an individual table. However, if the user selects BUILD XML within a Dataset or one of the sources, then the job queue kicks off for the process with each table, dataflow, workflow, and etc. building code as a step.

View the following page by navigating to Monitor > Job Queues from the Syniti Migrate Homepage.  The page is setup as two separate processes.  Within the field in top left corner for Jobs, the user may select either Executable Jobs or Scheduled Jobs to display a list of the jobs available for that selection. 

Executable Jobs

In this example below, the Executable Jobs list displays all jobs with their status.  These jobs have been initiated within the sections of Migration for Datasources, Dataset Design, Mapping and etc. 

Most Executable jobs are created by initiating a process such as Importing Metadata or Building XML or Creating Reports as examples.  Executable jobs are not created manually in the Job Queue.  The Executable jobs may be set to repeat processing on a scheduled basis forming a Scheduled Job.  The most typical Executable jobs are listed below:

Notification of Job Complete

Each user in Syniti Migrate should be setup with a valid Email address as shown below. Access Profile > My Profile to view your profile information.

Should the user with a valid Email address initiate a Job Queue, the notification of completion and status is sent in an email once the job is complete. An example for the email notification is provided below.

Note

Email notifications are sent from skp-no-reply@syniti.com . To prevent notifications from going to your spam folder, add skp-no-reply@syniti.com to your contact list.

Re-Run or Refresh an Executable Job

Should an Executable job fail, or should the user want to run another time, they have a chance to correct the issue and run that same job again successfully by a click the Refresh icon listed in the Jobs panel. Only Executable Jobs that have status of COMPLETE or ERROR may be refreshed.  Executable Jobs with Status of INITIALIZE or IN PROGRESS have not been completed and cannot be refreshed. 

Note

If the job finishes with some of the steps showing step status of COMPLETE (successful) and some of the steps showing step status of ERROR (Failed), the Job Status overall shows as ERROR to provide indicator that the job did not complete successfully for all steps.

A confirmation dialog box is displayed when user clicks the Refresh icon. Clicking OK initiates the re-run of the job. If the issues are resolved, this run should end successful. 

The Created Date updates to current date and time.  The Status changes to New, and the Steps reset to 0/X number as shown below.

As it runs, it displays Status of IN PROGRESS.  Once complete, it displays STATUS of COMPLETE or ERROR with the count of Steps Passed updated to XX/XX. 

Once the job is complete, it provides the count of Steps Passed, and Status updated to new values for this run.  A click on any of the Steps provides details for the Created On, Created By, Modified On, and Modified By values.  It is within the details that the user can see that the job is a Re-Run.

Note

If the Step Name produced exceeds 100 Characters, it is reduced to first 100 Characters and then ... is added at the end to show the name was reduced in length. 

Scheduled Jobs

Filtering the list of Jobs for Scheduled Jobs provides the same panels and information based upon jobs that repeat on a time table.  The Status for these jobs is SCHEDULE, and they may also have multiple steps.  The frequency of the run is displayed in the grid. 

Add a Scheduled Job

Once a Job has been initiated and run (as an Executable Job), it can then be scheduled at a required frequency and timings to execute the job at a set time in the background. In the example below, the customer master target readiness reports ran successfully as a one-time Executable Job.

1. Click the Edit icon of this job to open the Job Queue Details dialog box.
   

2. Click the Schedule   icon to open the Schedule Job Queue dialog box to setup a cron schedule for this Job.

3. In the Cron Schedule field, enter a valid cron expression to schedule this job in the server.

   Note

   The Schedule Description field aids in understanding the job’s schedule. The scheduled job is always stored in UTC time zone.

4. Click Save to schedule this job. Upon saving, a cron job for this scheduled job is added to the crontab in the server.

Understanding Cron Jobs and Cron Expressions

A cron job is an automated scheduled task that runs on a server at a set time. You can define a cron job’s frequency and timings using cron expressions. A cron expression is a syntax that specifies when a cron job runs. The syntax contains five space-delimited fields as shown below.

* * * * *
| | | | |____ Day of the week (0 - 7) (Sunday is both 0 and 7)
| | | |______ Month (1 - 12)
| | |________ Day of the month (1 - 31)
| |__________ Hour (0 - 23)
|____________ Minute (0 - 59)

Fields can contain any of the allowed values with various combinations of the allowed special characters , - * / for that field. Each field can be expressed as follows:

• A single wildcard *, which covers all values for the field. So, a * in Day of the Month field means all days of a month.

• A single value, for example 5. A valid value that varies for each field.

• A comma-delimited , list of values, for example, 1,2,3,4.

• A range where the minimum and maximum are specified using a hyphen -, for example, 1-10.

• An interval specification using a backslash /, for example, */4, meaning every 4th value of the field. You can also use it in a range, as in 1-6/2.

• You can also combine all of the above in a field, for example: 1-5,10,12,20-30/5

Note

The Month and Day of the Week fields accept string values and are not case-sensitive. You can specify Jan-Dec for Month and Sun-Sat  for Day of the Week.

Cron Expression Examples

1. Run the job once per day at midnight: 0 0 * * *

   • 0: Minute (0th minute of the hour)

   • 0: Hour (Midnight, or 00:00)

   • *: Every day of the month

   • *: Every month

   • *: Every day of the week

2. Run the job once per week at 5pm on Friday: 0 17 * * 5

   • 0: Minute (0th minute of the hour)

   • 17: Hour (5 PM in 24-hour format)

   • *: Every day of the month

   • *: Every month

   • 5: Friday (0-Sun, 1-Mon, 2-Tue, 3-Wed, 4-Thu, 5-Fri, 6-Sat)

3. Run the job on the 1st of every month at 6:30 am: 30 6 1 * *

   • 30: Minute (30th minute of the hour)

   • 6: Hour (06:00)

   • 1: The 1st day of the month

   • *: Every month

   • *: Every day of the week

View Scheduled Job

To view the new Scheduled Job, filter Jobs for Scheduled.  The new job displays in the list with a Created Date.

Edit a Scheduled Job

To view the timing for the scheduled job, first click on the Edit  icon.  Next, click on the Schedule  icon as show below.  The schedule displays within the Schedule JobQueue window and is editable. 

When finished with edits, click the Save  icon to complete.

Delete Jobs Prior to a Date

The user may choose to delete Executable jobs based upon a date if they click on the Delete   icon within the header Jobs title bar.  A window displays with entry for a Date to clear all jobs prior to the selected date.  Click on the Delete button to proceed. 

A Warning message displays "Are you sure you want to Clear?".  Click OK to proceed. 

All of the jobs prior to the date selected are removed. 

Delete a specific job

If the user would like to delete a specific job from either the Scheduled Jobs or Executable Jobs lists, they may click on the Delete  icon on the row of the Job as shown below.  A Warning message displays "Are you sure you want to Delete?"  Click OK to proceed.

Interpreting Steps, Parameters, and Values

The Steps are sequential, and some steps are set as a clean up step to another step.  For instance, if Step 490 is in ERROR, then the Clean Up Step (500) provides the action or result of Step 490 failing.  In this case, Step 500 is updating the STATUS_MESSAGE with details of what failed in Step 490.  Step 450 completed successfully, therefore Step 460 carries status of SKIPPED as it was not needed. 

In the process of debugging for failed jobs, the user may copy and paste SQL values into the SQL Server Query page and run that scripting to see what fails. 

Note

This example shown is running the Generate Reports Job Queue where multiple reports produce multiple independent steps.  Each Step has opportunity to run multiple times if they fail so that system issues do not fail a job (Ex: system timeout or System is Busy). 

In other cases, it may be necessary to combine running the job with using the Debug Log to determine the error messaging.  Refer to section Monitor > Debug Log for more details. 

Importing Table Data SQL Server

In the case where the job queue is importing Datasource table data, the job queue steps show a status for importing the Datasource and Snapshot Datasource.

Importing Table Data to a Datasource

In the case of Import Metadata for a Datasource, the system runs a single line in the Job Queue. In this case, the Datasource is marked as Import All.

Click the Debug icon to view details for this job. The details display the total number of tables imported as part of this job.

Job Status

Depending on the size of the job, it may be necessary to wait a few minutes and refresh periodically by clicking the browser Refresh button to monitor the status from "IN PROGRESS" to "COMPLETION" or "ERROR".

You may encounter the following Status messages during a run of a job:

1. NEW - Job is created

2. INITIALIZE - Job is prepping to run

3. IN PROGRESS - Job execution is in progress

4. COMPLETE - Job finished successfully

5. ERROR - Job did NOT finish and contains errors

The Search  icon on the top right allows you to search for a specific job.

Job Status Details

Generally, a job should INITIALIZE (new), IN PROGRESS (run), and either COMPLETE (run successfully) or ERROR (fail to complete).  However, sometimes there is messaging that requires additional review. 

Job Queue #1 - In the case where the Job Queue is showing as ERROR and the Steps Passed show as YY/XX (where XX is the total count of steps, and YY is the total count of steps that passed), the difference of the two counts equal the number of Job Steps that show ERROR for Status.  This job has Completed with Errors. 

Resolution:  The user should review the individual steps to determine the cause of ERROR(s) and then manually correct the situation within the database.