Pipeline Settings

Data Hub pipelines have settings that either determine how data is extracted or drive the metrics available in summary views. For example, pipeline settings control how many days back historical data for a timecard is processed.

During Data Hub implementation, when data structures are deployed for a tenant, default values are populated for the pipeline settings for that tenant. These settings can be changed based on your business needs. When you edit and save changes to the settings, the next time the pipelines run, all summary metrics (except HISTORICALSTARTDATE) recalculate based on the updated pipeline values.

Go to the Pipeline Settings page to see:

  • All pipelines for your Data Hub tenant.
  • The pipeline setting name and value.
  • The date the setting value was last updated. When Data Hub is first implemented, this date reflects the initial Data Hub deployment.

From the Pipeline Settings page, you view and edit pipeline setting values. The values you enter depend on the type of pipeline. Values include numbers, days of the week, text strings, and JSON strings. For example, for the schedule_week_start_day setting of the schedulePost pipeline, the setting value is a day of the week. 

There are two capture methods:

  • For most pipelines, the data capture method is drop and replace. This method deletes data and then reloads (redelivers) it for a span of time.
  • A subset of pipelines are enabled by default for change data capture (CDC), in which the Change Indicator Service (CIS) used by Data Hub manages the CDC process and ensures the data delivery includes only data that has changed since the last extraction request. If the change indicator service becomes unavailable or has connection issues, then the capture method automatically switches to drop and replace until the service is restored.

Caution: Data Hub uses the ADP Workforce Manager CIS with the pipelines enabled for CDC. If you plan to subscribe to an external (non-Data Hub) CIS, you must contact an ADP Functional Expert before you proceed. Otherwise, you may experience problems running CDC-enabled pipelines.

This section describes the available default pipeline settings.

Pipeline Setting Category Pipeline Setting Name Description
Standard Pipeline Settings HISTORICALSTARTDATE Determines the historical date from which pipelines start delivering ADP Workforce Manager data into Data Hub. If this value is blank (null), all pipelines will fail.
BCK_DAYS Determines the range of days back from the current date a pipeline drops and replaces data. Not displayed for CDC pipelines with CDC_Enabled set to True.
FWD_DAYS Determines the range of days forward from the current date the pipeline delivers data. Not displayed for CDC pipelines with CDC_Enabled set to True.
WEEK_OFFSETS Determines the range of weeks forward or back the pipeline delivers data. For example, to generate labor forecasts for the next two weeks from the current week, set this to 2. Not displayed for CDC pipelines with CDC_Enabled set to True.
Accrual Balances

INTERVAL_BEGIN_MM_DD

Determines the reporting period in which the data is stored.

Options are:

  • Weekly. Choose the day of the week you want to report the accrual balances.
  • Monthly. Choose the month begin or the month end. This determines the first month the data will be stored.
  • Pay Period End. Choose the pay period end date. The accrual balance will be as of the specified end date.
INTERVAL

Determines the partitionDate used in the tables. Depending on the INTERVAL selected, the partition date is weekly, monthly, or pay period end date.

Supported values include:

"Weekly-Monday", "Weekly-Tuesday", "Weekly-Wednesday", "Weekly-Thursday", "Weekly-Friday", "Weekly-Saturday", "Weekly-Sunday", "Monthly-Begin", "Monthly-End", and "Pay-Period-End"

Note: accrualBalance data can only be stored in one interval and if any changes are made to these settings, an historical reload is required for the change to take effect.

Change Data Capture CDC_Enabled

Determines whether CDC is used when processing data. Note the following:

  • When set to True (the default), only the data that has changed since the last time the pipeline ran is delivered. In general, it is recommended that it always be set to True for the pipelines that support it, including (but not limited to) actualVolume, availability, laborForecast, scheduleOpenShift, timecardMetrics, timecardTransaction, volumeForecast, and any pipelines that consume data from APIs that deliver data from timecard, people, and schedule details domains.
  • When set to True, configuration values for the BCK_DAYS, FWD_DAYS, and WEEK_OFFSET settings are unavailable for pipelines that support CDC.
  • Only clients using a CIS external to Data Hub need to set this to False, in which case the data capture method is drop and replace. Contact a ADP Service Representative before you subscribe to an external change indicator service to avoid pipeline problems.
Linked Category linked_category_configuration

Determines how to calculate totals in the following views for clients using linked categories in Forecaster: vVolumeForecast, vActualVolume, vVolumeBudget, vActualVolumeIntra, and vTrafficPatternForecast.

Options are:

  • Both (the default). Views will contain values for both primary and secondary nodes.
  • Primary. Views will contain only the primary node.
  • Secondary. Views will contain only the secondary node.
Multicurrency Exchange Rate Support multicurrency_support

When multicurrency is configured in ADP Workforce Manager, this setting controls whether these currency exchange values are stored in the summary dataset.

Options are:

  • No (the default). Summary views will not contain currency exchange.
  • Yes. Summary views will contain currency exchange rates.
Multicurrency Terminated Employee Data excludeEmployeeTerminatedData

Determines if terminated employee data displays in the summary dataset for days when the employee has an effective employment status of terminated.

Options are:

  • Yes (the default). Summary view terminated employee totals will be included. The summary data will exclude data for dates when the employee is terminated but will include the date that the employee was terminated. For example, if the employee is terminated on June 15th, then data for June 15th will be included in the summary tables, but data after June 15th will be excluded.
  • No. Only terminated employee information up to and including their termination date will display in views. No filtering will be applied for days where the employee has a status of terminated.

Note: If the setting is changed, the full historical run of the summary pipeline should be run.
Schedule Audits schedule_change_notice_period The number of hours before a scheduled shift that any changes can be made to a schedule. This is used by the schedule audits metrics to track schedule changes made too soon before the start of a scheduled shift. The default is 24.
Schedule Posting schedule_posting_deadline The number of hours used to calculate the schedule posting deadline time. The default is 80.
schedule_posting_job_filter A JSON string value that contains the list of jobs you want to exclude from the schedule posting metrics. If you do not want to calculate postings for managers and above, these job names need to be included in this list.
schedule_posting_location_type

Determines the business structure location type for which schedule postings are tracked. Options include Job, Department, and Site. The default is Job. Note the following, depending on the value you enter:

  • For Job, for each job in the structure, for each schedule week the system will track the date/time when schedules were posted for that job.
  • For Department, for each schedule week the system will track the date/time of the first job posted and the last job posted for each department.
  • For Site, for each schedule week, the system will track the date/time of the first job posted and the last job posted for each site.
schedule_week_start_date A string value that defines which day of week the schedule week starts. This will be used to determine schedule posting timeliness based on the week start date and the schedule posting deadline hours. The default value is Monday, but can be any day of the week.
Summary and SummaryIntra limit_hist_days_in_past

A numeric value that sets a limit on the number of days your historical data is retained and updated. For example, to maintain a year's worth of historical data, enter 365. By default, 9999 days of data history is retained and updated.

Note: Maintaining a smaller amount of historical data reduces the volume of data and the processing time of the pipelines.

Note the following:

  • Changes made to these settings in the Portal will be used for subsequent pipeline executions.
  • Pipeline setting values that are changed outside the Portal (by ADP Functional Expert) will be reflected in the Portal.
  • There are multiple settings for each pipeline.
  • Pipeline setting values cannot be blank. A valid value must be entered before you can apply your changes.
  1. Go to the Setup > Pipeline Settings page. All available pipelines on the tenant for the project display, along with pipeline setting names, values, and the date the values were last updated.
  2. Click the Edit icon for the pipeline whose value you want to edit. The Edit Pipeline Settings dialog opens. Only the setting value can be changed.
  3. Change the setting value as needed, depending on the type of pipeline. To see a description of all the default pipelines and the supported values, see Default pipeline settings.
  4. Click Apply. The Edit Pipeline Settings dialog closes and the Setting Value column updates with changed value. The next time pipelines run, the summary metrics will recalculate based on the updated setting values.