Data Loading

This section describes how the workflow engine loads data for each relationship between the workflow and data units. The OneStreamworkflow engine controls data loading from the staging data mart to the analytic model. The workflow engine includes intelligence about what data to load and how to load for each workflow unit. The loaded analytic cube gets this from the binding relationship between its input parent workflow profiles and base entities.

In addition, the workflow engine uses the origin dimension's import member exclusively when loading data. This predefined relationship provides a built-in level of data protection between imported data, manual data entry, and journal adjustments. The workflow engine manages how data is placed into the origin dimension's Import, Forms, and AdjInput members. The workflow engine also forces imported data to use the local member of the consolidation dimension.

The workflow engine always starts with a workflow data unit to control clearing, loading, and locking data for its entities. A workflow channel data unit is used if workflow channels are active in the workflow unit's analytic model relationship.

Data Load Execution Steps (Clear and Replace)

When a workflow unit's data load process executes, the engine does the following:

1. Checks workflow state

   • Implicitly locked (parent workflow certified)

   • Explicitly locked

2. Checks workflow profile data load switches

   • Can load unrelated entities (True / False)

   • Flow type no data zero view override (YTD / Periodic)

   • Balance type no data zero view override (YTD / Periodic)

   • Force balance accounts to YTD view (True / False)

   See Integration Settings under Workflow Profiles in Workflow for descriptions of these settings.

3. Analyze prior data loads

   • Evaluate previously loaded data units to list data units to clear during the load.

4. Execute clear data

   • User-defined workflow channel configuration not used

   • Clear Workflow data units loaded by the workflow unit.  A workflow data unit considers accounts and cube data unit standard members, so data clears at an account level by default.

User-Defined Workflow Channel Configuration Execution Steps

Clear all workflow channel data units loaded by the workflow unit. User-defined members and workflow data unit members are standard members of a workflow channel data unit, so data clears at a user-defined member entity, scenario, time, and account level by default.

1. Execute load data

   • Data loads using parallel processing by entity. Multiple entities process at the same time.

Workflow Profile Data Loading Behaviors

This section describes three specific data loading behavior patterns. These behaviors range from the basic data load process where one workflow profile loads one data unit, to more than one workflow profile loads one data unit.

Behavior 1: Single Workflow Profile - Loading One or More Entities on a Mutually Exclusive Basis:

This workflow profile configurationhas only one import child profile under the parent (Frankfurt). The workflow engine follows basic clear and replace data loading steps described in Data Load Execution Steps (Clear and Replace).

Behavior 2: Multiple Import Child Workflow Profiles

This workflow profile configuration has more than one import child profile under the parent (San Diego). The workflow engine must perform extra steps to determine how to load data in the child profiles. 

In this case the two import child profiles may try to load the same cube or workflow data unit because they have the same input parent workflow profile and are trying to load the same entities. When the import GL data or import sales detail workflow profiles execute the data load step, the following process determines how to correctly load data from both pofiles to the cube. 

Multiple Import Child Data Load Evaluation Steps:

1. Check for overlapped data units between import child siblings (import GL data or import sales detail).

2. Determine the siblings have overlapped data units.

   If yes, clear all previously loaded data units for both import child siblings, then reload both using an accumulate load method in the order they appear in the workflow hierarchy. If the two siblings are loading to the same cells, the values are be added together and placed in the cell.

   If no, use the basic clear and replace data loading steps described in Data Load Execution Steps (Clear and Replace).

Behavior 3: Multiple Import Parent Workflow Profiles Loading One or More Common Entities

This workflow profile configuration has a central input parent profile that may load data assigned to another workflow profile. The Central HR Load workflow profile must have the Can Load Unrelated Entities set to True, so the workflow engine will let it try to write data for unassigned entities.

In this situation, when either Central HR Load or Houston executes a data load, the basic clear and replace data loading steps described in the previous section are used. However, Central HR Load does not control any entities so it checks and abides by the workflow and locking status of the workflow profiles that own the entities. For example, the workflow engine disallow updates if Houston is certified or locked and Central HR Load tries to load an entity owned by Houston.

Managing Sibling Imports

Certain application designs may require a workflow parent to have multiple sibling import channels. These designs typically use parallel processing techniques to load multiple non-overlapping sibling import children. The Load Overlapped Siblings setting on the parent boosts parallel processing performance in these workflow designs by eliminating overlapping checks between sibling channels. It only happens when the sibling channels' data sources do not contain overlapping data unit data records. This switch lets applications optimize data partitioning with parallel processing using the least workflow profiles.

Load Overlapped Siblings

• True: Default behavior, sibling channels check for overlapping data units.

• False: Do not check sibling channels for overlapping data units. If an overlapping condition occurs, the last processed channel overwrites the prior.