An object in Salesforce can contain a large amount of data, which can cause queries to time out according to Salesforce’s limits. This can become a problem when trying to mask sensitive data, as querying for that data may cause the query to reach its time limit and throw an exception. To address this issue, DataMasker has introduced the feature of SOQL Slicing. This feature allows the user to break down their queries according to the Created Date into smaller, more manageable pieces, making it easier to query and mask large amounts of data without running into time-out issues.
DataMasker will fetch the earliest record created date and starts creating slices up to the latest record created date. By using SOQL Slicing, users can effectively mask their sensitive data without having to worry about query time-out exceptions.
The below screenshot shows how the slicing of an object looks like in the Execution –
There is a field named ‘Max Record Per Slice’ in which the User can decide how many records should be accommodated in a single slice. Whatever value is added into the “Max Record Per Slice” field that many records only will be accommodated in a single slice.
There is two way to define the Max Record Per Slice in DataMasker.
- Object Masking –
While adding new object masking user will see this field in the Advanced Settings section –
- Custom Metadata Type –
I. Go to Custom Metadata Types
II. Click on the Manage Record at DM App Configs
III. Scroll down to the field ‘Max Record Per Slice’
Initially, the value of Max Record Per Slice is 3 Million by default. User can change it as per the record count.
If the user adds the value in Max Record Per Slice at the Object Masking level then DataMasker will consider that value and if the user doesn’t add any value at the Object Masking level then it will directly pick the value from Custom Metadata Type.
How Slicing Works?
- DataMasker runs in different stages so the SOQL Slicing is one of the stages which runs after Get Total Records as shown in the below screenshot
- When the User clicks on the Run Masking button DataMasker fetches the earliest record Created Date for the specific object.
- Then DataMasker starts creating slices for the object as per the Max Record Per Slice up to the latest record created date.
- The Slices get created in the execution logs after completion of SOQL Slicing batch gets completed.
- So each slice fetches separated data to mask because of that records will not intermediate between each other and doesn’t go into row locking issue.
- Also by using SOQL Slicing there will not be any query timeout exception happens.
In summary, slicing is a process used in DataMasker to mask data from Salesforce objects. It works by creating slices of data based on the Max Record Per Slice setting and the earliest and latest record-created dates. By breaking the data down into smaller slices, DataMasker can process the data more efficiently, ensuring that records do not interfere with each other and preventing query timeout exceptions.