An object in Salesforce can contain a large amount of data, which can cause queries to time out according to Salesforce’s limits.
This can become a problem when trying to mask sensitive data, as querying for that data may cause the query to reach its time limit and throw an exception.
To address this issue, DataMasker has introduced the SOQL Slicing feature.
This feature allows the user to break down their queries according to the Created Date into smaller, more manageable pieces, making it easier to query and mask large amounts of data without running into time-out issues.
DataMasker will fetch the earliest record created date and start creating slices up to the latest record created date. With SOQL Slicing, users can effectively mask their sensitive data without worrying about query time-out exceptions.
The screenshot below shows what the slicing of an object looks like in the Execution.
There is a field named ‘Max Record Per Slice’ in which the User can decide how many records should be accommodated in a single slice. Whatever value is added to the “Max Record Per Slice” field, many records will only be accommodated in a single slice.
There are two ways to define the Max Record Per Slice in DataMasker.
Object Masking
While adding new object masking user will see this field in the Advanced Settings section
Custom Metadata Type
- Go to Custom Metadata Types
- Click on the Manage Record at DM App Configs
- Scroll down to the field ‘Max Record Per Slice’
Initially, the value of Max Record Per Slice is 3 Million by default. User can change it as per the record count.
If the user adds the value in Max Record Per Slice at the Object Masking level then DataMasker will consider that value and if the user doesn’t add any value at the Object Masking level then it will directly pick the value from Custom Metadata Type.
How Slicing Works?
- DataMasker runs in different stages so the SOQL Slicing is one of the stages which runs after Get Total Records as shown in the below screenshot
- When the User clicks on the Run Masking button, DataMasker fetches the earliest record Created Date for the specific object.
- Then DataMasker starts creating slices for the object as per the Max Record Per Slice up to the latest record created date.
- The Slices get created in the execution logs after completion of SOQL Slicing batch gets completed.
- So, each slice fetches separate data to mask, and because of that, records will not intermediate between each other and will not go into row locking issues.
- Also by using SOQL Slicing there will not be any query timeout exception happens.
In summary, slicing is a process used in DataMasker to mask data from Salesforce objects. It works by creating slices of data based on the Max Record Per Slice setting and the earliest and latest record-created dates.
By breaking the data down into smaller slices, DataMasker can process the data more efficiently, ensuring that records do not interfere with each other and preventing query timeout exceptions.
