...
Specify the execution type from the following options:
- Validate & Execute: Performs limited execution of the Snap (up to 50 records) during Pipeline validation; performs full execution of the Snap (unlimited records) during Pipeline execution.
- Execute only: Performs full execution of the Snap during Pipeline execution; does not execute the Snap during Pipeline validation.
Disabled: Disables the Snap and, by extension, its downstream Snaps.
...
On this page
Table of Contents | ||||
---|---|---|---|---|
|
Overview
Use this Snap to hide sensitive information in your dataset before you export the dataset for analytics. With the Mask Snap, you can ensure sensitive data protection by using masking algorithms that the Snap provides out of the box.
Snap Input and Output
...
- Min: 1
- Max: 1
...
- Mapper Snap
- MySQL - Select
- REST Get
...
- Min: 1
- Max: 1
...
- MySQL - Insert
- JSON Formatter
- CSV Formatter
...
A dataset where the specified data is masked.
Snap Settings
...
Field
...
The field/parent field in the input dataset that contains the data to be masked.
...
The mode the Snap must use to search sensitive data.
Select from the following options:
- Exact Path: The Snap searches for the exact field name.
- Recursive: The Snap searches through all the levels in the nested structure of the specified Field.
For example, if you want to mask $credit_card
field, you must enter $credit_card in Field for Exact Path Search Mode. However, if you are using the Recursive option as the Search Mode, then you can just enter $ in Field.
...
The type of field in the input data to be matched. Select from the following options:
- Key: Select if the data to be matched is the field name.
- Value: Select if the data to be matched is the field value.
Info |
---|
The Snap fails to validate if the Match Field is Key and the Mask Field is also Key. In such a case, select Value as the Mask Field. |
...
The match condition that determines whether the Match Field should be matched. For example, if your input dataset contains $credit_card Field and you enter the Match Field as Value and Match Condition as Number, then the Snap masks all $credit_card fields that contain a number as the value. If the field contains text as the value, then the Snap skips masking that value.
Select from the following options:
- Regex Match: Matches the Key or Value when the key or value is the expression entered in Match Pattern. If Match Pattern is blank, the Snap matches all values.
Credit Card: Matches the Key or Value when the key or value is the number pattern used in major credit cards. The supported credit card types are: Visa, MasterCard, JCB, American Express, Discover, and Diner's Club.
The Snap supports both long digits or groups of 4-digits that are separated by a dash or space. For example, 4111 1111 1111 1111, 3589-0731-0185-9601, and 6011000000000004.
- SSN: Matches the Key or Value if the key or value is in the Social Security Number (SSN) pattern. The Snap supports values entered in the SSN format only, which is, XXX-YY-ZZZZ. You can use blanks or dashes as separators. For example, 123-12-1234 and 123 12 1234.
- Number: Matches the Value when the value is an integer or decimal.
- Number (Text): Matches the Key or Value when the key or value is a number in text format. The Snap does not support commas within the number text.
- Date: Matches the Value when the value is a date object.
- Date (Text): Matches the Key or Value when the key or value is a date in text or a pattern entered in Match Pattern. You can specify the format in Match Pattern. The Snap supports the following formats:
- "2018-08-12"
- "2018-08-12T12:34:56"
- "2018/08/12 12:34:56.780"
- "2018/08/12 12:34:56.78"
- "2018-08-12T12:34:56.78"
- "2018-08-12 12:34:56"
...
The field that contains sensitive data and will be masked if the matching conditions are met. Select from the following options:
- Key: Select if the data to be masked is the key.
- Value: Select if the data to be masked is the value.
Info |
---|
The Snap fails to validate if the Match Field is Key and the Mask Field is also Key. In such a case, select Value as the Mask Field. |
...
The method to use to mask sensitive information. Select from the following options:
- Replace: Replaces the matched Value with the value you enter in Mask Value.
- Shuffle: Shuffles the matched data Value randomly.
Remove: Removes the matched Key or Value.
- If the Mask Field is Value, the Snap returns null as the new value. If the Mask Field is Key, the Snap removes the whole Key-Value pair.
- With arrays and Recursive Search Mode, if the Mask Field is Value, the value is removed from the array. If all values are removed, the Snap returns an empty array. However, if the Mask Field is Key and the Snap removes the array along with the key.
- Random: Replaces the matched Value with a new random value. The Snap derives the random value based on the data type of the matched value and behaves as described below:
- Text: Randomly replaces the value with alphanumeric characters having the same length as the original data.
- Integer: Randomly replaces the value with an integer value in the range between 0 to an integer of the same digit as the actual data. The new value is always different from the original value. For example, if the original data is 120, the possible replacement value is between 0 to 999, except for 120.
- Decimal: Randomly replaces the value with a value that is based on the precision of the original value from 0 to the precision on the same digit. For example, if the original value is 0.023, the possible replacement value is between 0.000 to 0.099, except for 0.023.
- Boolean: Randomly replaces the value with either true or false.
- Start of Month: Replaces the matched Value with the first day of the month. If the value is a text, the Snap converts the text to date using the same way as the Auto mode in the Type Converter Snap. In this mode, the Snap automatically converts text to Date.
- Start of Year: Replaces the matched Value with the first day of the same year. If the value is a text, the Snap converts the text to date using the same way as the Auto mode in the Type Converter Snap. In this mode, the Snap automatically converts text to Date.
...
- String
- Numeric
...
On this page
Table of Contents | ||||
---|---|---|---|---|
|
Overview
Use this Snap to hide sensitive information in your dataset before you export the dataset for analytics. With the Mask Snap, you can ensure sensitive data protection by using masking algorithms that the Snap provides out of the box.
Snap Input and Output
Input/Output | Type of View | Number of Views | Compatible Upstream and Downstream Snaps | Description |
---|---|---|---|---|
Input | Document |
|
| A dataset where some of the data must be masked. |
Output | Document |
|
| A dataset where the specified data is masked. |
Snap Settings
Parameter Name | Data Type | Description | Default Value | Example | ||
---|---|---|---|---|---|---|
Label | String | Required. The name for the Snap. Modify this to be more specific, especially if there is more than one of the same Snap in the Pipeline. | N/A | Mask | ||
Mask Policy | N/A | Required. Enables you to specify the policies that you want to use to mask data in the input dataset. | N/A | N/A | ||
Field | String | The field/parent field in the input dataset that contains the data to be masked. | N/A | name | ||
Search Mode | N/A | The mode the Snap must use to search sensitive data. Select from the following options:
For example, if you want to mask | N/A | Exact Path | ||
Match Field | String | The type of field in the input data to be matched. Select from the following options:
| N/A | Value | ||
Match Condition | N/A | The match condition that determines whether the Match Field should be matched. For example, if your input dataset contains $credit_card Field and you enter the Match Field as Value and Match Condition as Number, then the Snap masks all $credit_card fields that contain a number as the value. If the field contains text as the value, then the Snap skips masking that value. Select from the following options:
| N/A | Date | ||
Match Pattern | String | The expression of the information pattern to be matched in the input dataset. This is applicable only when the Match Condition is Regex Match or Date (Text). | N/A | Hello | ||
Mask Field | N/A | The field that contains sensitive data and will be masked if the matching conditions are met. Select from the following options:
| N/A | Value | ||
Mask Method | N/A | The method to use to mask sensitive information. Select from the following options:
| N/A | Remove | ||
Mask Value |
| The value that must replace the sensitive information in the input dataset. You can enter either a fixed value or an expression. This is applicable only when the Mask Method is Replace. | N/A | 0 | ||
Execute during preview | N/A | Specify the execution type from the following options:
| Validate & Execute | Validate & Execute |
Example
This Pipeline demonstrates how the Mask Snap helps you hide sensitive information in a dataset. In this example, the input dataset is a demographic of the Oscar award winners. We want to pass on the dataset to a third party for some analytics. However, before we pass on the data, we want to hide sensitive information from the dataset. We use the Mask Snap to identify and replace/remove all sensitive information from the Oscar award winners list. While the dataset here is public data, you can apply the masking policies that the Mask Snap provides to hide confidential or sensitive data from your organization.
Download the Pipeline.
Expand | ||
---|---|---|
| ||
The dataset is derived from Kaggle and is a demographic of Oscar award winners in the Best Director category from 1927 through 1976. The input document is picked up using the File Reader Snap and is passed through the CSV Parser Snap. A preview of the dataset in CSV format is as follows: The dataset is passed to the Mask Snap to hide sensitive data. In this example, we want to mask the following fields:
For this purpose, the Mask Snap is configured as follows: We have added three rows under Mask Policy, and each row enables us to mask one of the fields discussed above. The first row under Mask Policy enables us to mask the The second row under Mask Policy enables us to mask the In Recursive mode, you can mask multiple fields at a time depending on the condition specified. For example, if you think your data is nested, or in an array, and that you cannot specify the exact path, you can use Recursive mode. In this example, we are not sure how many fields contain URLs. Hence, we use recursive mode to mask all field values that contain a URL. The third row under Mask Policy enables us to mask the name of the winner. We specify the Field as $person and select the Search Mode as Exact Path. The Match Field is Key and the Match Condition is Regex Match, and the Match Pattern is .* which matches any value that is entered in the $person field. The Mask Method is Replace and the Mask Field is Value. The Mask Value is Winner name is masked. The Snap replaces all After the dataset is passed through the Mask Snap and the different mask conditions are applied to it, the output dataset is as follows: In the output dataset preview, we can see that the The masked dataset output of the Mask Snap is then converted to CSV format using the CSV Formatter Snap and then passed to a File Writer Snap. |
...
See Also
...