DataMasque Portal

DataMasque Warnings

This document describes warnings that DataMasque may generate in the course of normal masking runs, and, how to disable them (if they can be disabled).

Multiple Rules Warning

This warning is displayed if a table or column is masked by more than one rule in a single ruleset, which may indicate that a rule was mistakenly included more than once.

Example

The following tables are configured to be masked multiple times by `mask_table` tasks in the ruleset: …

The following columns are configured to be masked multiple times in the ruleset: …

To Disable

Add disable_warning_on_duplicate_rules: true at the top level of the ruleset.

Value Truncation Warning

This warning is displayed when auto_truncate_value is enabled and a masked value is wider than its target string column, so DataMasque truncates the value to fit. Only string-type columns are truncated. The truncated value may no longer be valid, for example a shortened email address.

Example

One or more masked values for column `email` exceeded its maximum width of 25 and were truncated. The truncated values may no longer be valid.

To Disable

Add disable_warning_on_auto_truncate_value: true at the top level of the ruleset, or on the specific rule where you want to suppress the warning.

Glacier Files Warning

On Amazon S3, when masking files or performing a file discovery run, this warning will be shown if a file has been archived with S3 Glacier.

For file masking, this means the file will not be copied to the destination connection. For discovery, it means that DataMasque will not be able to determine if the file contains sensitive information.

Example

Skipping masking file "archived_with_glacier.csv": 
files archived with Glacier can not be masked.

To Disable

Add disable_warning_on_glacier_storage: true to the specific mask_file, mask_tabular_file or run_file_data_discovery task where you want to disable the warning. For a run_file_data_discovery task this also excludes Glacier-archived files from the Files Could Not Be Scanned Warning.

Files Could Not Be Scanned Warning

At the end of a file data discovery run, this warning is displayed if any files could not be scanned: the file could not be read (including files archived with S3 Glacier), a Parquet file contained a column with an unsupported data type, or an unexpected error occurred during discovery. The discovery results may therefore be incomplete. The count is an aggregate; the file discovery summary above it breaks the files down by reason, and the run report lists every affected file.

Files that were not scanned for any of the following reasons do not trigger this warning, because nothing went wrong when DataMasque considered them. The reason names are the ones used in the run report and in the file discovery summary:

  • File type unsupported by data discovery
  • File type could not be determined
  • File was empty
  • Did not match any include filter
  • Matched a skip filter

Example

3 files could not be scanned. See the messages above for details on each file, or download the run report from the Run Logs page for the full list.

To Disable

This warning cannot be disabled in general. The exception is files archived with S3 Glacier: add disable_warning_on_glacier_storage: true to the run_file_data_discovery task to exclude those files from the warning and its count.

No Files Scanned Warning

At the end of a file data discovery run, this warning is displayed if the source contained files but none of them were scanned: for example, a source holding only unsupported file types, or include/skip filters that matched nothing scannable. The file discovery summary above the warning shows, per file extension, why each file was not scanned.

Example

No files were scanned. All 243 files in the source were skipped for the reasons shown in the summary above.

To Disable

This warning cannot be disabled. To resolve it, adjust the task's include/skip filters, or point the connection at a source containing file types supported by discovery: CSV, JSON, NDJSON or Parquet.

Use Of Conditionals Warning

Displayed when using conditional masking features that may leave some data unmasked. For example, skip, where and if/else.

Example

Use of the conditional masking features `where`, `skip`, or `if`/`else`, 
may mean your masking rules are not applied to some database rows or values.

To Disable

Add disable_warning_on_use_of_conditionals: true to the specific mask_table task where you want to disable the warning.

Missing Unique Index Warning

For mask_unique_key tasks, the key column(s) being masked are expected to have a unique index. If a unique index does not exist, then DataMasque will temporarily create one for the duration of the masking run, and will issue a warning that one was created.

Example

Unique requirement for specified key could not be validated, 
as no adequate unique index is found for key column: …

To Disable

Add disable_warning_on_missing_unique_index: true to the mask_unique_key task to disable the warning for that task. DataMasque will silently add the unique index during masking, without showing a warning.

Alternatively, add a unique key/index to the keys(s) before beginning the masking task.

Skipped Character Classes Warning

When using a from_unique_imitate mask, if either of the skip_letters or skip_digits parameters are specified true, a warning is shown to let you know that the configuration will leave some data unmasked.

Example

No letters will be masked.
If the column to be masked only contains letters, nothing will be masked.

No digits will be masked.
If the column to be masked only contains digits, nothing will be masked.

To Disable

Add disable_warning_on_skipped_characters: true as a parameter to the from_unique_imitate mask.

Snowflake Time Travel Warning/Error

When masking data on Snowflake databases, if a table has Snowflake Time Travel enabled, the unmasked data can be recovered by restoring a snapshot of the table from a previous point in time. As such, DataMasque disallows masking until Time Travel is turned off for that table.

Refer to the Snowflake documentation for more information about Time Travel, including how to disable it at the table, schema, database or account level.

Example Error

Snowflake Time Travel is enabled for table `myschema.mytable` (configured explicitly on the table).
When Snowflake Time Travel is enabled, the unmasked data can be recovered. As such, masking is not secure.
Please disable Snowflake Time Travel on the table `myschema.mytable` before masking.
Refer to https://docs.snowflake.com/en/user-guide/data-time-travel for more information.

To Disable

If you want to mask the table without turning off Time Travel, specify allow_masking_tables_with_time_travel: true at the top level of the ruleset. The above message will appear in the run logs as a warning instead of an error, and masking will proceed as normal.