References to Ruleset Libraries

This page describes how to reference values in a library using $ref syntax.

Introduction
- Example
$ref Syntax and Limitations in Rulesets
$ref Syntax and Limitations in Libraries
- Limitations
Common Elements for both Rulesets and Libraries
- Path Syntax
- Data Types

Introduction

References are YAML dictionaries with the key $ref.

Rulesets use references to refer to values in a library.
Libraries can also use them to refer to other places in the same library.

When a masking run starts:

The ruleset is checked to see which libraries it needs.
Those libraries are loaded, and any $refs in them are resolved.
Finally, all $refs in the ruleset are resolved against the libraries, producing the final ruleset YAML for masking.

Example

Consider this ruleset, which masks a date_of_birth column:

# Ruleset not using libraries
version: "1.0"

tasks:
  - type: mask_table
    table: clients
    key: id
    rules:
      - column: date_of_birth
        masks:
          - type: retain_age
            force_change: true

The retain_age mask can be extracted into a ruleset library, so that it can be reused across multiple rulesets:

# Ruleset library "pii_library" in namespace "clients"
version: "1.0"

masks:
  date_of_birth_mask:
    type: retain_age
    force_change: true

If the above library is saved with the name pii_library in namespace clients, then it can be referred to in a ruleset as follows, using imports and $ref:

# Ruleset using library
version: "1.0"

# 1. Add import block with library names and namespaces
imports:
  - "clients/pii_library"

tasks:
  - type: mask_table
    table: clients
    key: id
    rules:
      - column: date_of_birth
        masks:
          # 2. Refer to a value in the library using `$ref`
          - $ref: "clients/pii_library#masks/date_of_birth_mask"

Notice the $ref line, which means: "find the library pii_library in namespace clients, and in there, look up the YAML contents at the path masks/date_of_birth_mask". The YAML contents are then inserted into the ruleset at the point where the $ref is.

In the $ref value, the library namespace and name are joined with a /, and the path to the data within the library comes after a hash # to indicate the start of the path.

`$ref` Syntax and Limitations in Rulesets

To reference a library from within a ruleset, use a dictionary including the key $ref, where the value has the form "<library-namespace>/<library-name>#<path-to-value>". If the library has no namespace, leave it blank and skip the /: <library-name>#<path-to-value>.

For the format of path, see Path syntax.

It's a good idea to quote all $ref values to avoid any characters accidentally being interpreted as YAML markup characters. For example, if the library's name is "library with spaces and quotes", which starts and ends with a literal double quotation mark, you must quote the $ref value with single quotes:

# Incorrect: quotes are treated as YAML syntax,
# and the hash is interpreted as the start of a YAML comment.
$ref: "library with spaces and quotes"#masks/phone_mask

# Correct: use single quotes around the entire value to ensure YAML control characters are interpreted correctly.
$ref: '"library with spaces and quotes"#masks/phone_mask'

`imports` block

Rulesets must declare all referenced libraries in the imports block. If a $ref refers to a library which is not in imports, this will cause syntax validation to fail.

# Example showing the import block in a ruleset
version: "1.0"
imports:
  - common_library             # Library "common_library", with no namespace
  - sales/2026/orders_library  # Library "orders_library" in namespace "sales/2026"

If a ruleset does not use any libraries, the imports block can be left out.

Limitations

$ref cannot be used at the top level of the ruleset, or within version, doc (custom ruleset documentation) or imports. It can be used anywhere inside tasks, hash_columns, and hash_sources.
Using $ref for type fields of task types, mask types, checksums, and so on is not supported:

# INVALID: `type` may not be a reference
rules:
  - column: customer_id
    type:
      $ref: "my_library#other/customer_id_mask_type"

Using $ref for the self part of source: self in hash_sources or hash_columns is not supported.
Rulesets cannot refer to other rulesets, only to libraries.

Locality Interpolation

Locality interpolation is supported for rulesets that use libraries, including within library names, or the path section of a $ref. For example, given these masks in a library named locality_library:

masks:
  name_US:
    type: from_file
    seed_file: DataMasque_lastNames.csv
    seed_column: lastnames
  name_BR:
    type: from_file
    seed_file: DataMasque_BR_lastNames.csv
    seed_column: lastnames

A ruleset can dynamically choose from the masks according to the configured locality:

version: "1.0"
imports:
  - locality_library

tasks:
  - type: mask_file
    include:
      - glob: "*.json"
    rules:
      - masks:
          - type: json
            transforms:
              - path: ['name']
                masks:
                  - $ref: "locality_library#masks/name_{{ locality }}"  # e.g. name_US or name_BR

This ruleset will fail to validate if the configured locality is not US or BR, so be sure the library covers all locality settings you might use.

Here is an example of using locality interpolation in a library namespace (it works in library names as well). First, define a US-specific library:

# Library 'phone_library' in namespace 'nsUS'
version: "1.0"

masks:
  phone_mask:
    type: from_unique_imitate
    retain_prefix_length: 3  # US: 3-digit area codes

Then define a corresponding Brazilian library in a different namespace:

# Library 'phone_library' in namespace 'nsBR'
version: "1.0"

masks:
  phone_mask:
    type: from_unique_imitate
    retain_prefix_length: 2  # Brazil: 2-digit area codes

A ruleset can then select the appropriate library using locality interpolation in the namespace:

# Ruleset selects library according to locality setting
version: "1.0"
imports:
  - ns{{ locality }}/phone_library  # This expands to `nsUS/phone_library` or `nsBR/phone_library`

tasks:
  - type: mask_file
    include:
      - glob: "*.json"
    rules:
      - masks:
          - type: json
            transforms:
              - path: ['phone']
                masks:
                  - $ref: "ns{{ locality }}/phone_library#masks/phone_mask"

`$ref` Syntax and Limitations in Libraries

Libraries can reference other values in the same library. To do this, use the same syntax, but with an empty library name, i.e. the reference consists only of the # symbol and the path.

For the format of path, see Path syntax.

Here is a simple example, where the retain_prefix_length parameter value is read from a different section of the library:

version: "1.0"

masks:
  phone_mask:
    type: from_unique_imitate
    retain_prefix_length:
      $ref: "#other/area_code_length"
    skip_letters: true

other:
  area_code_length: 3

Notice that it doesn't matter that the referenced value is defined after the place it is referenced. References can point to anywhere in the same library. The only restriction is that references must not form a circular chain, as in the following example:

# INVALID: circular reference
version: "1.0"

other:
  circular_reference_a:
    $ref: "#other/circular_reference_b"
  circular_reference_b:
    $ref: "#other/circular_reference_a"

Limitations

References are only valid within the seven main sections of the library. (masks, database_rules, tabular_file_rules, file_rules, columns, tasks, and other). Using $ref at the top level of the library, or in the version field, is not valid.
Currently, libraries can only reference themselves. Cross-library references are not supported.
Libraries cannot refer to rulesets. (As such, they do not contain the imports block that rulesets have.)
Locality interpolation is not supported in libraries.

Common Elements for both Rulesets and Libraries

Path Syntax

The path section of a reference is the full path to the value, starting from the top level of the library. It is written as a series of path segments separated by /. Each segment is either a dictionary key or an array index (zero-based).

Note: Path traversal via .. is not supported. As the path always starts from the top level of the library, it isn't necessary to use ...

Given this example library called my_library:

version: "1.0"

masks:
  phone_mask:
    type: from_unique_imitate
    retain_prefix_length: 3

other:
  my_dict:
    numbers:
      - 1
      - 2
      - 3
    us_states:
      - "Delaware"
      - "Wyoming"
      - "Connecticut"

then references resolve as detailed below:

# Points to the `phone_mask` under `masks`.
# Resolves to the dictionary {type: from_unique_imitate, retain_prefix_length: 3}.
$ref: "my_library#masks/phone_mask"

# Points to the array `numbers`.
# Resolves to the array [1, 2, 3].
$ref: "my_library#other/my_dict/numbers"

# Points to the second item of the states array (the first item has index zero).
# Resolves to the string "Wyoming".
$ref: "my_library#other/my_dict/us_states/1"

# Invalid: `phone_mask` does not exist at the top level of the library.
# Needs `masks/` prefix to locate the `phone_mask` within the `masks` section.
$ref: "my_library#phone_mask"

# Invalid: the array doesn't have enough elements.
$ref: "my_library#other/my_dict/us_states/5"

# Valid: refers to the last item in the array, i.e. the string "Connecticut".
# (Likewise, -2 is the second-to-last item in the array, and so on.)
$ref: "my_library#other/my_dict/us_states/-1"

Though an uncommon use case, you can refer to things within any section, including masks, tasks and so on. For example, in the above library, $ref: "my_library#masks/phone_mask/retain_prefix_length" is a valid reference which resolves to the number 3.

Data Types

All data types (scalars, arrays and dictionaries) can be replaced with a $ref. The result must still be valid for the ruleset to be usable. For example, replacing a boolean mask parameter with an array results in the ruleset not passing validation as the parameter is now of the wrong type.

Scalars

To replace a scalar, replace it with a dictionary containing only a $ref. Even when the value being replaced is a scalar, you still need to replace it with a dictionary.

# Correct: scalar replaced with dictionary containing only `$ref`
tasks:
  - type: mask_table
    table:  # Scalar replaced with dictionary
      $ref: "my_library#other/table_name"
    key: id
    rules: [...]

Show invalid examples

This is invalid YAML syntax:

# INVALID: bad YAML syntax
tasks:
  - type: mask_table
    table: $ref: "my_library#other/table_name"
    key: id
    rules: [...]

And this is also incorrect (it tries to mask a table literally named $ref: ...):

# INVALID: literal string value instead of dictionary
tasks:
  - type: mask_table
    table: '$ref: "my_library#other/table_name"'
    key: id
    rules: [...]

Arrays

To reference an array, replace the entire array with a dictionary containing only a $ref. For these examples, we'll assume a library declaring an array value like this:

# Library
version: "1.0"

other:
  order_statuses:
    - in_progress
    - shipped
    - delivered

To reference it in a ruleset, you replace the whole array with a dictionary. Do not make the $ref an array item (notice there is no - before $ref):

# Correct: array replaced with dictionary containing only `$ref`
rules:
  - column: order_status
    masks:
      - type: from_choices
        choices:
          $ref: "my_library#other/order_statuses"

Show invalid example

This example shows the $ref as an array item, which is incorrect:

# Wrong: will produce one choice, which is an array value
rules:
  - column: order_status
    masks:
      - type: from_choices
        choices:
          - $ref: "my_library#other/order_statuses"

This produces a nested array [ [in_progress, shipped, delivered] ]. The resulting mask will have only one choice of masked value, and that value will be the entire inner array. This will likely produce an error from the database engine, and even if not, the result will not be what was intended.

Adding extra items is not supported

It is not currently possible to add extra items to a referenced array. For example, this is invalid YAML syntax:

# INVALID: bad YAML syntax
rules:
  - column: order_status
    masks:
      - type: from_choices
        choices:
          $ref: "my_library#other/order_statuses"
          - cancelled

And this produces the same nested array issue as above, where some of the masked values may be arrays instead of strings:

# Wrong: produces a nested array
rules:
  - column: order_status
    masks:
      - type: from_choices
        choices:
          - $ref: "my_library#other/order_statuses"
          - cancelled

If you need extra items, you have to duplicate the array definition, either in the ruleset or in the library.

# Library
version: "1.0"

other:
  order_statuses:
    - in_progress
    - shipped
    - delivered
  order_statuses_with_cancelled:
    - in_progress
    - shipped
    - delivered
    - cancelled

Dictionaries

To reference a dictionary, replace the dictionary with one containing $ref. This is the most common case, used when referencing masks, rules and tasks. As an example, suppose the library defines the following mask:

# Library
version: "1.0"

masks:
  customer_code:
    type: from_unique_imitate
    skip_letters: true
    retain_prefix_length: 3

You can use it like this:

# Valid ruleset
version: "1.0"
imports:
  - my_library

tasks:
  - type: mask_tabular_file
    include:
      - glob: "*.csv"
    rules:
      - column: customer_code
        masks:
          - $ref: "my_library#masks/customer_code"

Notice that because the referenced mask is a dictionary, it still needs the array item marker - before $ref, as masks takes an array of mask dictionaries.

Overriding fields

Referencing dictionaries is the one case where you can add fields alongside $ref. Values at the same level as a $ref take priority over what that $ref reads in from the library. For example, using the same library with the customer_code mask:

# Valid ruleset demonstrating field addition and override
version: "1.0"
imports:
  - my_library

tasks:
  - type: mask_table
    table: customers
    key: id
    rules:
      - column: customer_code
        masks:
          - $ref: "my_library#masks/customer_code"
            retain_prefix_length: 4

The resulting mask has skip_letters set to true (from the library) and retain_prefix_length set to 4 (from the ruleset).

It is not possible to delete a field, but you can use the above method to set the field's value to whatever its documented default value is, normally null. This will have the same effect as if the field hadn't been specified.

# Valid ruleset demonstrating field reset to default
version: "1.0"
imports:
  - my_library

tasks:
  - type: mask_table
    table: customers
    key: id
    rules:
      - column: customer_code
        masks:
          - $ref: "my_library#masks/customer_code"
            retain_prefix_length: null

References to Ruleset Libraries

Introduction

Example

$ref Syntax and Limitations in Rulesets

imports block

Limitations

Locality Interpolation

$ref Syntax and Limitations in Libraries

Limitations

Common Elements for both Rulesets and Libraries

Path Syntax

Data Types

Scalars

Arrays

Dictionaries

Overriding fields

`$ref` Syntax and Limitations in Rulesets

`imports` block

`$ref` Syntax and Limitations in Libraries