DataMasque Portal

Upgrading to Projects

During an upgrade from a version without the Projects feature, ruleset libraries and discovery config libraries that have a namespace move into projects made from their namespaces, and all other resources move into a project named Default. Built-in seed files stay global, and belong to no project. These projects can be renamed or deleted after the upgrade.

What changes for users:

  • This version cannot be downgraded without a backup taken before the upgrade.
  • The upgrade can rename projects, connections and table references, and it rewrites references in the YAML of rulesets, libraries, ruleset generation configurations, discovery configurations and discovery config libraries to use the new names.
  • Deleting a project moves all its resources that are not deleted to the destination project you choose, or fails. Deleted resources and their runs stay with the deleted project.
  • Moving a ruleset or discovery configuration moves its earlier runs to the destination project's views.
  • In-flight masking ruleset plans that name an uploaded seed file without its project are disabled until you add the project (see Update in-flight masking ruleset plans).
  • A request that creates a resource must name its project when there is more than one active project (see Update scripts and API clients).

This page describes what the upgrade changes and what to check after the upgrade is complete.

Before upgrading

Take a complete backup of your DataMasque instance, as described in Backup and Restore. Downgrading from this version is not supported, so restoring a backup is the only way back.

What the upgrade does

Projects are created from namespaces

  • Existing resources are put into a project named Default, except libraries and discovery config libraries that have a namespace.
  • Each library namespace becomes a project with the same name, which contains the libraries of that namespace.
  • Runs are put into the project of their ruleset (for masking runs) or discovery configuration (for discovery runs).

Built-in seed files never belong to any project, and can be used from any project.

Renamed projects

Project names cannot contain / or .., and must be unique ignoring case. The upgrade renames each project made from a namespace that breaks these rules, and logs each rename:

  • Each / or .. in a name becomes an underscore (_), so sales_db/2026 becomes sales_db_2026.
  • When project names differ only by case, such as Sales and sales, all but one get a suffix consisting of an underscore and a number, such as Sales_2. The Default project keeps its name, or else the oldest project does.
  • When a project's new name is already taken by another project, or by a folder or file that DataMasque uses, it gets a suffix in the same way, so sales_db/2026 becomes sales_db_2026_2 if sales_db_2026 already exists.
  • A namespace named Default becomes a project such as Default_1, and the upgrade rewrites references such as Default/<library> to name that project.

A reference in a YAML document, such as a ruleset or library, that uses an old project name stops working. The upgrade process rewrites these references where possible (see Rewritten references). Where it cannot rewrite a reference, you must update it to use the new name.

Renamed table references and connections

  • Invalid characters such as / in table reference names are replaced with underscores (_).
  • Connection names must be unique within a project. When two or more connections have the same name, all but one get a suffix, such as customers_db_2.

Rewritten references

The upgrade updates the references stored in rulesets, ruleset generation configs, discovery configs and libraries to use the new names:

  • library names in imports and $ref
  • table reference names in table_reference
  • seed file names in seed_file, file_name, sql_file and sql_select_file, and in the filename of a discovery configuration's seed file matchers

The rest of the YAML is unchanged, including comments and quoting. The upgrade does not change any references that include {{ }} locality interpolation. You will need to update any references kept outside DataMasque, such as in Git or scripts.

Empty imports

imports must be a list, and each entry in it must name a library. If the ruleset, ruleset generation config or discovery config does not need any imports, you can use imports: [], or simply omit imports altogether.

  • In discovery configs, an imports: key that is set to null is an invalid value which was previously erroneously accepted as valid. The upgrade removes an imports: null (or any YAML equivalent) if it finds one, and clears the config's Git snapshot. The config's file in Git still has the invalid key, so a config that auto-pulls from Git cannot be used in a discovery run until you push the config from DataMasque, or make an edit to fix or remove the imports in the checked-in file.
  • Each ruleset, ruleset generation config and discovery config with an empty string ("") in its imports block is marked invalid. Remove any empty entries, then save it to validate it again.

Git file locations

Projects change where rulesets, libraries, discovery configs and discovery config libraries are stored in Git. The upgrade does not move or delete any files in your repository. After the upgrade, push each Git-synced resource (see Push Git-synced resources), then remove any old files that DataMasque no longer uses with git rm.

Note: All paths below are relative to the Git directory. This is the Directory path in Git Repository Settings, or your own directory if you set one on the My Account page.

Old file locations

Each resource created before the upgrade uses the following old file locations, until you rename the resource or move it to another project:

  • A ruleset uses <name>.yml at the top of the Git directory.
  • A discovery config uses discovery-configs/<name>.yml.
  • A library in the Default project uses libraries/<name>.yml.
  • A library in the project made from the Default namespace, such as Default_1, uses libraries/Default/<name>.yml.
  • A library in a project renamed for a / or .. in its name, such as sales_db_2026, uses its old folder, such as libraries/sales_db/2026/<name>.yml.
  • Any other library already uses its project's folder, libraries/<project>/<name>.yml.

Discovery config libraries follow the same rules as libraries, in discovery-config-libraries/.

New file locations

Resources created after the upgrade push to and pull from a folder named for their type and project, such as rulesets/Sales/customers.yml or libraries/Sales/emails.yml. A ruleset or discovery config created after the upgrade cannot auto-pull until you push it.

Resources created before the upgrade start using these new locations from their next push after you rename them or move them to another project. Until that push, a renamed or moved ruleset or discovery config cannot auto-pull. Their old files stay in the repository.

Name clashes

In these rare cases, two projects or resources had names that clash:

  • A project renamed because its name differed from another only by case, such as Sales_2, uses its new folders, such as libraries/Sales_2/. DataMasque no longer uses its old files in libraries/Sales/ and discovery-config-libraries/Sales/.
  • Rulesets with the same name, such as a database and a file ruleset, shared one file before the upgrade. The one pushed or pulled most recently, and not edited since, keeps that file; if neither has been synced since its last edit, the one created first keeps it. The other uses the new file location described above, which does not yet exist, so its auto-pull fails until you push it. Database and file discovery configs with the same name work the same way.
  • A new library in the Default project pushes to libraries/Default/<name>.yml, which could be the old file of a library in the project made from the Default namespace, such as Default_1. If so, the Default library cannot be pushed until you move or rename the other library. When you pull, files in libraries/Default/ are listed under the Default project, and pulling one into a new library sets its project to Default, so check the project before you save.

Note: Apart from the shared files above, a database and a file ruleset with the same name in one project cannot both be pushed, and neither can a database and a file discovery config: see Where ruleset files are.

After upgrading

Check the upgrade log

The upgrade writes a record of each rename, and any references or configs it could not update, to the masque_admin_server.log file. Administrators can download this log file by clicking Audit Logs under Admin in the sidebar, then clicking DOWNLOAD APPLICATION LOGS… and choosing Web application logs (see Application Logs). Clear the Recent logs only (zip under 10 MB, for email) checkbox before you download, because that option leaves out the older log files, which hold the upgrade lines. For example:

Renamed project "sales_db/2026" to "sales_db_2026".
Renamed table reference "crm/customers" in project "Default" to "crm_customers".
Left ruleset "nightly" (id 3f2a9c1e-8b4d-4e6f-9a2b-5c7d1e0f4a8b) in project "sales_db_2026" as written: its YAML could not be parsed, for example because a value with a `{{ }}` parameter isn't quoted.
Left ruleset "weekly" (id 7b0e4d21-5f3c-4a9e-8d16-c2a5f7e9b043) in project "sales_db_2026" as written: references "sales_db/2026/lib" could not be rewritten in place.
Left references in ruleset "monthly" (id e81c6a35-29d4-4b7f-a0e2-6f9d3b1c5a87) in project "Default" as written: references "sales_db/{{ locality }}/lib" name a project or table reference through a `{{ }}` parameter.
Left table reference "crm/customers" in ruleset library "customers" (id 9d1b7e42-0c3a-4f58-b6e1-2a4c8f9d0e73) in project "Default" as written: the rulesets that import it resolve it differently. After the upgrade it names table reference "customers" in project "crm", if there is one.
Left discovery configuration "pii-scan" (id 5c2e8a17-3d9f-4b61-a0c4-e7f2b9d63a15) as written: its YAML does not parse.
Left discovery configuration "card-scan" (id b4a91f6e-7c25-4d08-9e3b-1f6d2c8a5e47) as written: its empty `imports:` key is not on a line of its own.

Each Left … line names the ruleset or config by its id, which is in the address of its editor:

Resource in the log line Editor address
ruleset /ruleset-editor/<id>
ruleset library /libraries/<id>/edit
ruleset generation configuration /rg-configs/<id>/edit
discovery config /discovery-configs/<id>/edit
discovery config library /discovery-config-libraries/<id>/edit

For a discovery config left as written, remove its empty imports: key yourself (see Empty imports).

Fix references the upgrade didn't change

The upgrade leaves a reference unchanged when:

  • its project is named through a {{ }} parameter
  • the YAML of the ruleset or config does not parse, or the reference cannot be rewritten in place
  • it is a table reference in a library, and the rulesets that import the library resolve it differently

When such a reference no longer refers to the same resource as before the upgrade, the upgrade marks the ruleset or config invalid with a reason such as:

The upgrade left references in this ruleset as written: references "sales_db/{{ locality }}/lib" name a project or table reference through a `{{ }}` parameter. Fix them, then save a change to it to validate it again.

Rulesets and configs that import an invalid library are also invalid until the library is fixed. Saving a ruleset or config with unchanged YAML does not validate it again, so to keep a reference as it is, make a small edit, such as adding a comment, before you save.

A table reference left as written, such as crm/customers, now names the table reference customers in a project called crm, if one exists, so check each one.

Push Git-synced resources

Each ruleset, library, discovery config and discovery config library that the upgrade rewrites will now be out of sync with Git. After the upgrade, push each Git-synced resource before you pull it, so that a pull does not bring back references with the old names. For the Git file each resource uses after the upgrade, see Git file locations.

Update in-flight masking ruleset plans

In-flight masking does not use projects, so ruleset plans must name customer-uploaded seed files with their project, such as Default/first_names.csv (see In-flight masking).

The upgrade rewrites each plan that names an uploaded seed file by its bare name to Default/<name>, when Default holds a seed file of that name. Plans that use only built-in seed files are not changed.

A plan that the upgrade cannot rewrite is disabled, and the in-flight masking server logs its name and the reason. A masking request to that plan returns an error saying the plan is disabled. Update the plan to name the seed file as <project>/<file name>.

Move files copied into subfolders

When no seed file matches a name, DataMasque reads it as a path under the files directory, such as scripts/sql/setup.sql, and writes a warning to the run log. This applies to seed_file, sql_file, sql_select_file, the from_blob file_name and the email seed file options. Discovery configurations and in-flight masking do not read a path. Upload the file as a seed file, and name it as <file name> or <project>/<file name>.

A path under seeds/ or auth/ is refused.

Update scripts and API clients

  • The namespace field of ruleset libraries and discovery config libraries, and the endpoints that list namespaces, are deprecated and may be removed in a future version. Use the project field instead (see API Endpoints).
  • A request that creates a resource must name its project when there is more than one active project.
  • The Version 1 export and import endpoints are deprecated. Use GET /api/export/v2/ and POST /api/import/v2/ instead. A Version 2 export names the project that each resource is in.

Note: The deprecated namespace field replaces each / and .. with _ before it looks up the project, so the old name sales_db/2026 still finds the project sales_db_2026. It does not follow a suffix that the upgrade added to fix a name collision: if the project was renamed to sales_db_2026_2 because sales_db_2026 already existed, the old name finds the other project, sales_db_2026 (see The namespace field). Use the project field instead, with the project ID or the <id>-<slug> reference from GET /api/projects/.

Downgrading

Downgrading from this version is not supported. To return to an earlier version, restore a backup taken before the upgrade (see Backup and Restore).