DataMasque Portal

API Endpoints

Authentication

The DataMasque API uses token authentication. Tokens are 40-character strings containing 0-9 and a-f. Tokens should be included in the Authorization HTTP header for each request, with the word Token prepended.

For example

GET /runs/123/
Authorization: Token abcdef1234567890abcdef1234567890abcdef12

There are two types of authentication tokens:

  1. A non-expiring API Token which has access to only some endpoints. You can get this token from the My Account page.
  2. A User Token that is valid for only 12 hours, but has access to all endpoints. User tokens are granted by posting your username and password to the /api/auth/token/login/ endpoint.

The documentation for each endpoint on this page includes the type of token that is required to access it. If an endpoint does not require the use of the Authorization header then its authorization is noted as Anonymous.

The purpose and use case of each token type is explained below. Both token types serve distinct purposes within the DataMasque API, offering a balance between security and usability.

API Token

The API Token is a long-lived credential retrieved from the My Account page or the /api/users/me/ endpoint. It remains valid indefinitely, unless revoked (also on the My Account page). This token is valid only for use with specific API endpoints.

It is designed to be used in automated scripts whose content may not be stored securely, therefore it mainly has access to controlling masking runs and checking their status.

User Token

The User Token is exclusively issued after a successful login, either through the user interface or by making a request to /api/auth/token/login/.

This token offers enhanced security due to its limited lifetime, expiring after 12 hours, and is only accessible after a successful login. When accessing DataMasque through the UI, the token is granted as a cookie which will expire after 1 hour of inactivity.

It can be used against all API endpoints, and grants access based on the user account's permissions.

To revoke a user token before it expires, use the /api/auth/token/logout/ endpoint.

POST /api/auth/token/login/

Authorization: Anonymous.

Login with a username and password to obtain a user_token.

POST /api/auth/token/login/ Parameters

Field Type Required Location Description
username string Yes Request Body The username of the user you are logging in as.
password string Yes Request Body The password for the user.

POST /api/auth/token/login/ Responses

Status Code Description
200 A JSON serialised login object, including a short-lived API key.

POST /api/auth/token/login/ Postman example

  1. Open Postman.
  2. Create a new request.
  3. Set the method to POST and the URL to https://<your-datamasque-host>/auth/token/login/.
  4. Under Headers, add Content-Type as a key and set the value as application/json.
  5. Select the Body tab then the raw button.
  6. Include your DataMasque login details in this format in the text editor shown:
{
  "username": "<your-username>",
  "password": "<your-password>"
}
  1. Press the blue Send button to the right of the URL bar.

POST /api/auth/token/login/ curl example

curl -X POST "https://<your-datamasque-host>/api/auth/token/login/" \
     -H "Content-Type: application/json" \
     -d '{"username": "<your-username>", "password": "<your-password>"}'

Login Object

Field Type Description
key string The short-lived API token to be included in the Authorization header on subsequent API requests.
client_ip string Origin IP address of the login request.
client_browser string Browser, user agent or program making the login request.
client_os string Operating system of the computer making the login request. For automation tools like curl, this will be set to Other.
client_device string Device type of the computer making the login request. For automation tools like curl, this will be set to Other.
date_time_created string Date and time this login request was made, in ISO 8601 format and UTC timezone. Example: 2025-11-30T20:44:01.094138Z
date_time_expires string Date and time this login request expires, in ISO 8601 format and UTC timezone. Example: 2025-12-01T08:44:01.093754Z
  • The key value is the short-lived authorization token to be passed on the Authorization header, for example Authorization: Token abcdef1234567890abcdef1234567890abcdef12.
  • The date_time_expires indicates the UTC time after which the authorization token in the key field expires. After this time, requests using this authorization token are rejected with status code 401. Make a new request to /api/auth/token/login/ to obtain a new authorization token.

POST /api/auth/token/logout/

Authorization: User token only.

Logs out the supplied user token. After logging out, the token can no longer be used for authenticating API requests; a new token must be obtained from the /api/auth/token/login/ endpoint.

POST /api/auth/token/logout/ Parameters

No parameters.

POST /api/auth/token/logout Responses

Status Code Description
204 Token successfully logged out. The token can no longer be used to authenticate requests.

POST /api/auth/jwt/login/

Authorization: Anonymous.

Login with a username and password to obtain a JSON Web Token (JWT), for use with in-flight masking. The response contains both an access_token (for authorization), and a refresh_token (to request another access_token once it expires).

The lifetime of the tokens are:

  • access_token: 12 hours
  • refresh_token: 24 hours

JWTs also include information about the user's role(s) and DataMasque licensing. Therefore, if you make changes to a user's role, or update the DataMasque license, a new JWT to should be requested to get the most up-to-date details.

Note: SSO Users are unable to authenticate with username and password, so they are unable to get JWTs.

POST /api/auth/jwt/login/ Parameters

Field Type Required Location Description
username string Yes Request Body The username of the user you are logging in as.
password string Yes Request Body The password for the user.

POST /api/auth/jwt/login/ Responses

Status Code Description
200 Login was successful. A JWT credentials object is returned in the response.

JWT Credentials Object

Field Type Description
access_token string The token to be included in the Authorization header when using in-flight masking endpoints.
refresh_token string A token that can be used to request another access_token, if the access_token expires.
token_type string The prefix to include in the Authorization header before the access_token. Is always the string Bearer.

POST /api/auth/jwt/login/ curl example

curl -X POST "https://<your-datamasque-host>/api/auth/jwt/login/" \
     -H "Content-Type: application/json" \
     -d '{"username": "<your-username>", "password": "<your-password>"}'

POST /api/auth/jwt/refresh/

Authorization: Anonymous.

POST a JWT refresh token and receive a JWT access token with a renewed expiry.

POST /api/auth/jwt/refresh/ Parameters

Field Type Required Location Description
refresh string Yes Request Body The refresh token from the JWT Credentials Object

POST /api/auth/jwt/refresh/ Responses

Status Code Description
200 The posted refresh token is valid. A JWT Refresh Token Object is returned in the response.

JWT Refresh Token Object

Field Type Description
access_token string The token to be included in the Authorization header when using in-flight masking endpoints.

POST /api/auth/jwt/refresh/ curl example

curl -X POST "https://<your-datamasque-host>/api/auth/jwt/refresh/" \
     -H "Content-Type: application/json" \
     -d '{"refresh": "<refresh-token>"}'

POST /api/auth/jwt/verify/

Authorization: Anonymous.

POST a JWT refresh token and receive a 200 status code if it is valid, or 401 status code otherwise.

POST /api/auth/jwt/verify/ Parameters

Field Type Required Location Description
token string Yes Request Body A JWT access token.

POST /api/auth/jwt/verify/ Responses

Status Code Description
200 The token is valid.
401 The token invalid due to incorrect signature or it has expired. Check the response body for more information..

POST /api/auth/jwt/verify/ curl example

curl -X POST "https://<your-datamasque-host>/api/auth/jwt/verify/" \
     -H "Content-Type: application/json" \
     -d '{"token": "<access-token>"}'

User Object

User objects have the following fields:

Field Type Description
id integer The id of the User.
username string The username for the User. Used when logging in.
email string The email of the User.
date_joined date The date the User was created.
api_token string The API token for the User. Only available via /api/users/me/ for security reasons.
has_temporary_password boolean Whether user has a temporary password or not. If true, the user has not finalised their account creation.
is_active boolean Whether or not the user account is active. If false, the account is disabled.
is_staff boolean Whether or not the user is a staff account.
is_superuser boolean Whether or not the account is a superuser and has admin privileges.
is_sso_user boolean Whether or not the account is an SSO enabled account.
is_subscribed_to_sdd_updates boolean Whether or not the user has subscribed to sensitive data discovery updates.
user_roles array[string] List of roles assigned to the user. Full list of roles can be found in User Roles
user_permissions array[string] List of permissions assigned to the user.

User Roles

User objects may be assigned the below roles, as part of their user_roles array. All roles are mutually exclusive, except that users with the mask_builder role can also have the ruleset_library_managers role.

Role Description
admin Superuser role. Can perform all operations without restriction.
mask_runner A user with this role is responsible solely for executing masking operations.
mask_builder In addition to the capabilities of the mask_runner role, this role includes the ability to create and manage rulesets.
ruleset_library_managers Can be specified alongside mask_builder to grant the user the ability to create, delete and edit ruleset libraries.

GET /api/users/

Authorization: Admin User token only.

Returns a list of user accounts.

GET /api/users/ Parameters

No parameters.

GET /api/users/ Responses

Status Code Description
200 Returns a JSON serialised list of User objects.

GET /api/users/ curl example

curl "https://<your-datamasque-host>/api/users/" \
     -H "Authorization: Token <your-api-token>"

GET /api/users/{id}/

Authorization: Admin User token or the user themselves.

Retrieve information about a specific user.

GET /api/users/{id}/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the user.

GET /api/users/{id}/ Responses

Status Code Description
200 A JSON serialized User object for the specified user.
403 Forbidden: If the token does not have the required permissions.
404 Not Found: If the user with the specified id does not exist.

GET /api/users/{id}/ curl example

curl "https://<your-datamasque-host>/api/users/{id}/" \
     -H "Authorization: Token <your-api-token>"

`

GET /api/users/me/

Authorization: User token only.

Returns the details of the currently logged-in user.

GET /api/users/me/ Responses

Status Code Description
200 Returns a JSON serialised User object for the user that is currently logged in.

GET /api/users/me/ curl example

curl "https://<your-datamasque-host>/api/users/me/" \
     -H "Authorization: Token <your-api-token>"

POST /api/users/me/ curl example

curl -X POST "https://<your-datamasque-host>/api/users/me/profile/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{"git_directory_path": "path/to/root"}'

POST /api/users/

Authorization: Admin User token only.

Create a new user account.

POST /api/users/ Parameters

Field Type Required Location Description
username string Yes Request Body The username of the user being created.
password string Yes Request Body The password for the new user account.
re_password string Yes Request Body The password for the new user again, to confirm the password entered above.
email string Yes Request Body The email address of the new user.
user_roles array[string] No Request Body The role(s) assigned to the user. If provided, the user will be added to the specified group(s). Defaults to no role which has the same permissions as mask_runner. See User Roles for valid values. The deprecated field name role is also accepted for backward compatibility.
role array[string] No Request Body Deprecated -- alias for user_roles. Accepted for backwards compatibility; prefer user_roles.
has_temporary_password boolean No Request Body Whether user has a temporary password or not. If true, the user has not finalised their account creation. Defaults to true when not specified.

POST /api/users/ Responses

Status Code Description
201 A JSON serialized User object for the created user.
400 Bad Request: If the request data is invalid or user creation is disabled.
403 Forbidden: If the token does not have the required permissions.

POST /api/users/ curl example

curl -X POST "https://<your-datamasque-host>/api/users/" \
     -H "Authorization: Token <your-admin-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "username": "<your-new-username>",
           "password": "<your-new-password>",
           "re_password": "<your-new-password>",
           "email": "<your-new-email>",
           "user_roles": ["<your-user-role>"]
         }'

GET /api/users/me/

Authorization: User token only.

Returns the details of the currently logged-in user.

GET /api/users/me/ Responses

Status Code Description
200 Returns a JSON serialised User object for the user that is currently logged in.

GET /api/users/me/ curl example

curl "https://<your-datamasque-host>/api/users/me/" \
     -H "Authorization: Token <your-api-token>"

GET /api/users/{id}/

Authorization: Admin User token (to query any user's details) or the queried user's token.

Retrieve information about a specific user.

GET /api/users/{id}/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the user.

GET /api/users/{id}/ Responses

Status Code Description
200 A JSON serialized User object for the specified user.
403 Forbidden: If the token does not have the required permissions.
404 Not Found: If the user with the specified id does not exist.

GET /api/users/{id}/ curl example

curl "https://<your-datamasque-host>/api/users/{id}/" \
     -H "Authorization: Token <your-api-token>"

PATCH /api/users/{id}/

Authorization: Admin User token (to update any user's details) or the updating user's token.

Partially update information for a specified user.

PATCH /api/users/{id}/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the user to update.
username string No Request Body The new username of the user. Only an Admin User can update this.
email string No Request Body The new email address of the user. An Admin User or the user themselves can update this.
date_joined date No Request Body The date the User was created.
is_active boolean No Request Body Whether or not the user account is active. If false, the account is disabled. Only admin users can modify.
is_subscribed_to_sdd_updates boolean No Request Body Whether or not the user has subscribed to sensitive data discovery updates.
user_roles array[string] No Request Body The role(s) assigned to the user. If provided, the user will be added to the specified group(s). Only admin users can modify.
current_password string No Request Body The current password for the user. Needs to be specified when changing passwords, along with new_password and re_new_password.
new_password string No Request Body The new password for the user. Password changes are restricted to admin users and the account owner.
re_new_password string No Request Body The repeated new password for the user. Used when changing passwords.

PATCH /api/users/{id}/ Responses

Status Code Description
200 A JSON serialized User object for the updated user.
400 Bad Request: If the request data is invalid.
403 Forbidden: If the token does not have the required permissions.
404 Not Found: If the user with the specified id does not exist.

PATCH /api/users/{id}/ curl example

curl -X PATCH "https://<your-datamasque-host>/api/users/{id}/" \
     -H "Authorization: Token <your-admin-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "username": "<your-new-username>",
           "email": "<your-new-email>",
           "user_roles": ["<user-role>"]
         }'

PUT /api/users/{id}/

Authorization: Admin User token (to update any user's details) or the updating user's token.

Update information for a specified user.

PUT /api/users/{id}/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the user to update.
username string No Request Body The new username of the user. Only an Admin User can update this.
email string No Request Body The new email address of the user. An Admin User or the user themselves can update this.
user_roles array[string] No Request Body The role(s) assigned to the user. If provided, the user will be added to the specified group(s). Only an Admin User can update this.

PUT /api/users/{id}/ Responses

Status Code Description
200 A JSON serialized User object for the updated user.
400 Bad Request: If the request data is invalid.
403 Forbidden: If the token does not have the required permissions.
404 Not Found: If the user with the specified id does not exist.

PUT /api/users/{id}/ curl example

curl -X PUT "https://<your-datamasque-host>/api/users/{id}/" \
     -H "Authorization: Token <your-admin-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "username": "<your-new-username>",
           "email": "<your-new-email>",
           "user_roles": ["<user-role>"]
         }'

POST /api/users/{id}/reset-password/

Authorization: Admin User token only.

Reset the password for a specified user.

POST /api/users/{id}/reset-password/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the user whose password is being reset.

POST /api/users/{id}/reset-password/ Responses

Status Code Description
200 Returns a JSON object with the new temporary password.
403 Forbidden: If the token does not have the required permissions.
404 Not Found: If the user with the specified id does not exist.

POST /api/users/{id}/reset-password/ curl example

curl -X POST "https://<your-datamasque-host>/api/users/{id}/reset-password/" \
     -H "Authorization: Token <your-admin-api-token>" \
     -H "Content-Type: application/json"

DELETE /api/users/{id}/

Authorization: Admin User token only.

Permanently delete a user account. All associated personal data (tokens, profile, SSH keys, notifications) is removed. Audit logs and masking run history are preserved with the user reference set to null.

An admin cannot delete their own account. The last remaining active admin user cannot be deleted.

DELETE /api/users/{id}/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the user.

DELETE /api/users/{id}/ Responses

Status Code Description
204 The user has been permanently deleted.
403 Forbidden: Cannot delete own account, or cannot delete the last active admin user.
404 Not Found: If the user with the specified id does not exist.

DELETE /api/users/{id}/ curl example

curl -X DELETE "https://<your-datamasque-host>/api/users/{id}/" \
     -H "Authorization: Token <your-admin-api-token>"

Profile Object

A Profile object stores settings for a particular user. There is a one-to-one relationship between a user and their Profile. A Profile object may only be updated by the user that it belongs to (i.e. a user can only update their own Profile, admins cannot update Profiles of other users).

Profile objects have the following fields:

Field Type Description
git_directory_path string The Git directory path for this user when pushing/pulling rulesets to/from a Git repository.

Extra Field Notes

git_directory_path

This overrides the global Git directory for the DataMasque instance, for this user only. This value can be set even if Git integration is disabled, it will just have no effect.

GET /api/users/me/profile/

Authorization: User token only.

Returns the Profile Object for the currently logged-in user.

GET /api/users/me/profile/ Parameters

No parameters.

GET /api/users/me/profile/ Responses

Status Code Description
200 Returns a JSON serialised Profile object, with fields as described above.

GET /api/users/me/profile/ Parameters

No parameters.

GET /api/users/me/profile/ curl example

curl "https://<your-datamasque-host>/api/users/me/profile/" \
     -H "Authorization: Token <your-api-token>"

POST /api/users/me/profile/

Authorization: User token only.

Updates the Profile object for the current user. Partial updates are supported: only fields that are contained in the request will be updated (i.e. if a field is not present in the request then its stored value remains unchanged).

POST /api/users/me/profile/ Responses

Status Code Description
204 The Profile update was successful.

Run Object

Run objects have the following fields:

Field Type Description
id integer The id of the Run. Use this in API URLs that need a run id.
name string The name of the Run.
status string Indicates the Run status. The potential values are: queued, running, finished, finished_with_warnings, failed, cancelling, and cancelled. A status of finished or finished_with_warnings indicates the Run completed successfully; failed indicates an error. finished_with_warnings indicates there were warnings during the run, refer to the run log to view them.
mask_type string The masking type of the Run, valid options are "database" or "file".
connection string Deprecated, replaced by source_connection.
connection_name string Deprecated, replaced by source_connection_name.
source_connection string A UUID identifying the source connection used for this Run. For database connections, the source_connection also acts as the destination.
source_connection_name string The name of the source connection of the Run. For database connections, the source_connection also acts as the destination.
destination_connection Optional[string] A UUID identifying the destination connection used for this Run. Only present for file connections, as the source_connection also acts as the destination for database connections.
destination_connection_name Optional[string] The name of the destination connection of the Run. Only present for file connections, as the source_connection also acts as the destination for database connections.
ruleset string A UUID identifying the ruleset used for this Run.
ruleset_name string Ruleset name of the Run.
start_time string Start time of the Run, in ISO 8601 format.
end_time string End time of the Run, in ISO 8601 format.
options object An Option object of configuration for the Run.

GET /api/runs/

Authorization: User token or API token.

Get a list of DataMasque Runs.

GET /api/runs/ Parameters

Field Type Required Location Description
mask_type string No Query Parameter The mask type of the Run. The potential values are: database, file.
connection_ruleset_name string No Query Parameter The name of the source or destination connection name or the ruleset name of the Run.
status string No Query Parameter The status of the Run. The potential values are: queued, running, finished, finished_with_warnings, failed, cancelling, and cancelled.
limit integer No Query Parameter The number of results to return per page. E.g. 5
offset integer No Query Parameter The initial index from which to return the results. E.g. 0

GET /api/runs/ Responses

Status Code Description
200 A JSON serialised list of Run objects.

GET /api/runs/ curl example

curl "https://<your-datamasque-host>/api/runs/" \
     -H "Authorization: Token <your-api-token>"

GET /api/runs/ curl example with parameters

curl "https://<your-datamasque-host>/api/runs/?mask_type=database&run_status=running&limit=20&offset=40" \
     -H "Authorization: Token <your-api-token>"

POST /api/runs/

Authorization: User token or API token.

Start a new masking run.

POST /api/runs/ Parameters

Field Type Required Location Description
name string Yes Request Body The name of the Run.
connection string No Request Body Deprecated, replaced by source_connection.
source_connection string Yes Request Body A UUID identifying the source connection to be used for this Run. For database connections, the source_connection also acts as the destination.
destination_connection string Required only for runs on file connections. Request Body A UUID identifying the destination connection to be used for this Run.
ruleset string Yes Request Body A UUID identifying the ruleset to be used for this Run.
options object Yes Request Body An Option object of configuration for this Run.
mask_type string No Request Body A string "database", "file" or "file_data_discovery" to indicate the type of masking run. Defaults to "database" if only source_connection is specified, or "file" if destination_connection is also specified.

Note: To start a schema discovery or file data discovery run, the endpoints POST /api/schema-discovery/ or POST /api/run-file-data-discovery/ offer easier-to-use APIs.

POST /api/runs/ Responses

Status Code Description
200 A JSON serialised Run object.

POST /api/runs/ curl example

curl -X POST "https://<your-datamasque-host>/api/runs/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "name": "<run-name>",
           "source_connection": "<source-connection-uuid>",
           "destination_connection": "<destination-connection-uuid>",  # Include this only if required
           "ruleset": "<ruleset-uuid>",
           "options": {
             #... option object details ...
           }
         }'

GET /api/runs/{id}/

Authorization: User token or API token.

Retrieve information about a masking run.

GET /api/runs/{id}/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the Run.

GET /api/runs/{id}/ Responses

Status Code Description
200 A JSON serialised Run object.

GET /api/runs/{id}/ curl example

curl "https://<your-datamasque-host>/api/runs/{id}/" \
     -H "Authorization: Token <your-api-token>"

POST /api/runs/{id}/cancel/

Authorization: User token or API token.

Cancel a masking run.

GET /api/runs/validate/

Authorization: User token or API token.

Validate that the run actually occurred.

GET /api/runs/validate/ Parameters

Field Type Required Location Description
run_hash string Yes Query Parameter The hash of the run that can be retrieved from run_hash column in the DATAMASQUE_RUN_HISTORY table.
run_completion_time string Yes Query Parameter The finish time of the run that can be retrieved from the run log or from the completion_time column in the DATAMASQUE_RUN_HISTORY table. It must be in the datetime format: %Y-%m-%d %H:%M:%S
ruleset_content_sha256 string Yes Query Parameter The hash of the ruleset that can be retrieved from the run log or from the ruleset_content_sha256 column in the DATAMASQUE_RUN_HISTORY table.

GET /api/runs/validate/ curl example

Given the run log contains:

SHA256 hash of ruleset: 7ee08ef63db7fed2baf577f16d74427c2250ba05f6858b0a27b70e05ccbff6eb

Finished At: 2024-05-22 22:11:35 UTC

The DATAMASQUE_RUN_HISTORY table has:

run_hash: 8d34cc930ce7eae40a633e95aef3aee5d2108511eb20ac35805f2e0834115bb9

curl -X GET "https://<your-datamasque-host>/api/runs/validate/?run_hash=8d34cc930ce7eae40a633e95aef3aee5d2108511eb20ac35805f2e0834115bb9&run_completion_time=2024-05-22 22:11:35&ruleset_content_sha256=7ee08ef63db7fed2baf577f16d74427c2250ba05f6858b0a27b70e05ccbff6eb" \
     -H "Authorization: Token <your-api-token>"

POST /api/runs/{id}/cancel/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the Run.

POST /api/runs/{id}/cancel/ Responses

Status Code Description
201 Operation succeeded

POST /api/runs/{id}/cancel/ curl example

curl -X POST "https://<your-datamasque-host>/api/runs/{id}/cancel/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json"

GET /api/runs/{id}/sdd-report/

Authorization: User token only.

A binary serialised SDD Report object.

GET /api/runs/{id}/sdd-report/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the Run.

GET /api/runs/{id}/sdd-report/ Responses

Status Code Description
200 The server will return the SDD Report in the response body which can be downloaded as a CSV file.
404 If there is no SDD Report for a run, the server will return 404 status code.

GET /api/runs/{id}/sdd-report/ curl example

curl "https://<your-datamasque-host>/api/runs/{id}/sdd-report/" \
     -H "Authorization: Token <your-api-token>"

GET /api/runs/{id}/run-report/

Authorization: User token only.

A binary serialised Run Report object.

GET /api/runs/{id}/run-report/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the Run.

GET /api/runs/{id}/run-report/ Responses

Status Code Description
200 The server will return the Run Report in the response body which can be downloaded as a CSV file.
404 If there is no Run Report for a run, the server will return 404 status code.

GET /api/runs/{id}/run-report/ curl example

curl "https://<your-datamasque-host>/api/runs/{id}/run-report/" \
     -H "Authorization: Token <your-api-token>"

DELETE /api/runs/{id}/db-discovery-results/

Deletes the database discovery results for a run. Use this only when the results are no longer needed, for instance because you have completed another discovery run on the same database more recently.

Warning! Deletion of results is irreversible.

Note: This endpoint can only be used to delete discovery results that were created on versions of DataMasque v2.22 and later. It is not possible to delete discovery results from versions prior to v2.22.

DELETE /api/runs/{id}/db-discovery-results/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the Run.

DELETE /api/runs/{id}/db-discovery-results/report/ Responses

Status Code Description
204 Deletion was successful.
404 Not Found: There are no database discovery results for this run, or a run with the specified ID does not exist.

DELETE /api/runs/{id}/db-discovery-results/ curl example

curl -X DELETE "https://<your-datamasque-host>/api/runs/{id}/db-discovery-results/" \
     -H "Authorization: Token <your-api-token>"

GET /api/runs/{id}/db-discovery-results/report/

Downloads database schema discovery results as a CSV.

GET /api/runs/{id}/db-discovery-results/report/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the Run.

GET /api/runs/{id}/db-discovery-results/report/ Responses

Status Code Description
200 The server will return the discovery results in the response body which can be downloaded as a CSV file.
404 Not Found: There are no database discovery results for this run, or a run with the specified ID does not exist.

GET /api/runs/{id}/db-discovery-results/report/ curl example

curl -o report.csv "https://<your-datamasque-host>/api/runs/{id}/db-discovery-results/report/" \
     -H "Authorization: Token <your-api-token>"

Option Object

Option objects have the following fields:

Field Type Description
batch_size integer Specifies the number of rows to fetch in each batch retrieved from the database for masking. Must be a positive integer, at least 1. Defaults to 50,000. This is ignored for file masking.
dry_run boolean Indicates a dry run where no data in the database is actually changed. Values should either be true to indicate a dry run, or false to run normally. Default value is false. More information on dry runs is available in the Masking runs documentation.
max_rows integer Specifies the maximum number of rows that will be masked by each mask_table task1. Must be a positive integer, at least zero; zero is interpreted as no limit. Defaults to no limit. This is ignored for file masking.
continue_on_failure boolean If there is a task failure, and this option is false, DataMasque will skip all remaining unstarted tasks. If this option is true, DataMasque will continue performing other tasks even if there is a task failure. Default value is false.
run_secret string The run secret is used in the random generation of masked values. If left unspecified, a random secret will be automatically generated and returned in the API response 2. Masking runs performed on the same DataMasque instance with the same run secret will produce the same masked values for identical unmasked database inputs. You should only specify a run secret if you require consistent masking across runs, otherwise it is more secure to allow a new run secret to be automatically generated for each run. Run secrets must be at least 20 characters long.
disable_instance_secret boolean If this option is set to true, DataMasque will exclude its instance-specific secret and generate masked values based solely on the run secret. You may wish to disable the instance in order to achieve consistent masking across DataMasque instances. However, by disabling the instance secret, any DataMasque instance using the same run_secret could replicate your data masking.
diagnostic_logging boolean If set to true, the run log will include information to help diagnose errors. This includes information about the tables, columns and keys being masked, memory usage information and more verbose output. Defaults to false.
maxfilesize integer Maximum export file size in MB for Redshift and DynamoDB masking. This controls how much data is loaded into memory at once when masking. Must be an integer between 5 and 1,000. Defaults to 5MB for Redshift and 100MB for DynamoDB.
buffer_size (deprecated; will be removed in release 3.0.0) integer Replaced by batch_size.

1 max_rows does not apply to mask_unique_key tasks.

2 The run_secret contained in the API response can be provided in subsequent API calls to start runs, facilitating consistent masking across those runs.

Additionally, the following options apply to schema discovery runs (i.e. runs that include at least one run_schema_discovery task):

Field Type Description
custom_keywords array[string] List of keywords that, where a column's name matches one or more of the keywords, indicates the column contains sensitive data. Default value is an empty list.
ignored_keywords array[string] List of keywords that, where a column's name matches one or more of the keywords, indicates the column should be excluded from the schema discovery results. Default value is an empty list.
disable_global_custom_keywords boolean If set to true, then the user-defined global set of custom keywords will not be used to flag columns as sensitive. Default value is false.
disable_global_ignored_keywords boolean If set to true, then the user-defined global set of ignored keywords will not be used to exclude columns from the schema discovery results. Default value is false.
disable_built_in_keywords boolean If set to true, then DataMasque's built-in list of keywords will not be used to flag columns as sensitive. Default value is false.
schemas array[string] List of schema (database for MySQL/MariaDB) names against which to perform schema discovery. Default value is an empty list, meaning schema discovery will run against the schema configured on the database connection, or the database user's default schema. Default value is an empty list.

Runlog Object

Runlog objects have the following fields:

Field Type Description
run integer ID of the Run this Runlog was generated for.
timestamp string Timestamp of this Runlog's generation, in ISO 8601 format.
message string The log message passed from the masking worker.
log_level integer Numeric representation of the log level, values are 20 for INFO, 30 for WARNING, and 40 for ERROR.
status string Indicates the Run status. The potential values are: queued, running, finished, finished_with_warnings, failed, cancelling, and cancelled. A status of finished or finished_with_warnings indicates the Run completed successfully; failed indicates an error. finished_with_warnings indicates there were warnings during the run, refer to the run log to view them.
is_dry_run boolean Indicates whether the Run is a dry run.

GET /api/runs/{id}/log/

Authorization: User token or API token.

List all logs for a specified Run in a JSON response.

GET /api/runs/{id}/log/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the Run.
limit integer No Query Parameter The maximum number of RunLog entries to return.
offset integer No Query Parameter The starting position of the query in relation to the complete set of RunLogs for this Run.
ordering integer No Query Parameter Controls the order of the results. Available fields to order by are id and timestamp. Reverse the order by prefixing the field name with -. Multiple orderings may be specified separated by a comma.

GET /api/runs/{id}/log/ Responses

Status Code Description
200 A JSON serialised list of Runlog objects. Default is to return the all the logs for the run.

GET /api/runs/{id}/log/ curl examples

Fetch the complete run log:

curl "https://<your-datamasque-host>/api/runs/{id}/log/" \
     -H "Authorization: Token <your-api-token>"

Fetch the first 25 logs:

curl "https://<your-datamasque-host>/api/runs/{id}/log/?limit=25&offset=0" \
     -H "Authorization: Token <your-api-token>"

Fetch logs from 50-100:

curl "https://<your-datamasque-host>/api/runs/{id}/log/?limit=50&offset=50" \
     -H "Authorization: Token <your-api-token>"

Order by timestamp and id descending (newest first):

curl "https://<your-datamasque-host>/api/runs/{id}/log/?ordering=-timestamp,-id" \
     -H "Authorization: Token <your-api-token>"

GET /api/runs/{id}/log/download/

Authorization: User token only.

All logs for a specified Run in a plain text file.

GET /api/runs/{id}/log/download/ Parameters

Field Type Required Location Description
timezone string Yes Query Parameter Timezone offset to use for the Run logs in the format +HH:MM or -HH:MM. Example: +07:00, -05:00.

GET /api/runs/{id}/log/download/ Responses

Status Code Description
200 The server will return the Run Log content in the response body which can be downloaded as a log file.

GET /api/runs/{id}/log/download/ curl example

curl "https://<your-datamasque-host>/api/runs/{id}/log/download/?timezone=+07:00" \
     -H "Authorization: Token <your-api-token>"

Connection Object

Database Connection objects have the following fields:

Field Type Description
version string The connection version. This should be set to `1.0'.
id integer The id of the Connection. Use this in API URLs that need a connection id.
name string The name of the Connection.
user string The name of the user in the database connection.
db_type string The type of database the connection is connecting to.
database string The database the connection is connecting to.
host string The hostname of the database connection.
port integer The database port being connected through.
dbpassword string The password for the user connecting to the database.
schema string The schema of the database to connect to.
options object An Option object of configuration for the Run
service_name string Only used for Oracle connections. The service name for the connection. (Optional)
oracle_wallet string Only for Oracle connections The ID of the oracle-wallet attached with the connection, otherwise is null. (Optional)
connection_fileset string Only for MySQL and MariaDB connections. The ID of the connection fileset attached to this connection. (Optional)
mask_type string The type of masking the connection can perform, only database or file are valid. (Optional) Should be set to database for database Connections.
last_discovery_run_date string The created_time of the last run on this connection including a run_schema_discovery task, or null if no such run has been performed.
last_discovery_run_id string The ID of the last run on this connection including a run_schema_discovery task, or null if no such run has been performed.
is_read_only boolean Whether or not the connection to the database is read-only.
data_encoding string Only for Oracle, Postgres, MySQL, and MariaDB connections An encoding to be used when retrieving data containing different character sets from the database. Should match the encoding of the data stored, not the character set of the database. The list of supported encodings can be found on the Database Connections page.
iam_role_arn string Only for Amazon DynamoDB or Snowflake connections The IAM role ARN for DataMasque to assume role. For Snowflake, only used when snowflake_stage_location is aws_s3.
snowflake_account_id string Only for Snowflake connections Snowflake account identifier. Does not include the snowflakecomputing.com.
snowflake_warehouse string Only for Snowflake connections Default Snowflake warehouse to
snowflake_storage_integration_name string Only for Snowflake connections The storage integration that has access to the S3 bucket or Azure Blob Storage container, for retrieving staged data.
snowflake_role string Only for Snowflake connections The default Snowflake role to use. (Optional)
snowflake_private_key Optional[string] Only for Snowflake connections The file identifier of the Snowflake private key attached to this connection. (Optional)
snowflake_private_key_passphrase Optional[string] Only for Snowflake connections The passphrase for the Snowflake private key. (Optional)
snowflake_stage_location string Only for Snowflake connections Storage location type for Snowflake stage. Choose between aws_s3 or azure storage for intermediate data processing. Defaults to aws_s3.
s3_bucket_name string Only for Snowflake connections The name of the AWS S3 bucket used for intermediate data processing. Required when snowflake_stage_location is aws_s3.
snowflake_azure_container_name string Only for Snowflake connections The name of the Azure Blob Storage Container used for intermediate data processing. Required when snowflake_stage_location is azure.
snowflake_azure_connection_string string Only for Snowflake connections The Connection String used to connect to the Azure Blob Storage Container. Required when snowflake_stage_location is azure. (Note: This field is returned encrypted as snowflake_azure_connection_string_encrypted in GET responses)

File Connection objects have the following fields:

Field Type Description
version string The connection version. This should be set to `1.0'.
id integer The id of the Connection. Use this in API URLs that need a connection id.
name string The name of the Connection.
type string The type of file system the connection is connecting to. Valid options are "s3_connection", "azure_blob_connection" or "mounted_share_connection".
base_directory string The root file path where files intended to be masked are stored.
bucket string The name of the S3 bucket containing the base_directory. Only for S3 Connections.
container string The name of the Azure Blob Storage container containing the base_directory. Only for Azure Blob Connections.
connection_string string The connection string configured with the authorization information to access data in your Azure Storage account. Only for Azure Blob Connections.
mask_type string The type of masking the connection can perform, only database or file are valid. (Optional) Should be set to file for file Connections.
is_file_mask_source boolean A boolean if the connection is a source Connection for file masking. (Optional) Defaults to false if not provided.
is_file_mask_destination boolean A boolean if the connection is a destination Connection for file masking. (Optional) Defaults to false if not provided.

GET /api/connections/

Authorization: User token only.

Get a list of all DataMasque connections.

Optionally, you can add an {id} to the end of the request to only return the details of the connection with that specific id.

GET /api/connections/ Parameters

Can optionally follow the URL with the id of a specific connection to only return information on that connection.

GET /api/connections/ Responses

Status Code Description
200 A JSON serialised Connection object.

Quickstart example using curl

curl "https://<your-datamasque-host>/api/connections/" \
     -H "Authorization: Token <your-api-token>"

POST /api/connections/

Authorization: User token only.

Create a new connection object.

POST /api/connections/ Parameters

Database Connections

Field Type Required Location Description
version string Yes Request Body The connection version. This should be set to 1.0.
name string Yes Request Body The name of the Connection.
user string Yes Request Body The name of the user in the database connection.
db_type string Yes Request Body The type of database the connection is connecting to.
database string Yes Request Body The database the connection is connecting to.
host string Yes Request Body The hostname of the database connection.
port integer Yes Request Body The database port being connected through.
dbpassword string Yes, if password_encrypted is null. Request Body The password for the user connecting to the database.
password_encrypted string Yes, if dbpassword is null. Request Body The encrypted password for the user connecting to the database. Should only be specified if the encrypted_password was fetched from the GET endpoint for the connection.
schema string No Request Body The schema of the database to connect to.
service_name string No Request Body Only used for Oracle connections. The service name for the connection. (Optional)
oracle_wallet string No Request Body Only for Oracle connections The ID of the oracle-wallet attached with the connection, otherwise is null. (Optional)
connection_fileset string No Request Body Only for MySQL and MariaDB connections. The ID of the connection fileset attached to this connection. (Optional)
mask_type string No, defaults to database if not provided. Request Body The type of masking the connection can perform, only database or file are valid.
is_read_only boolean No, defaults to false if not provided. Request Body Whether or not the connection to the database is read-only.
iam_role_arn string No, role assumption will only take place if provided. Request Body The IAM role ARN for DataMasque to assume role as for S3 connections.
snowflake_private_key string No Request Body Only for Snowflake connections The identifier of an available Snowflake Private key. These can be uploaded via the Files page. If using key-pair authentication with Snowflake, dbpassword should be null.
snowflake_private_key_passphrase string No Request Body Only for Snowflake connections The passphrase for the private key (if needed).
snowflake_stage_location string No, defaults to aws_s3 if not provided. Request Body Only for Snowflake connections Storage location type for Snowflake stage. Valid options are aws_s3 or azure.
s3_bucket_name string Yes, when snowflake_stage_location is aws_s3. Request Body Only for Snowflake connections The name of the AWS S3 bucket used for intermediate data processing.
snowflake_azure_container_name string Yes, when snowflake_stage_location is azure. Request Body Only for Snowflake connections The name of the Azure Blob Storage Container used for intermediate data processing.
snowflake_azure_connection_string string Yes, when snowflake_stage_location is azure. Request Body Only for Snowflake connections The Connection String used to connect to the Azure Blob Storage Container.

File Connections

Field Type Required Location Description
version string Yes Request Body The connection version. This should be set to `1.0'.
name string Yes Request Body The name of the Connection.
type string Yes Request Body The type of file system the connection is connecting to. Valid options are "s3_connection", "azure_blob_connection" or "mounted_share_connection".
base_directory string Yes Request Body The root file path where files intended to be masked are stored.
bucket string Required only for S3 Connections. Request Body The name of the S3 bucket containing the base_directory.
container string Required only for Azure Blob Connections. Request Body The name of the Azure Blob Storage container containing the base_directory.
connection_string string Required only for Azure Blob Connections. Request Body The connection string configured with the authorization information to access data in your Azure Storage account.
mask_type string No, defaults to database if not provided. Request Body The type of masking the connection can perform, only database or file are valid.
is_file_mask_source boolean No, defaults to false if not provided. Request Body A boolean if the connection is a source Connection for file masking.
is_file_mask_destination boolean No, defaults to false if not provided. Request Body A boolean if the connection is a destination Connection for file masking.
iam_role_arn string No, role assumption will only take place if provided. Request Body Only for Amazon DynamoDB or Snowflake connections The IAM role ARN for DataMasque to assume role. For Snowflake, only used when snowflake_stage_location is aws_s3.

POST /api/connections/ Responses

Status Code Description
201 A JSON serialised Connection object.
403 Forbidden: If the token does not have the required permissions, or if the active license type does not permit creating connections.

POST /api/connections/ curl example

curl -X POST "https://<your-datamasque-host>/api/connections/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "version": "1.0",
           "name": "<connection_name>",
           "user": "<database_user>",
           "db_type": "<database_type>",
           "database": "<database_name>",
           "host": "<database_host>",
           "port": <database_port>,
           "password": "<database_password>",
           "schema": "<database_schema>",
           "mask_type": "database"
         }'

PUT /api/connections/{id}/

Authorization: User token only.

Update a connection with a specified id with new values.

PUT /api/connections/{id}/ Parameters

Database Connections

Field Type Required Location Description
version string Yes Request Body The connection version. This should be set to 1.0.
name string Yes Request Body The name of the Connection.
user string Yes Request Body The name of the user in the database connection.
db_type string Yes Request Body The type of database the connection is connecting to.
database string Yes Request Body The database the connection is connecting to.
host string Yes Request Body The hostname of the database connection.
port integer Yes Request Body The database port being connected through.
dbpassword string Yes, if password_encrypted is null. Request Body The password for the user connecting to the database.
password_encrypted string Yes, if dbpassword is null. Request Body The encrypted password for the user connecting to the database. Should only be specified if the encrypted_password was fetched from the GET endpoint for the connection.
schema string No Request Body The schema of the database to connect to.
service_name string No Request Body Only used for Oracle connections. The service name for the connection. (Optional)
oracle_wallet string No Request Body Only for Oracle connections The ID of the oracle-wallet attached with the connection, otherwise is null. (Optional)
connection_fileset string No Request Body Only for MySQL and MariaDB connections. The ID of the connection fileset attached to this connection. (Optional)
mask_type string No, defaults to database if not provided. Request Body The type of masking the connection can perform, only database or file are valid.
is_read_only boolean No, defaults to false if not provided. Request Body Whether or not the connection to the database is read-only.
iam_role_arn string No, role assumption will only take place if provided. Request Body The IAM role ARN for DataMasque to assume role as for S3 connections.
snowflake_private_key string No Request Body Only for Snowflake connections The identifier of an available Snowflake Private key. These can be uploaded via the Files page. If using key-pair authentication with Snowflake, dbpassword should be null.
snowflake_private_key_passphrase string No Request Body Only for Snowflake connections The passphrase for the private key (if needed).
snowflake_stage_location string No, defaults to aws_s3 if not provided. Request Body Only for Snowflake connections Storage location type for Snowflake stage. Valid options are aws_s3 or azure.
s3_bucket_name string Yes, when snowflake_stage_location is aws_s3. Request Body Only for Snowflake connections The name of the AWS S3 bucket used for intermediate data processing.
snowflake_azure_container_name string Yes, when snowflake_stage_location is azure. Request Body Only for Snowflake connections The name of the Azure Blob Storage Container used for intermediate data processing.
snowflake_azure_connection_string string Yes, when snowflake_stage_location is azure. Request Body Only for Snowflake connections The Connection String used to connect to the Azure Blob Storage Container.

File Connections

Field Type Required Location Description
version string Yes Request Body The connection version. This should be set to `1.0'.
name string Yes Request Body The name of the Connection.
type string Yes Request Body The type of file system the connection is connecting to. Valid options are "s3_connection", "azure_blob_connection" or "mounted_share_connection".
base_directory string Yes Request Body The root file path where files intended to be masked are stored.
bucket string Required only for S3 Connections. Request Body The name of the S3 bucket containing the base_directory.
container string Required only for Azure Blob Connections. Request Body The name of the Azure Blob Storage container containing the base_directory.
connection_string string Required only for Azure Blob Connections. Request Body The connection string configured with the authorization information to access data in your Azure Storage account.
mask_type string No, defaults to database if not provided. Request Body The type of masking the connection can perform, only database or file are valid.
is_file_mask_source boolean No, defaults to false if not provided. Request Body A boolean if the connection is a source Connection for file masking.
is_file_mask_destination boolean No, defaults to false if not provided. Request Body A boolean if the connection is a destination Connection for file masking.
iam_role_arn string No, role assumption will only take place if provided. Request Body The IAM role ARN for DataMasque to assume role as for S3 connections.

PUT /api/connections/{id}/ Responses

Status Code Description
200 A JSON serialised Connection object with the new updated values.

PUT /api/connections/{id}/ curl example

curl -X PUT "https://<your-datamasque-host>/api/connections/{connection_id}/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "version": "1.0",
           "name": "<connection_name>",
           "user": "<database_user>",
           "db_type": "<database_type>",
           "database": "<database_name>",
           "host": "<database_host>",
           "port": <database_port>,
           "password": "<database_password>",
           "schema": "<database_schema>",
           "mask_type": "database",
         }'

DELETE /api/connections/{id}/

Authorization: User token only.

Delete the connection with the specified id.

DELETE /api/connections/{id}/ Parameters

No parameters.

DELETE /api/connections/{id}/ Responses

Status Code Description
204 Operation succeeded.
403 Forbidden: If the token does not have the required permissions, or if the active license type does not permit deleting connections.

DELETE /api/connections/{id}/ curl example

curl -X DELETE "https://<your-datamasque-host>/api/connections/{id}/" \
     -H "Authorization: Token <your-api-token>"

POST /api/connections/{id}/duplicate/

Authorization: User token only.

Create a duplicate of an existing connection.

POST /api/connections/{id}/duplicate/ Parameters

Field Type Required Location Description
id string Yes URL Path The UUID of the connection to duplicate. Must be an existing connection in the system.

POST /api/connections/{id}/duplicate/ Responses

Status Code Description
201 Connection successfully duplicated. Returns the new Connection Object.
404 Connection with the specified ID not found.
403 Forbidden: If the token does not have the required permissions, or if the active license type does not permit creating connections.

POST /api/connections/{id}/duplicate/ Response Body

The response contains a complete Connection Object representing the newly created duplicate connection. The duplicated connection will have:

  • A new unique UUID (different from the source connection)
  • An automatically generated unique name with _copy suffix (e.g., original_connection_copy)
  • All configuration preserved from the source connection
  • No run history
  • Clean state ready for immediate use

POST /api/connections/{id}/duplicate/ curl example

curl -X POST "https://<your-datamasque-host>/api/connections/{connection-id}/duplicate/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json"

POST /api/connections/test/

Authorization: User token only.

Test a connection to validate that it is able to successfully connect to the target database.

Recommended: For testing existing connections, use POST /api/connections/{id}/test/ instead.

POST /api/connections/test/ Parameters

Database Connections

Field Type Required Location Description
version string Yes Request Body The connection version. This should be set to 1.0.
name string No Request Body The name of the Connection.
user string Yes Request Body The name of the user in the database connection.
db_type string Yes Request Body The type of database the connection is connecting to.
database string Yes Request Body The database the connection is connecting to.
host string Yes Request Body The hostname of the database connection.
port integer Yes Request Body The database port being connected through.
dbpassword string Yes, if password_encrypted is null. Request Body The password for the user connecting to the database.
password_encrypted string Yes, if dbpassword is null. Request Body The encrypted password for the user connecting to the database. Should only be specified if the encrypted_password was fetched from the GET endpoint for the connection.
schema string No Request Body The schema of the database to connect to.
service_name string No Request Body Only used for Oracle connections. The service name for the connection. (Optional)
oracle_wallet string No Request Body Only for Oracle connections The ID of the oracle-wallet attached with the connection, otherwise is null. (Optional)
connection_fileset string No Request Body Only for MySQL and MariaDB connections. The ID of the connection fileset attached to this connection. (Optional)
mask_type string No, defaults to database if not provided. Request Body The type of masking the connection can perform, only database or file are valid.
is_read_only boolean No, defaults to false if not provided. Request Body Whether or not the connection to the database is read-only.
iam_role_arn string No, role assumption will only take place if provided. Request Body The IAM role ARN for DataMasque to assume role as for S3 connections.
snowflake_private_key string No Request Body Only for Snowflake connections The identifier of an available Snowflake Private key. These can be uploaded via the Files page. If using key-pair authentication with Snowflake, dbpassword should be null.
snowflake_private_key_passphrase string No Request Body Only for Snowflake connections The passphrase for the private key (if needed).
snowflake_stage_location string No, defaults to aws_s3 if not provided. Request Body Only for Snowflake connections Storage location type for Snowflake stage. Valid options are aws_s3 or azure.
s3_bucket_name string Yes, when snowflake_stage_location is aws_s3. Request Body Only for Snowflake connections The name of the AWS S3 bucket used for intermediate data processing.
snowflake_azure_container_name string Yes, when snowflake_stage_location is azure. Request Body Only for Snowflake connections The name of the Azure Blob Storage Container used for intermediate data processing.
snowflake_azure_connection_string string Yes, when snowflake_stage_location is azure. Request Body Only for Snowflake connections The Connection String used to connect to the Azure Blob Storage Container.

File Connections

Field Type Required Location Description
version string Yes Request Body The connection version. This should be set to `1.0'.
name string Yes Request Body The name of the Connection.
type string Yes Request Body The type of file system the connection is connecting to. Valid options are "s3_connection", "azure_blob_connection" or "mounted_share_connection".
base_directory string Yes Request Body The root file path where files intended to be masked are stored.
bucket string Required only for S3 Connections. Request Body The name of the S3 bucket containing the base_directory.
container string Required only for Azure Blob Connections. Request Body The name of the Azure Blob Storage container containing the base_directory.
connection_string string Required only for Azure Blob Connections. Request Body The connection string configured with the authorization information to access data in your Azure Storage account.
mask_type string No, defaults to database if not provided. Request Body The type of masking the connection can perform, only database or file are valid.
is_file_mask_source boolean No, defaults to false if not provided. Request Body A boolean if the connection is a source Connection for file masking.
is_file_mask_destination boolean No, defaults to false if not provided. Request Body A boolean if the connection is a destination Connection for file masking.
iam_role_arn string No, role assumption will only take place if provided. Request Body The IAM role ARN for DataMasque to assume role as for S3 connections.

POST /api/connections/test/ Responses

Status Code Description
200 Operation succeeded

POST /api/connections/test/ curl example

curl -X POST "https://<your-datamasque-host>/api/connections/test/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "name": "<your-connection-name>",
           "user": "<your-connection-user>",
           "db_type": "oracle",
           "database": "<your-database>",
           "host": "<your-host>",
           "port": 1433,
           "dbpassword": "<your-password>",
           "schema": "<optional-schema>",
           "service_name": "<optional-service-name>",
           "connection_fileset": "<optional-connection-fileset>",
           "snowflake_private_key": "<optional-snowflake-private-key>",
           "snowflake_private_key_passphrase": "<optional-snowflake-private-key-passphrase>"
           "version": "1.0"
         }'

POST /api/connections/{id}/test/

Authorization: User token only.

Test an existing connection with optional field overrides. This endpoint retrieves the saved connection configuration and merges it with any provided overrides, then attempts to establish the connection.

POST /api/connections/{id}/test/ Parameters

Field Type Required Location Description
id string Yes URL Path The id of the existing Connection to test.

Request Body: Any connection configuration fields you want to override. All fields are optional since they will be merged with the existing connection configuration.

Common override fields include:

  • host, port, user, dbpassword - Test with different connection details
  • database, schema - Test against different database/schema

POST /api/connections/{id}/test/ Responses

Status Code Description
200 Operation succeeded.
400 Bad Request: Invalid connection configuration or connection test failed.
404 Not Found: Connection with specified ID does not exist.

POST /api/connections/{id}/test/ curl example

# Test existing connection with password override
curl -X POST "https://<your-datamasque-host>/api/connections/{connection-id}/test/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "dbpassword": "<new-password>"
         }'

# Test existing connection with host and port override
curl -X POST "https://<your-datamasque-host>/api/connections/{connection-id}/test/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "host": "<new-host>",
           "port": 5432
         }'

Ruleset Object

Ruleset objects have the following fields:

Field Type Description
id integer The id of the Ruleset. Use this in API URLs that need a ruleset id.
name string The name of the Ruleset.
config_yaml string The contents of the Ruleset, including of all the masking rules.
ruleset_validation_status string Whether or not the Ruleset is valid, and can be used for masking runs. See below.
validation_error_type string If the ruleset is invalid, the category of the validation failure. See below. Null if valid.
validation_error string If the ruleset is invalid, a human-readable description of the validation error. Null if valid.
mask_type string The masking type of the Ruleset. This can be "database" or "file".

Possible ruleset_validation_status values are:

  • valid: The ruleset is valid, and can be used in a masking run.
  • invalid: The ruleset is invalid because it contains YAML syntax errors and/or it does not conform to the DataMasque ruleset specification. The ruleset cannot be used in a masking run until the errors have been corrected.
  • in_progress: The ruleset is currently in the process of being validated. The ruleset will continue to be validated when used in a masking run.
  • unknown: Default status when no other status is applicable. The ruleset will be validated when used in a masking run.

Possible validation_error_type values are:

  • ruleset: Ruleset has invalid YAML or does not meet the Ruleset Specification.
  • library_missing: A referenced library does not exist.
  • library_invalid: A referenced library failed validation.
  • expansion: The expanded ruleset (with library references resolved) does not meet the Ruleset Specification.

The validation_error field uses a prefix based on the validation_error_type: "Ruleset error: ...", "Library missing: ...", "Library invalid: ...", or "Expansion error: ...".

Requests related to Ruleset Object:

GET /api/rulesets/

Authorization: User token only.

Returns a list of all rulesets.

GET /api/rulesets/ Parameters

No parameters.

GET /api/rulesets/ Responses

Status Code Description
200 A paginated JSON list of Ruleset objects. List view excludes config_yaml and errors for performance.

GET /api/rulesets/ curl example

curl "https://<your-datamasque-host>/api/rulesets/" \
     -H "Authorization: Token <your-api-token>"

GET /api/rulesets/{id}/

GET /api/rulesets/{id}/ Parameters

No parameters.

GET /api/rulesets/ Responses

Status Code Description
200 A JSON serialised Ruleset object. 
curl "https://<your-datamasque-host>/api/rulesets/{id}/" \
     -H "Authorization: Token <your-api-token>"

POST /api/rulesets/

Authorization: User token only.

Creates a new ruleset.

POST /api/rulesets/ Parameters

Field Type Required Location Description
name string Yes Request Body The name of the Ruleset.
config_yaml string Yes Request Body The YAML contents of the Ruleset.
mask_type string No Request Body The masking type of the Ruleset. Valid options are "database" or "file". Defaults to "database".
upsert string No Query Param If present, creates a new ruleset or updates an existing one with matching name and mask_type.

POST /api/rulesets/ Responses

Status Code Description
201 A JSON serialised Ruleset object. Returned when a new ruleset is created.
200 A JSON serialised Ruleset object. Returned when an existing ruleset is updated via upsert.

POST /api/rulesets/ curl example

# Create a new ruleset
curl -X POST "https://<your-datamasque-host>/api/rulesets/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "name": "<your-new-name>",
           "config_yaml": "version: \"1.0\"\ntasks:\n  - type: run_data_discovery"
         }'

# Create new or update existing ruleset with upsert
curl -X POST "https://<your-datamasque-host>/api/rulesets/?upsert=true" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "name": "<your-ruleset-name>",
           "config_yaml": "version: \"1.0\"\ntasks:\n  - type: run_data_discovery"
         }'

PUT /api/rulesets/{id}/

Authorization: User token only.

Update an existing ruleset.

PUT /api/rulesets/{id}/ Parameters

Field Type Required Location Description
name string Yes Request Body The name of the Ruleset.
config_yaml string Yes Request Body The YAML contents of the Ruleset.
mask_type string No Request Body The masking type of the Ruleset. Valid options are "database" or "file".

PUT /api/rulesets/{id}/ Responses

Status Code Description
200 A JSON serialised Ruleset object with the updated values.

PUT /api/rulesets/{id}/ curl example

curl -X PUT "https://<your-datamasque-host>/api/rulesets/{id}/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "name": "<your-new-name>",
           "config_yaml": "version: \"1.0\"\ntasks:\n  - type: run_data_discovery"
         }'

DELETE /api/rulesets/{id}/

Authorization: User token only.

Deletes the ruleset with the specified id.

DELETE /api/rulesets/{id}/ Parameters

No parameters.

DELETE /api/rulesets/{id}/ Responses

Status Code Description
204 Operation succeeded

DELETE /api/rulesets/{id}/ curl example

curl -X DELETE "https://<your-datamasque-host>/api/rulesets/{id}/" \
     -H "Authorization: Token <your-api-token>"

DELETE /api/rulesets/

Authorization: User token only.

Bulk delete or archive multiple rulesets. Rulesets that have associated run history will be archived (soft deleted) instead of permanently deleted, to preserve the run history.

DELETE /api/rulesets/ Parameters

Field Type Required Location Description
ids array Yes Request Body A list of ruleset UUIDs to delete.
mode string No Request Body "delete" (default) for hard delete, or "archive" for soft delete (reversible).
confirm boolean No Request Body Must be true for hard delete mode. Ignored for archive mode.

DELETE /api/rulesets/ Responses

Status Code Description
200 Operation succeeded. Response includes counts and IDs of deleted/archived rulesets.
400 Validation error or confirmation not provided for hard delete mode.
404 One or more ruleset IDs were not found.

Success response (mode "delete"):

{
  "deleted_count": 2,
  "deleted_ids": ["9d641e97-adf7-4f22-9089-afc3711bf222", "04ea20f0-ad4c-498e-881f-b0bc79d83ba7"],
  "archived_count": 1,
  "archived_ids": ["c0f5b5bb-a2ce-4cea-9248-1b8ef6539a0e"]
}

Note: In "delete" mode, rulesets with run history are automatically archived instead of deleted. The response separates the two so you can see which rulesets were permanently deleted and which were archived.

Success response (mode "archive"):

{
  "archived_count": 3,
  "archived_ids": ["9d641e97-adf7-4f22-9089-afc3711bf222", "04ea20f0-ad4c-498e-881f-b0bc79d83ba7", "c0f5b5bb-a2ce-4cea-9248-1b8ef6539a0e"]
}

DELETE /api/rulesets/ curl examples

Delete rulesets (with confirmation):

curl -X DELETE "https://<your-datamasque-host>/api/rulesets/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "ids": ["9d641e97-adf7-4f22-9089-afc3711bf222", "04ea20f0-ad4c-498e-881f-b0bc79d83ba7"],
           "mode": "delete",
           "confirm": true
         }'

Archive rulesets (soft delete):

curl -X DELETE "https://<your-datamasque-host>/api/rulesets/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "ids": ["9d641e97-adf7-4f22-9089-afc3711bf222"],
           "mode": "archive"
         }'

GET /api/v2/rulesets/

Authorization: User token only.

Performance-optimized endpoint. Excludes config_yaml by default for faster response times.

GET /api/v2/rulesets/ Parameters

Field Type Required Location Description
mask_type string No Query Parameter Filter by "database" or "file".
search string No Query Parameter Search ruleset name and content.
name string No Query Parameter Filter by name (substring match).
name_exact string No Query Parameter Filter by exact name match.
name_startswith string No Query Parameter Filter by name prefix.
name_regex string No Query Parameter Filter by name using regex pattern.
content string No Query Parameter Search within ruleset YAML content.
content_regex string No Query Parameter Filter by content using regex pattern.
validation_status string No Query Parameter Filter by validation status: "valid", "invalid", "in_progress", or "unknown".
created__gte string No Query Parameter Filter created on or after date. Format: YYYY-MM-DD.
created__lte string No Query Parameter Filter created on or before date. Format: YYYY-MM-DD.
created__year integer No Query Parameter Filter created in specified year.
modified__gte string No Query Parameter Filter modified on or after date. Format: YYYY-MM-DD.
modified__lte string No Query Parameter Filter modified on or before date. Format: YYYY-MM-DD.
ordering string No Query Parameter Sort by name, created, or modified. Prefix - for descending. Default: name.
include_yaml boolean No Query Parameter Include config_yaml field. Default: false.

GET /api/v2/rulesets/ Responses

Status Code Description
200 List of Ruleset objects.

GET /api/v2/rulesets/ curl example

curl "https://<your-datamasque-host>/api/v2/rulesets/?search=customer&mask_type=database" \
     -H "Authorization: Token <your-api-token>"

Ruleset Library Object

Ruleset Library objects contain reusable masking rules that can be imported into rulesets using the imports directive.

Field Type Description
id string The uuid of the Ruleset Library. Use this in API URLs that need a ruleset library id.
name string The name of the Ruleset Library. Must be unique among non-archived libraries and cannot contain /. Max 64 characters.
config_yaml string The YAML contents of the Ruleset Library, containing reusable masking rules.
is_valid string The validation status of the library. See below for possible values.
validation_error string If the library is invalid, a human-readable description of the validation error. Null if valid.
created datetime The date and time the library was created.
modified datetime The date and time the library was last modified.
errors object Validation errors if the library is invalid. Only present when is_valid is "invalid".
usage_count integer The number of non-archived rulesets that import this library.

Possible is_valid values are:

  • valid: The library is valid and can be imported into rulesets.
  • invalid: The library contains YAML syntax errors or does not conform to the library specification.
  • in_progress: The library is currently being validated.
  • unknown: Default status when no other status is applicable.

Permissions:

  • View (list, retrieve): Admin users, staff users, or mask builder users.
  • Manage (create, update, delete): Admin users or users who are members of BOTH the mask builder AND ruleset library manager groups.

Requests related to Ruleset Library Object:

GET /api/ruleset-libraries/

Authorization: User token only.

Returns a paginated list of ruleset libraries. By default, archived libraries are excluded.

GET /api/ruleset-libraries/ Parameters

Field Type Required Location Description
limit integer No Query Parameter Maximum number of results to return. Default: system default.
offset integer No Query Parameter Number of results to skip for pagination.
search string No Query Parameter Search by library name.
name string No Query Parameter Filter by name (case-insensitive substring match).
name_exact string No Query Parameter Filter by exact name match.
name_startswith string No Query Parameter Filter by name prefix (case-insensitive).
name_regex string No Query Parameter Filter by name using regex pattern.
namespace string No Query Parameter Filter by namespace (case-insensitive substring match).
namespace_exact string No Query Parameter Filter by exact namespace match.
content string No Query Parameter Search within library YAML content.
content_regex string No Query Parameter Filter by content using regex pattern.
validation_status string No Query Parameter Filter by validation status: "valid", "invalid", "in_progress", or "unknown".
archived boolean No Query Parameter Filter by archived status. Default: false (only non-archived libraries).
created__gte string No Query Parameter Filter created on or after date. Format: YYYY-MM-DD.
created__lte string No Query Parameter Filter created on or before date. Format: YYYY-MM-DD.
modified__gte string No Query Parameter Filter modified on or after date. Format: YYYY-MM-DD.
modified__lte string No Query Parameter Filter modified on or before date. Format: YYYY-MM-DD.
ordering string No Query Parameter Sort by name, created, or modified. Prefix - for descending. Default: name.

GET /api/ruleset-libraries/ Responses

Status Code Description
200 A paginated JSON list of Ruleset Library objects. List view excludes config_yaml for performance.

GET /api/ruleset-libraries/ curl example

curl "https://<your-datamasque-host>/api/ruleset-libraries/?search=pii&ordering=-modified" \
     -H "Authorization: Token <your-api-token>"

POST /api/ruleset-libraries/

Authorization: User token only. Requires admin or ruleset library manager permissions.

Creates a new ruleset library.

POST /api/ruleset-libraries/ Parameters

Field Type Required Location Description
name string Yes Request Body The name of the library. Must be unique, max 64 characters, no /.
config_yaml string Yes Request Body The YAML contents of the library.

POST /api/ruleset-libraries/ Responses

Status Code Description
201 A JSON serialised Ruleset Library object.
400 Validation error. The response body contains details about the invalid fields.

POST /api/ruleset-libraries/ curl example

curl -X POST "https://<your-datamasque-host>/api/ruleset-libraries/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "name": "pii-masks",
           "config_yaml": "version: \"1.0\"\nrules:\n  mask_email:\n    masks:\n      - type: from_fixed"
         }'

GET /api/ruleset-libraries/{id}/

Authorization: User token only.

Returns the details of a specific ruleset library, including the full config_yaml content.

GET /api/ruleset-libraries/{id}/ Parameters

Field Type Required Location Description
id string Yes URL Path The UUID of the ruleset library.

GET /api/ruleset-libraries/{id}/ Responses

Status Code Description
200 A JSON serialised Ruleset Library object.
404 Library not found.

GET /api/ruleset-libraries/{id}/ curl example

curl "https://<your-datamasque-host>/api/ruleset-libraries/<library-uuid>/" \
     -H "Authorization: Token <your-api-token>"

PUT /api/ruleset-libraries/{id}/

Authorization: User token only. Requires admin or ruleset library manager permissions.

Updates an existing ruleset library.

PUT /api/ruleset-libraries/{id}/ Parameters

Field Type Required Location Description
id string Yes URL Path The UUID of the ruleset library.
name string Yes Request Body The name of the library. Must be unique, max 64 characters, no /.
config_yaml string Yes Request Body The YAML contents of the library.

PUT /api/ruleset-libraries/{id}/ Responses

Status Code Description
200 A JSON serialised Ruleset Library object.
400 Validation error. The response body contains details about the invalid fields.
404 Library not found.

PUT /api/ruleset-libraries/{id}/ curl example

curl -X PUT "https://<your-datamasque-host>/api/ruleset-libraries/<library-uuid>/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "name": "pii-masks-v2",
           "config_yaml": "version: \"1.0\"\nrules:\n  mask_email:\n    masks:\n      - type: from_fixed"
         }'

DELETE /api/ruleset-libraries/{id}/

Authorization: User token only. Requires admin or ruleset library manager permissions.

Soft-deletes (archives) a ruleset library. Archived libraries are excluded from list views by default but can be viewed using the archived=true filter.

DELETE /api/ruleset-libraries/{id}/ Parameters

Field Type Required Location Description
id string Yes URL Path The UUID of the ruleset library.

DELETE /api/ruleset-libraries/{id}/ Responses

Status Code Description
204 Library successfully archived.
404 Library not found.

DELETE /api/ruleset-libraries/{id}/ curl example

curl -X DELETE "https://<your-datamasque-host>/api/ruleset-libraries/<library-uuid>/" \
     -H "Authorization: Token <your-api-token>"

GET /api/ruleset-libraries/{id}/usage/

Authorization: User token only. Requires admin, staff, or mask builder permissions.

Returns a list of UUIDs of non-archived rulesets that import the specified library.

GET /api/ruleset-libraries/{id}/usage/ Parameters

Field Type Required Location Description
id string Yes URL Path The UUID of the ruleset library.

GET /api/ruleset-libraries/{id}/usage/ Responses

Status Code Description
200 A JSON list of ruleset UUID strings. Returns an empty list if no rulesets use the library.
404 Library not found or archived.

GET /api/ruleset-libraries/{id}/usage/ curl example

curl "https://<your-datamasque-host>/api/ruleset-libraries/<library-uuid>/usage/" \
     -H "Authorization: Token <your-api-token>"

GET /api/ruleset-libraries/{id}/rulesets/

Authorization: User token only. Requires admin, staff, or mask builder permissions.

Returns a list of non-archived rulesets that import the specified library. Each returned ruleset uses the lightweight list representation (excludes config_yaml).

GET /api/ruleset-libraries/{id}/rulesets/ Parameters

Field Type Required Location Description
id string Yes URL Path The UUID of the ruleset library.

GET /api/ruleset-libraries/{id}/rulesets/ Responses

Status Code Description
200 A paginated JSON list of Ruleset objects. List view excludes config_yaml and errors for performance.
404 Library not found.

GET /api/ruleset-libraries/{id}/rulesets/ curl example

curl "https://<your-datamasque-host>/api/ruleset-libraries/<library-uuid>/rulesets/" \
     -H "Authorization: Token <your-api-token>"

Files

Oracle Wallets

Oracle Wallet objects have the following fields:

Field Type Description
id string The uuid of the Oracle Wallet. Use this in API URLs that need a oracle_wallet id.
name string The name of the Oracle Wallet.
created_date datetime The date the Oracle Wallet was uploaded.

Note The /api/oracle-wallets/ endpoint is being deprecated and will be removed in a future release. Use the /api/files/oracle-wallets/ endpoint instead.

GET /api/files/oracle-wallets/

GET /api/oracle-wallets/

Authorization: User token only.

Returns a list of Oracle wallets. These are used to connect to encrypted Oracle connections.

GET /api/files/oracle-wallets/ Parameters

No parameters.

GET /api/files/oracle-wallets/ Responses

Status Code Description
200 A JSON serialised list of Oracle Wallets.

GET /api/files/oracle-wallets/ curl example

curl "https://<your-datamasque-host>/api/oracle-wallets/" \
     -H "Authorization: Token <your-api-token>"

POST /api/files/oracle-wallets/

POST /api/oracle-wallets/

Authorization: User token only.

Create a new Oracle wallet.

POST /api/files/oracle-wallets/ Parameters

Field Type Required Location Description
name string Yes Form Field The name of the Oracle Wallet.
zip_archive file Yes Form Field The Zip archive file.

POST /api/files/oracle-wallets/ Responses

Status Code Description
201 A JSON serialised Oracle Wallet object of the wallet created.

POST /api/files/oracle-wallets/ curl example

curl -X POST "https://<your-datamasque-host>/api/files/oracle-wallets/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: multipart/form-data" \
     -F "name=<wallet_name>" \
     -F "zip_archive=@</path/to/your/file.zip>"

DELETE /api/files/oracle-wallets/{id}/

DELETE /api/oracle-wallets/{id}/

Authorization: User token only.

Delete the Oracle wallet with the specified id.

DELETE /api/files/oracle-wallets/{id}/ Parameters

No parameters.

DELETE /api/files/oracle-wallets/{id}/ Responses

Status Code Description
204 Operation succeeded.

DELETE /api/files/oracle-wallets/{id}/ curl example

curl -X DELETE "https://<your-datamasque-host>/api/files/oracle-wallets/{id}/" \
     -H "Authorization: Token <your-api-token>"

DELETE /api/oracle-wallets/{id}/ Responses

Status Code Description
204 Operation succeeded.

DELETE /api/oracle-wallets/{id}/ curl example

curl -X DELETE "https://<your-datamasque-host>/api/oracle-wallets/{id}/" \
     -H "Authorization: Token <your-api-token>"

GET /api/files/oracle-wallets/{id}/download/

Authorization: Admin user only.

Download the Oracle wallet zip archive with the specified id.

GET /api/files/oracle-wallets/{id}/download/ Parameters

No parameters.

GET /api/files/oracle-wallets/{id}/download/ Responses

Status Code Description
200 The server will return the Oracle wallet zip archive in the response body.
403 Forbidden. Only administrators can download Oracle wallets.

GET /api/files/oracle-wallets/{id}/download/ curl example

curl -o wallet.zip "https://<your-datamasque-host>/api/files/oracle-wallets/{id}/download/" \
     -H "Authorization: Token <your-api-token>"

Connection Filesets

Note The /api/connection-filesets/ endpoint is being deprecated and will be removed in a future release. Use the /api/files/connection-filesets/ endpoint instead.

Connection Fileset objects have the following fields:

Field Type Description
id string The uuid of the Connection Fileset. Use this in API URLs that need a connection_fileset id.
name string The name of the Connection Fileset.
database_type string The type of database the Connection Fileset is associated with (currently only mysql is supported; this will work with both MySQL and MariaDB connections).
created_date datetime The date the Connection Fileset was uploaded.

GET /api/files/connection-filesets/

GET /api/connection-filesets/

Authorization: User token only.

Returns a list of Connection Filesets. These may be used to encrypt connections to MySQL and MariaDB databases.

GET /api/files/connection-filesets/ Parameters

No parameters.

GET /api/files/connection-filesets/ Responses

Status Code Description
200 A list of JSON serialised Connection Filesets.

GET /api/files/connection-filesets/ curl example

curl "https://<your-datamasque-host>/api/files/connection-filesets/" \
     -H "Authorization: Token <your-api-token>"

POST /api/files/connection-filesets/

POST /api/connection-filesets/

Authorization: User token only.

Create a new Connection Fileset.

POST /api/files/connection-filesets/ Parameters

Field Type Required Location Description
name string Yes Form Field The name of the Connection Fileset.
database_type string Yes Form Field The type of database the Connection Fileset is associated with (currently only mysql is supported; this will work with both MySQL and MariaDB connections).
zip_archive file Yes Form Field The Zip archive file.

POST/api/files/connection-filesets/ Responses

Status Code Description
201 A JSON serialised object of the Connection Fileset that was created.

POST /api/files/connection-filesets/ curl example

curl -X POST "https://<your-datamasque-host>/api/files//connection-filesets/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: multipart/form-data" \
     -F "name=<fileset_name>" \
     -F "database_type=<database_type>" \
     -F "zip_archive=@</path/to/your/file.zip>"

DELETE /api/files/connection-filesets/{id}/

DELETE /api/connection-filesets/{id}/

Authorization: User token only.

Deletes the Connection Fileset with the specified id. You may not delete a Connection Fileset associated to an existing connection.

DELETE /api/files/connection-filesets/{id}/ Parameters

No parameters.

DELETE /api/files/connection-filesets/{id}/ Responses

Status Code Description
204 Operation succeeded.

DELETE /api/files/connection-filesets/{id}/ curl example

curl -X DELETE "https://<your-datamasque-host>/api/files/connection-filesets/{id}/" \
     -H "Authorization: Token <your-api-token>"

GET /api/files/connection-filesets/{id}/download/

Authorization: Admin user only.

Download the Connection Fileset zip archive with the specified id.

GET /api/files/connection-filesets/{id}/download/ Parameters

No parameters.

GET /api/files/connection-filesets/{id}/download/ Responses

Status Code Description
200 The server will return the Connection Fileset zip archive in the response body.
403 Forbidden. Only administrators can download Connection Filesets.

GET /api/files/connection-filesets/{id}/download/ curl example

curl -o fileset.zip "https://<your-datamasque-host>/api/files/connection-filesets/{id}/download/" \
     -H "Authorization: Token <your-api-token>"

Snowflake Keys

These are used for key-pair authentication with Snowflake connections.

Snowflake Key Object

Field Type Description
id string The uuid of the Snowflake key.
name string The name of the Snowflake key.
created_date datetime The date that the Snowflake key was created.
modified_date datetime The date that the Snowflake key was modified.

GET /api/files/snowflake-keys/

Authorization: User token only.

Returns a list of Snowflake keys.

GET /api/files/snowflake-keys/ Parameters

No parameters.

GET /api/files/snowflake-keys/ Responses

Status Code Description
201 A JSON serialised list of Snowflake keys.

GET /api/files/snowflake-keys/ curl example

curl "https://<your-datamasque-host>/api/files/snowflake-keys/" \
     -H "Authorization: Token <your-api-token>"

GET /api/files/snowflake-keys/{id}/

Authorization: User token only.

Get a Snowflake key by id.

GET /api/files/snowflake-keys/{id}/ Parameters

No parameters.

GET /api/files/snowflake-keys/{id}/ Responses

Status Code Description
201 A JSON serialised Snowflake Key.

GET /api/files/snowflake-keys/ curl example

curl "https://<your-datamasque-host>/api/files/snowflake-keys/{id}/" \
     -H "Authorization: Token <your-api-token>"

POST /api/files/snowflake-keys/

Authorization: User token only.

Upload a new Snowflake key.

POST /api/files/snowflake-keys/ Parameters

Field Type Required Location Description
name string Yes Form Field The name of the Snowflake key.
key_file file Yes Form Field The Snowflake private key file.

POST /api/files/snowflake-keys/ Responses

Status Code Description
201 A JSON serialised Snowflake key object.

POST /api/files/snowflake-keys/ curl example

curl -X POST "https://<your-datamasque-host>/api/files/snowflake-keys/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: multipart/form-data" \
     -F "name=<fileset_name>" \
     -F "key_file=@</path/to/your/private.key>"

DELETE /api/files/snowflake-keys/{id}/

Authorization: User token only.

Delete the Snowflake key with the specified id. You may not delete a Snowflake Key if it is associated to a connection.

DELETE /api/files/snowflake-keys/{id}/ Parameters

No parameters.

DELETE /api/files/snowflake-keys/{id}/ Responses

Status Code Description
204 Operation succeeded.

DELETE /api/files/snowflake-keys/{id}/ curl example

curl -X DELETE "https://<your-datamasque-host>/api/files/snowflake-keys/{id}/" \
     -H "Authorization: Token <your-api-token>"

Seed Object

Field Type Description
id integer The id of the Seed.
name string The name of the Seed.
seed_file string The location of the Seed.
created date datetime The date that the Seed was uploaded.
filename string The file name of the uploaded Seed.

GET /api/seeds/

Authorization: User token only.

Get a list of all DataMasque seed files.

Optionally, you can add an {id} to the end of the request to only return the details of the seed with that specific id.

GET /api/seeds/ Parameters

No parameters.

GET /api/seeds/ Responses

Status Code Description
200 A JSON serialised list of Seed objects.

GET /api/seeds/ curl example

curl "https://<your-datamasque-host>/api/seeds/" \
     -H "Authorization: Token <your-api-token>"

POST /api/seeds/

Authorization: User token only.

Create a new seed from a csv file.

POST /api/seeds/ Parameters

Field Type Required Description
name string No The name of the csv file.
description string No A description of the seed file to displayed on the files menu.
seed_file file No The seed file.

POST /api/seeds/ Responses

Status Code Description
201 A JSON serialised Seed object.
400 Invalid CSV data, e.g. column mismatch, non-UTF8 encoding, etc.

POST /api/seeds/ curl example

curl -X POST "https://<your-datamasque-host>/api/seeds/" \
     -H "Authorization: Token <your-api-token>" \
     -F "name=<fileset_name>" \
     -F "seed_file=@</path/to/your/seed_file.csv>"

Audit Log Object

Field Type Description
id integer The id of the audit log.
timestamp datetime The timestamp of when the audit log was created.
username string The username which created the audit log.
category string The category for the audit log, one of the following: auth, run, ruleset, or connection
action string The action taken. One of the following: logged_in logged_out, for auth actions, started, cancelled, for masking run actions, created, modified, deleted for connection or ruleset actions.
description string A short description of what happened during the action.

Audit Log CSV

A CSV representation of the Audit Log Object

The CSV file contains the following headers:

Field Type Description
timestamp datetime The timestamp of when the audit log was created.
username string The username which created the audit log.
category string The category for the audit log, one of the following: auth, run, ruleset, or connection
action string The action taken. One of the following: logged_in logged_out, for auth actions, started, cancelled, for masking run actions, created, modified, deleted for connection or ruleset actions.
description string A short description of what happened during the action.

GET /api/audit-logs/

Authorization: User token only.

Retrieve all Audit Logs.

GET /api/audit-logs/ Parameters

No parameters.

GET /api/audit-logs/ Response

Status Code Description
200 A list of JSON serialised list of Audit Log objects

GET /api/audit-logs/ curl example

curl "https://<your-datamasque-host>/api/audit-logs/" \
     -H "Authorization: Token <your-api-token>"

GET /api/audit-logs/download/

Authorization: User token only.

Retrieve all Audit Logs.

GET /api/audit-logs/download/ Parameters

No parameters.

GET /api/audit-logs/download/ Response

Status Code Description
200 The server will return the audit logs in the response body which can be then downloaded as a CSV file.

GET /api/audit-logs/download/ curl example

curl -o <your-downloads-path>/<your-download-name>.csv -X GET "https://<your-datamasque-host>/api/audit-logs/" \
     -H "Authorization: Token <your-api-token>"

Schema Discovery

POST /api/schema-discovery/

Authorization: User token only.

Executes schema discovery against a database connection.

POST /api/schema-discovery/ Parameters

Field Type Required Description
connection string Yes The id of the Connection.
custom_keywords array[string] Yes List of keywords that, where a column name matches one or more of the keywords, indicates the column contains sensitive data.
disable_built_in_keywords boolean Yes If set to true, then DataMasque's built-in list of keywords will not be used to flag columns as sensitive.
disable_global_custom_keywords boolean Yes If set to true, then the user-defined global set of custom keywords will not be used to flag columns as sensitive.
disable_global_ignored_keywords boolean Yes If set to true, then the user-defined global set of ignored keywords will not be used to exclude columns from the discovery results.
ignored_keywords array[string] Yes List of keywords that, where a column name matches one or more of the keywords, indicates the column should be excluded from the schema discovery results.
in_data_discovery object No In-data discovery options. An object containing enabled, row_sample_size, custom_rules, non_sensitive_rules and force options. Defaults to {enabled: false}.
schemas array[string] Yes List of schema names (or database for MySQL/MariaDB) against which to perform schema discovery. Send an empty list to run against the schema configured on the database connection, or the database user's default schema if one is not specified for the connection.

POST /api/schema-discovery/ Responses

Status Code Description
201 A JSON serialised Run object.

POST /api/schema-discovery/ Example

curl -X POST "https://<your-datamasque-host>/api/schema-discovery/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "connection": "<your-connection-id>",
           "custom_keywords": [],
           "ignored_keywords": [],
           "disable_global_custom_keywords": false,
           "disable_global_ignored_keywords": false,
           "disable_built_in_keywords": false,
           "in_data_discovery": {
             "enabled": true,
             "row_sample_size": 500,
             "custom_rules": [
               {
                 "name": "temp_staff",
                 "pattern": "temp.*"
               }
             ],
             "non_sensitive_rules": [
               {"pattern": "retired.*"}
             ],
           }
         }'

GET /api/schema-discovery/{connection_id}/

Authorization: User token or API token.

Retrieve schema discovery results.

GET /api/schema-discovery/{connection_id}/ Parameters

None

GET /api/schema-discovery/{connection_id}/ Response

Status Code Description
200 A JSON serialised object containing a Schema Discovery object and a Run object.
Field Type Description
data object A Schema Discovery.
last_sdd_run object A JSON serialised Run object.

Schema Discovery Object

Schema Discovery objects have the following fields:

Field Type Description
options object List of ignored_keywords and customised_keywords.
schemas list[object] List of schema objects each with name and list of tables. tables contain name and a list of columns.
sd_version string Schema discovery version e.g. "1.1.1".

Schema Discovery Column Object

Column objects have the following fields:

Field Type Description
name string The column name
data_type string The data type for this field e.g varchar, integer, numeric, timestamp without time zone.
categories list[string] A list of classifications for the flagged sensitive data: PII, PHI, PCI and/or Custom.
max_length number The column length
description string The reason the column was flagged as sensitive.
foreign_keys list[object] A list of foreign key objects containing name and referenced_column.
is_unique_key boolean Is the column a unique key.
numeric_scale number If the data_type is numeric this refers to the maximum number of decimal places.
ruleset_match boolean The type of information detected by sensitive data discovery, used internally by the the ruleset generator to suggest a suitable masking rule.
in_data_result list[object] A list of In Data matches.
is_primary_key boolean Is the column a primary key.
numeric_precision number If the data_type is numeric this refers to the maximum number of digits present.
constraint_columns list[string] A list of column names participating in the constraint.
pk_constraint_name string The name of the primary key constraint.
uk_constraint_name string The name of the unique key constraint.
unique_index_names list[string] A list of index names for this column.
allow_in_data_override boolean A boolean representing that a Sensitive Data match can be overridden by an In Data match.
referencing_foreign_keys list[string] A list of foreign keys referencing this column.

GET /api/schema-discovery/v2/{run_id}/

Authorization: User token or API token.

Retrieve schema discovery results with server-side pagination, sorting, filtering and searching.

GET /api/schema-discovery/v2/{run_id}/ Parameters

Field Type Required Location Description
limit number No Query Parameter The maximum number of results to return. Defaults to 50 if not set.
offset number No Query Parameter The index of the first item to be returned within the whole set of results. Defaults to 0 if not set.
ordering string No Query Parameter Controls the sort order of results. Specify one or more columns separated by commas. To specify descending sort order, prefix the field name with '-'. Defaults to ?ordering=schema,table,column.
search string No Query Parameter Performs a case-insensitive partial match on the schema, table or column name.
categories string No Query Parameter Filters the categories (Data Classifications) using an exact match. Valid values are PII, PHI or PCI.
data_type string No Query Parameter Filters the data type name (excluding the length or numeric precision/scale) e.g ?data_type=varchar.
description string No Query Parameter Searches the description using a case-insensitive partial match.
flagged_by string No Query Parameter Filters the Flagged By field using an exact match. Valid values are In-Data Discovery or Metadata Discovery.
is_sensitive boolean No Query Parameter Filters the results for sensitive matches. Set to true to return only sensitive results, or false for only non-sensitive.
constraint string No Query Parameter Filters for results with either Primary or Unique constraints. Valid values are primary or unique (case-insensitive).

GET /api/schema-discovery/v2/{run_id}/ Response

Status Code Description
200 A JSON serialised object containing pagination meta-data and a list of Schema Discovery Result objects.
Field Type Description
count number Total number of unpaginated results.
next string Pagination link to the next page of results.
previous string Pagination link to the previous page of results.
results list[object] A list of Schema Discovery Result objects.
table_metadata object Table constraint metadata for all tables appearing in the current page results. Structure: {"schema_name": {"table_name": {"primary_keys": [{"columns": ["col1"]}], "unique_keys": [{"columns": ["col2"]}], "foreign_keys": [{"columns": ["col3"]}]}}}

Schema Discovery Result object

Schema Discovery Result objects have the following fields:

Field Type Description
id number A unique id for the result.
column string The column name.
table string The table name.
schema string The schema name.
data object A v2 Schema Discovery Column Object.

v2 Schema Discovery Column Object

v2 Schema Discovery Column objects have the following fields:

Field Type Description
data_type string The data type for this field e.g varchar, integer, numeric, timestamp without time zone with the max_length or numeric_precision and numeric_scale appended.
max_length number The column length.
foreign_keys list[object] A list of foreign key objects containing name and referenced_column as a string containing schema.table.column.
discovery_matches list[object] A list of Discovery Match objects sorted by priority.
numeric_precision number The numeric precision of the column, the meaning of which depends on the database and data type.
numeric_scale number The numeric scale of the column, the meaning of which depends on the database and data type. Default is null.
constraint_columns list[string] A list of column names participating in the constraint.
pk_constraint_name string The name of the primary key constraint. Default is null.
uk_constraint_name string The name of the unique key constraint. Default is null.
unique_index_names list[string] A list of index names for this column.
referencing_foreign_keys list[object] A list of foreign keys referencing this column. The objects contain a name and referencing_column as a string containing schema.table.column.
categories list[string] A list of classifications for the flagged sensitive data: PII, PHI, PCI and/or Custom.
description string The reason the column was flagged as sensitive (blank for non-sensitive columns).
flagged_by string Indicates whether the column was flagged by In-Data Discovery or Metadata Discovery (or blank for non-sensitive columns).
constraint string Indicates whether the column is either a Primary or Unique key.

Discovery Match Object

Discovery Match objects have the following fields:

Field Type Description
label string A name for the rule that flagged the match. Can also be custom, custom_non_sensitive or ignore for user-defined match rules.
categories list[string] A list of classifications for the flagged sensitive data: PII, PHI, PCI and/or Custom.
flagged_by string Indicates whether the column was flagged by In-Data Discovery or Metadata Discovery.
description string The reason the column was flagged as sensitive.
hit_ratio Optional[int] The In-Data Discovery match rate. Contains null for Metadata Discovery matches.

Table Metadata Object

The table_metadata field in the schema discovery v2 API response provides constraint information for all tables that appear in the current page results. This is useful for understanding table relationships and constraints when generating rulesets.

Structure: A 3-level nested object: schema_name → table_name → constraint metadata

Example:

{
  "table_metadata": {
    "public": {
      "users": {
        "primary_keys": [{"columns": ["id"]}],
        "unique_keys": [{"columns": ["email"]}, {"columns": ["username"]}],
        "foreign_keys": [{"columns": ["department_id"]}]
      },
      "orders": {
        "primary_keys": [{"columns": ["order_id"]}],
        "unique_keys": [],
        "foreign_keys": [{"columns": ["user_id"]}, {"columns": ["product_id"]}]
      }
    }
  }
}

Constraint Types:

Field Type Description
primary_keys list[object] List of primary key constraint objects. Each object contains {"columns": ["col1", "col2"]} for the constraint columns (supports composite keys).
unique_keys list[object] List of unique key constraint objects. Each object contains {"columns": ["col1", "col2"]} for the constraint columns (supports composite unique keys).
foreign_keys list[object] List of foreign key constraint objects. Each object contains {"columns": ["col1", "col2"]} for the constraint columns (supports composite foreign keys).

Notes:

  • Only tables that appear in the current page of results are included in table_metadata.
  • For each included table, constraint information is based on ALL columns in that table, not just the columns visible in the current page.
  • Empty arrays indicate no constraints of that type exist for the table.
  • Composite constraints (spanning multiple columns) are represented as arrays with multiple column names.

Generating Rulesets

POST /api/generate-ruleset/

Authorization: User token only.

Returns a ruleset string for selected columns of a connection.

Prerequisite: Make sure you have the schema-discovery report for the connection specified in the post data.

POST /api/generate-ruleset/[v1/|v2/] Parameters

Field Type Required Location Description
connection string Yes Request Body The UUID of the connection to generate a ruleset for.
selected_columns dict Yes Request Body A nested dictionary specifying which columns to include: {"schema_name": {"table_name": ["column1", "column2"]}}
hash_columns dict No Request Body Optional hash columns specification for deterministic masking with table-level defaults and column-level overrides. Structure: {"schema_name": {"table_name": {"table": ["default_hash1"], "columns": {"column_name": ["hash_col1"], "other_col": null}}}}

POST /api/generate-ruleset/[v1/|v2/] curl example

curl -X POST "https://<your-datamasque-host>/api/generate-ruleset/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "connection": "<your-connection-id>",
           "selected_columns": {
             "schema_name": {
               "table_name": [
                 "column_name_1",
                 "column_name_2"
               ]
             }
           },
           "hash_columns": {
             "schema_name": {
               "table_name": {
                 "table": ["default_hash_column"],
                 "columns": {
                   "column_name_1": ["id", "user_code"],
                   "column_name_2": null
                 }
               }
             }
           }
         }'

POST /api/generate-ruleset/[v1/] Response

The default response for a version 1 request is a json encoded string containing the ruleset yaml. The trailing /v1/ is optional for version 1.

POST /api/generate-ruleset/v2/ Response

The version 2 response is a plain text containing the ruleset yaml.

POST /api/generate-file-ruleset/

Authorization: User token only.

Returns a ruleset string for selected data of a file connection.

The selected data is a list of file groups, each of which contains:

  • A list of files which are the full paths relative to the base directory of the connection.
  • A list of locators, which are either JSON locators or strings containing a single header column name. JSON locators must be formatted as lists even if they consist of a single element.

Each file group will generate at least one task in the ruleset (either mask_file or mask_tabular_file).

Generally, only one task will be generated per file group, but in cases where files have different extensions, delimiters or encodings, multiple tasks will be generated to cater for these settings.

File groups should only contain files of the same type, that is, don't specify object files, multi-record files, or tabular files in the same file group. If multiple file types are mixed, then the generated ruleset will attempt to split into multiple tasks, but the results may be unexpected.

Prerequisite: Make sure you have the file-discovery report for the connection specified in the POST data so that a discovery run has been completed on the connection and the files can be selected from the report.

POST /api/generate-file-ruleset/ curl example

curl -X POST "https://<your-datamasque-host>/api/generate-file-ruleset/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "connection": "<your-connection-id>",
           "selected_data": [
             {
               "files": ["file1.json", "file2.json"],
               "locators": [["age"], ["users", "*", "name"]]
             },
             {
               "files": ["file1.csv", "file2.csv"],
               "locators": ["gender", "address"]
             },
             [repeated for different file groups…]
           ],
         }'

POST /api/generate-file-ruleset/ Response

The response is plain text containing the ruleset yaml.

Generate Ruleset Result Object

Generate Ruleset Result objects are returned by DataMasque for the async-generate-ruleset family of APIs. They have the following fields:

Field Type Description
connection string The ID of the connection for which a ruleset is being generated.
generated_ruleset string The ruleset that has been generated. Not applicable if ruleset generation was started using the from-csv API.
status string The status of the ruleset generation task. One of queued, running, finished, failed, or cancelled.
status_message string A status message describing the progress of the ruleset generation task.
error_message string The error message when generating the ruleset has failed.
last_updated string The timestamp of the last update to this Generate Ruleset Result, in ISO 8601 format.

Endpoint to query to get the generated ruleset:

GET /api/async-generate-ruleset/{connection_id}/

Authorization: User token only.

Returns result of generating ruleset progress.

GET /api/async-generate-ruleset/{connection_id}/ Parameters

Field Type Required Location Description
connection_id string Yes URL Path The id of the Connection.

GET /api/async-generate-ruleset/{connection_id}/ Responses

Status Code Description
200 A JSON serialised Generate Ruleset Result Object.
404 Not Found: No connection with the specified ID exists.

GET /api/async-generate-ruleset/{connection_id}/ curl example

curl "https://<your-datamasque-host>/api/async-generate-ruleset/{connection_id}/" \
     -H "Authorization: Token <your-api-token>"

POST /api/async-generate-ruleset/{connection_id}/

Authorization: User token only.

Start generating ruleset for selected columns of a database connection or for selected data of a file connection.

POST /api/async-generate-ruleset/{connection_id}/ Parameters

Field Type Required Location Description
connection_id string Yes URL Path The id of the Connection.
selected_columns dict Yes Request Body A nested dictionary specifying which columns to include: {"schema_name": {"table_name": ["column1", "column2"]}}
selected_data list Yes* Request Body For file connections: list of file groups with files and locators. *Required for file connections, not database connections.
hash_columns dict No Request Body Optional hash columns specification for deterministic masking with table-level defaults and column-level overrides. Structure: {"schema_name": {"table_name": {"table": ["default_hash1"], "columns": {"column_name": ["hash_col1"], "other_col": null}}}}

POST /api/async-generate-ruleset/{connection_id}/ Responses

Status Code Description
201 A JSON serialised Generate Ruleset Result Object.
404 Not Found: No connection with the specified ID exists.

POST /api/async-generate-ruleset/{connection_id}/ curl example

For generating rulesets on database connections:

curl -X POST "https://<your-datamasque-host>/api/async-generate-ruleset/{connection_id}/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "selected_columns": {
             "schema_name": {
               "table_name": [
                 "column_name_1",
                 "column_name_2"
               ]
             }
           },
           "hash_columns": {
             "schema_name": {
               "table_name": {
                 "table": ["default_hash_column"],
                 "columns": {
                   "column_name_1": ["id", "user_code"],
                   "column_name_2": null
                 }
               }
             }
           }
         }'

For generating rulesets for file connections:

POST /api/async-generate-ruleset/{connection_id}/ curl example

curl -X POST "https://<your-datamasque-host>/api/async-generate-ruleset/{connection_id}/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "selected_data": [
             {
               "files": ["file1.json", "file2.json"],
               "locators": [["age"], ["users", "*", "name"]]
             },
             {
               "files": ["file1.csv", "file2.csv"],
               "locators": ["gender", "address"]
             },
             [repeated for different file groups…]
           ],
         }'

DELETE /api/async-generate-ruleset/{connection_id}/

Authorization: User token only.

Cancels ruleset generation currently in progress for a connection. If the ruleset generation has already finished, deletes any generated ruleset.

Warning! Deletion of the generated ruleset is irreversible.

DELETE /api/async-generate-ruleset/{connection_id}/ Parameters

Field Type Required Location Description
connection_id string Yes URL Path The id of the Connection.

DELETE /api/async-generate-ruleset/{connection_id}/ Responses

Status Code Description
200 Ruleset generation cancelled before any results were processed.
204 Ruleset generation had finished. The generated ruleset has been deleted.
404 Not Found: No connection with the specified ID exists.

DELETE /api/async-generate-ruleset/{connection_id}/ curl example

curl -X DELETE "https://<your-datamasque-host>/api/async-generate-ruleset/{connection_id}/" \
     -H "Authorization: Token <your-api-token>"

POST /api/async-generate-ruleset/{connection_id}/from-csv/

Authorization: User token only.

Start generating a ruleset for selected columns of a database connection. The columns are specified by modifying the CSV report retrieved from the /api/runs/{run_id}/db-discovery-results/report/ endpoint. Specifically, there is one discovered database column detailed in each row of the CSV report, and if that column is to be included in ruleset generation, the Selected column of the CSV should be marked with 1, true, y or yes (case-insensitive).

POST /api/async-generate-ruleset/{connection_id}/from-csv/ Parameters

Field Type Required Location Description
connection_id string Yes URL Path The id of the Connection.
csv_or_zip_file file Yes Request Body The byte content of the CSV, or the ZIP file containing one or more CSVs.
target_size_bytes int No Request Body Generate rulesets of approximately this size in bytes. Defaults to 512,000 (500 KiB).
force_run boolean No Request Body If set to true, cancel any existing ruleset generation and restart it. Defaults to false.

POST /api/async-generate-ruleset/{connection_id}/from-csv/ Responses

Status Code Description
201 A JSON serialised Generate Ruleset Result Object.
404 Not Found: No connection with the specified ID exists.

POST /api/async-generate-ruleset/{connection_id}/from-csv/ curl example

curl -X POST "https://<your-datamasque-host>/api/async-generate-ruleset/{connection_id}/" \
     -H "Authorization: Token <your-api-token>" \
     -F "csv_file=@selected_report.csv" \
     -F "target_size_bytes=250000"

GET /api/async-generate-ruleset/{connection_id}/download-rulesets/

Authorization: User token only.

Once ruleset generation invoked via POST /api/async-generate-ruleset/{connection_id}/from-csv/ is completed, query this endpoint to download the rulesets in a ZIP file.

GET /api/async-generate-ruleset/{connection_id}/download-rulesets/ Parameters

Field Type Required Location Description
connection_id string Yes URL Path The id of the Connection.

GET /api/async-generate-ruleset/{connection_id}/download-rulesets/ Responses

Status Code Description
200 Returns a streamed ZIP file containing the generated rulesets.
400 Bad Request: The ruleset generation is still in progress, or has failed.

If an error response is received, query the GET /api/async-generate-ruleset/{connection_id}/ endpoint to check the status of ruleset generation.

GET /api/async-generate-ruleset/{connection_id}/download-rulesets/ curl example

curl -o rulesets.zip "https://<your-datamasque-host>/api/async-generate-ruleset/{connection_id}/download-rulesets/" \
     -H "Authorization: Token <your-api-token>"

File Data Discovery

POST /api/run-file-data-discovery/

Authorization: User token only.

Executes data discovery against files on a file connection. The file connection must already be configured. Use the UUID of the file connection in the request, which can be found:

  • at the top of the page when you view the connection in the DataMasque UI, or
  • in the URL when you view the connection in the DataMasque UI, or
  • in the id field of the Connection Object.

Discovery keywords

By default, DataMasque's extensive list of built-in keywords is used to identify which fields and attributes in the files are considered sensitive. DataMasque matches the name of the field or attribute against each keyword using a case-insensitive, partial match. For example, a field named credit_CARD_NUMBER will match the Credit card keyword.

You can use various options to refine the set of discovery keywords.

  • Setting disable_built_in_keywords to true means that the built-in keyword list linked above will not be used. In this case, the discovery process will use only the keywords given in custom_keywords and any configured global custom keywords.
  • The custom_keywords option allows you to specify a list of additional keywords to match on. Any fields or attributes whose name includes one or more of those keywords will be flagged as sensitive.
  • A match between a field or attribute's name and a value in the ignored_keywords list will cause a field or attribute to be completely excluded from the results, even if its name suggests that the field may contain sensitive data.
  • Global keywords, as configured through the Settings page of the DataMasque UI, are also considered unless disable_global_custom_keywords and/or disable_global_ignored_keywords (as appropriate) are set to true.

Warning! Ignored keywords have priority. If a field or attribute name matches both a built-in, global, or custom keyword and also matches an entry in ignore_keywords or a global ignored keyword, the field or attribute will not be included in the discovery results.

Specifying files to discover

Supported filetypes for discovery are:

  • JSON (.json)
  • NDJSON (.ndjson)
  • Parquet (.parquet)
  • CSV (.csv)

Note: Files' types are determined solely by the file extension, not by their content.

Use the include, skip and recurse options to control which files are included in the discovery process. These have the same syntax and meaning as in a from_file task definition. If none of these options are included, DataMasque will run discovery against all files (of the supported filetypes) in the base directory specified on the connection, but will not recurse into subdirectories.

See also Choosing files to mask with include/skip for an exact specification of the behaviour of, and some common examples of, include and skip rules.

Warning! If a file matches both an include and a skip rule, that file will not be included in data discovery.

Note: Take care to correctly escape backslashes in include or skip regexes. For example, if you want to match a literal dot (.) in a filename, the regex needs to escape the dot with a backslash and this backslash must itself be escaped as part of JSON encoding rules, since the request body is in JSON format. So you might use the JSON object {"regex": "file\\.[0-9]+\\.csv"}, representing the regex file\.[0-9]+\.csv which will match file.53.csv but not filex53.csv.

Encoding of CSV files

The encoding option controls how DataMasque interprets CSV files. The default encoding is utf-8. Refer to Python Standard Encodings for a list of supported encodings.

Supported Parquet column types

The list of Parquet column data types supported by file data discovery is the same as the list of supported data types for Parquet masking. See the list of supported data types here.

For complex columns (those of struct, map and list type), also called nested columns, all fields of scalar data type within the columns are discovered separately. In the file discovery reports, the locators for the individual scalar fields are given as JSON paths with the column name as the first element.

Note: This differs from the syntax used for masking these fields where the column name must be specified separately from the path to the field within the column.

For example, with a column named staff of type map<string, struct<name: string, employee_id: int64, salary_history: list<float>>> (a map where the keys are strings and the values are a structure type with keys name, employee_id, and salary_history, the latter being a list), the discovered fields will all have one of the following path formats:

  • staff/<key value>/name
  • staff/<key value>/employee_id
  • staff/<key value>/salary_history/*

where <key value> is a key in the top-level map. Notice that all list indices are replaced with the wildcard * and treated as a single field.

Custom and ignored keywords match on the name of the individual field (such as name in the above example), not the name of the column. For list fields, they match on the last string element of the path (ignoring list indices), for example salary_history.

In-data discovery options

The in_data_discovery parameter on the API request body allows you to control whether and how the discovery process uses in-data discovery to refine sensitive data matches. It is an object parameter with the following fields.

  • You must specify the enabled parameter (true or false).
  • Optional parameters are a row_sample_size (positive integer), force (a boolean), a list of zero or more custom_rules, and a list of zero or more non_sensitive_rules.
  • Each entry in custom_rules is an object with parameters name and pattern, where name is any user-defined name and pattern is a regex.
  • Each entry in non_sensitive_rules is an object with a pattern parameter, again a regex.
  • row_sample_size defaults to 1000.
  • force defaults to false.
  • custom_rules and non_sensitive_rules are empty by default.

When enabled, in-data discovery applies the built-in rules, alongside any specified custom_rules and non_sensitive_rules, matching against the data within tabular file columns, or scalar values within JSON documents or complex Parquet columns.

Warning! Non-sensitive rules have priority. If a field or attribute name matches a keyword, built-in IDD rule or custom IDD rule, and also matches a non-sensitive rule, the field or attribute will be marked in the discovery results as Custom Non-Sensitive.

The row_sample_sizecontrols how many samples the in-data discovery process will examine to try to identify the type of data. Configure the row_sample_size according to your needs, bearing in mind that in-data discovery samples only the first <row_sample_size> rows or values encountered when processing the file (so the first 1000 rows in a CSV file, for example, with the default sample size). Use of very large sample sizes can slow down data discovery and consume a lot of RAM (see also this table of memory limits for in-data discovery).

  • If your files are small and/or consistent in that they have the same kind of data present in most or all rows, then a sample size of 100-500 rows is sufficient.
  • If you have large files with sparse data (many nulls) and/or differing data formats within a column or JSON path, use a larger sample size.

When enabled, force will run IDD on a column even if schema discovery has already flagged the column as containing sensitive data.

POST /api/run-file-data-discovery/ Parameters

Field Type Required Description
connection string Yes The id of the Connection.
in_data_discovery object No In-data discovery options. An object containing enabled, row_sample_size, custom_rules, ignore_rules and force options. Defaults to {enabled: false}.
custom_keywords array[string] No List of keywords that, where a field or attribute's name matches one or more of the keywords, indicates the column contains sensitive data. Default value is an empty list.
ignored_keywords array[string] No List of keywords that, where a field or attribute's name matches one or more of the keywords, indicates the field or attribute should be excluded from the schema discovery results. Default value is an empty list.
disable_global_custom_keywords boolean No If set to true, then the user-defined global set of custom keywords will not be used to flag fields or attributes as sensitive. Default value is false.
disable_global_ignored_keywords boolean No If set to true, then the user-defined global set of ignored keywords will not be used to exclude fields or attributes from the discovery results. Default value is false.
disable_built_in_keywords boolean No If set to true, then DataMasque's built-in list of keywords will not be used to flag fields or attributes as sensitive. Default value is false.
include array[object] No Files to discover, specified as glob or regex. Default value is an empty list, meaning everything will be included.
skip array[object] No Files to exclude, specified as glob or regex. Default value is an empty list, meaning everything will be included.
recurse boolean No Whether to recurse into subdirectories of the base directory, or of items matched by include. Default value is false.
encoding string No File byte encoding. Only applies to CSV files. Default value is utf-8.
workers integer No Number of workers. Refer to the File Ruleset Generator page for information. Allowed range is 1-32. Defaults to 1.

POST /api/run-file-data-discovery/ Responses

Data discovery runs asynchronously as a special type of masking run. This API endpoint returns a Run object which contains an id field. Use the GET /api/runs/{id}/ endpoint with this run ID to query the status of the data discovery process. To retrieve the file discovery results when the run is complete, use the GET /api/runs/{id}/file-discovery-results/ endpoint with this run ID.

Status Code Description
201 A JSON serialised Run object.

POST /api/run-file-data-discovery/ curl example

curl -X POST "https://<your-datamasque-host>/api/run-file-data-discovery" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "connection": "<your-connection-id>",
           "in_data_discovery": {
             "enabled": true,
             "row_sample_size": 500,
             "custom_rules": [
               {
                 "name": "temp_staff",
                 "pattern": "temp.*"
               }
             ],
             "non_sensitive_rules": [
               {"pattern": "retired.*"}
             ],
             "force": false
           },
           "custom_keywords": ["id1", "id2"],
           "ignored_keywords": ["ignore1"],
           "include": [
             {"glob": "*.ndjson"},
             {"glob": "*.json"},
           ],
           "skip": [
             {"regex": "backup/staff[0-9]+\\.json"},
           ],
           "recurse": true,
           "workers": 4
         }'

GET /api/runs/{id}/file-discovery-results/

Authorization: User token or API token.

Retrieve file discovery results.

GET /api/runs/{id}/file-discovery-results/ Parameters

Field Type Required Location Description
id integer Yes URL Path The id of the Run.

GET /api/runs/{id}/file-discovery-results/ Responses

Status Code Description
200 A JSON serialised list of File Discovery Result objects.

GET /api/runs/{id}/file-discovery-results/ curl example

curl "https://<your-datamasque-host>/api/runs/{id}/file-discovery-results/" \
     -H "Authorization: Token <your-api-token>"

GET /api/runs/{id}/file-discovery-results/ Example response

This shows a group of results where one file was discovered with a Metadata match on Passenger ID, an In-Data match on Name and no matches on Ticket.

[
  {
    "id": 1,
    "connection": {
      "id": "f795b7f1-d654-41c8-bb7c-db741d81dc19",
      "name": "example_file_source"
    },
    "file_type": "csv",
    "files": [
      {
        "path": "example.csv",
        "delimiter": ",",
        "encoding": "utf-8",
        "file_type": "csv"
      }
    ],
    "results": [
      {
        "locator": "PassengerId",
        "matches": [
          {
            "label": "identifiers",
            "categories": ["PII", "PHI"],
            "flagged_by": "Metadata Discovery",
            "description": "Identification",
            "hit_ratio": null
          }
        ],
        "data_types": ["int"]
      },
      {
        "locator": "Name",
        "matches": [
          {
            "label": "name",
            "categories": ["PII", "PCI", "PHI"],
            "flagged_by": "In-Data Discovery",
            "description": "Full Names",
            "hit_ratio": 80
          }
        ],
        "data_types": ["str"]
      },
      {
        "locator": "Ticket",
        "matches": [],
        "data_types": ["str"]
      }
    ]
  }
]

File Discovery Result Object

File Discovery Result objects have the following fields:

Field Type Description
id integer The id of the File Discovery Result.
connection object The UUID and name identifying the connection used for this File Discovery Result.
file_type string The file type (csv, parquet, json, or ndjson). File Discovery Results are grouped by file type.
files array[object] A list of File objects.
results array[object] A list of Result objects.

File Object

File objects have the following fields:

Field Type Description
path string The discovered file's path, relative to the base directory of the connection.
file_type string The file type (csv, parquet, json, or ndjson).
delimiter Optional[string] For delimited text files, the field separator. e.g "," for csv
encoding Optional[string] The file encoding, for example "utf-8".

Result Object

Result objects have the following fields:

Field Type Description
locator array['string' or 'int'] or string Either a JSON locator or a column name.
matches array['object'] A list of Match objects.
data_types array['string'] The list of data types found for this field: int, long, str, date, time, year, timestamp, boolean, float, or decimal.

Match Object

Match objects have the following fields:

Field Type Description
categories array['string'] A list of classifications for the flagged sensitive data: PII, PHI, PCI and/or Custom.
flagged_by string Whether the column was flagged for sensitive data through in-data discovery or through the standard sensitive data discovery / keyword matching process. Metadata Discovery or In-Data Discovery.
description string The name of the rule which caused the column to be flagged for sensitive data.
label string Machine-readable representation of description.
hit_ratio Optional[int] The In-Data Discovery match rate. Contains null for Metadata Discovery matches.

Git Setting Object

Git settings are global for the DataMasque instance and can only be updated by an admin user. Git settings are updated on the Settings page in the DataMasque UI.

Git Setting objects have the following fields:

Field Type Description
git_repository_url string The URL of where the Git repository is hosted.
git_branch string The name of the Git branch from which DataMasque will push or pull.
git_directory_path string The directory that DataMasque will push and pull rulesets to, relative to the root of the repository. Note that DataMasque does not support pushing/pulling rulesets in subdirectories of this directory.

GET /api/git-setting/

Authorization: User token only.

Retrieve a Git Setting Object with information about the DataMasque instance's Git settings.

GET /api/git-setting/ Parameters

No parameters.

GET /api/git-setting/ Responses

Status Code Description
200 A JSON serialized Git Setting Object for the DataMasque instance.

GET /api/git-setting/ curl example

curl "https://<your-datamasque-host>/api/git-setting/" \
     -H "Authorization: Token <your-api-token>"

GET /api/git-setting/user/

Authorization: User token only.

Retrieve a Git Setting Object with information about the DataMasque instance's Git settings. If the current user has specified a git_directory_path, this will be present in the response. Otherwise, the git_directory_path will be the global one for the DataMasque instance.

GET /api/git-setting/user/ Parameters

No parameters.

GET /api/git-setting/user/ Responses

Status Code Description
200 A JSON serialized Git Setting Object for the DataMasque instance.

GET /api/git-setting/user/ curl example

curl "https://<your-datamasque-host>/api/git-setting/user/" \
     -H "Authorization: Token <your-api-token>"

SSH Key Object

SSH Key objects have the following fields:

Field Type Description
name string The specified filename of the SSH Key file.
date_uploaded string The ISO 8601 datetime string of when the user uploaded the SSH key.

GET /api/git-ssh-key/

Authorization: User token only.

Retrieve an SSH Key Object for information about the current user's uploaded SSH Key.

GET /api/git-ssh-key/ Parameters

No parameters.

GET /api/git-ssh-key/ Responses

Status Code Description
200 A JSON serialized SSH Key Object which is the most recent SSH Key Upload for the user which made the request.

GET /api/git-ssh-key/ curl example

curl "https://<your-datamasque-host>/api/git-ssh-key/" \
     -H "Authorization: Token <your-api-token>"

PUT /api/git-ssh-key/

Authorization: User token only.

Upload an SSH Key to be used to access a Git remote repository.

Warning: A user may have only one SSH key at a time, so the existing key will be deleted and replaced with the uploaded key for the user making the request.

PUT /api/git-ssh-key/ Parameters

Field Type Required Location Description
key_file file Yes Form Field The SSH Key file.
name string Yes Form Field The name of the file.

PUT /api/git-ssh-key/ Responses

Status Code Description
200 A JSON serialized SSH Key Object, which is the most recent SSH Key Upload for the user making the request.

PUT /api/git-ssh-key/ curl example

curl -X PUT "https://<your-datamasque-host>/api/git-ssh-key/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: multipart/form-data" \
     -F "key_file=@</path/to/your/file>" \
     -F "name=<your-ssh-key-filename>"

DELETE /api/git-ssh-key/

Authorization: User token only.

Delete the current user's uploaded SSH key.

DELETE /api/git-ssh-key/ Parameters

No parameters.

DELETE /api/git-ssh-key/ Responses

Status Code Description
204 The SSH key associated with the requesting user has been deleted.

DELETE /api/git-ssh-key/ curl example

curl DELETE -X "https://<your-datamasque-host>/api/git-ssh-key/" \
     -H "Authorization: Token <your-api-token>"

GET /api/ruleset-git/

Authorization: User token only.

Pull the content of a specific ruleset given its commit ID. The current user's Git SSH key is used for authentication.

How File Paths Are Built

Internally, DataMasque generates the name of the file by appending the specified extension to ruleset_name. The file name is then appended to git_directory_path (from the DataMasque Git Settings) to build the full file path. For example, for a ruleset_name of My Ruleset, extension of .yml and git_directory_path of masking/rulesets, the file masking/rulesets/My Ruleset.yml will be retrieved. Its contents will be that as at the specified commit ID.

GET /api/ruleset-git/ Parameters

Field Type Required Location Description
commit_id string Yes Query Parameter The Git commit ID for the ruleset.
ruleset_name string Yes Query Parameter The name of the ruleset. Used to build the path as per How File Paths Are Built above.
extension string No Query Parameter The extension to save with the ruleset name. Must be .yml or .yaml. Default to .yml if missing.

GET /api/ruleset-git/ Responses

Status Code Description
200 A JSON object with a single key, config_yaml, that contains the ruleset content

GET /api/ruleset-git/ curl example

curl "https://<your-datamasque-host>/api/ruleset-git/?commit_id=<your-full-commit-id>&ruleset_name=<your-ruleset-name>&extension=.yaml" \
     -H "Authorization: Token <your-api-token>"

POST /api/ruleset-git/

Authorization: User token only.

Commit then push changes upstream for a specific ruleset.

POST /api/ruleset-git/ Parameters

Field Type Required Location Description
commit_message string Yes Request Body The Git commit message for the ruleset changes.
ruleset_name string Yes Request Body The name of the ruleset. Used to build the path as per How File Paths Are Built above.
extension string No Request Body The extension to save with the ruleset name. Must be .yml or .yaml. Default to .yml if missing.
ruleset_content string Yes Request Body The YAML contents of the ruleset.

POST /api/ruleset-git/ Responses

Status Code Description
200 Operation succeeded.

POST /api/ruleset-git/ curl example

curl -X POST "https://<your-datamasque-host>/api/ruleset-git/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "commit_message": "Update ruleset",
           "ruleset_name": "<your-ruleset-filename>",
           "extension": ".yml",
           "ruleset_content": "version: \"1.0\"\ntasks:\n  - type: run_data_discovery"
         }'

GET /api/ruleset-git/files/

Authorization: User token only.

This endpoint lists the git_directory_path in the remote repository configured for the DataMasque instance. It considers any files ending in .yml to be ruleset files, and will fetch the list of commits for each of them. It does not enter into subdirectories of git_directory_path.

GET /api/ruleset-git/files/ Parameters

No parameters.

GET /api/ruleset-git/files/ Responses

Example Response

The response is a JSON object with each key being the name of a file with a .yml extension in the git_directory_path. Each file entry has an array objects with a commit ID, commit date and commit message.

{
  "Ruleset1.yml": [
    {"commit": "f061s…46756", "date": "2024-01-10 12:31:45", "message": "Added Column"},
    {"commit": "64c18…1a279", "date": "2024-01-09 10:19:13", "message": "Removed Column"}
  ],
  "Another Ruleset.yml": [
    {"commit": "377f5…b32f4", "date": "2023-12-25 12:31:45", "message": "Update rule"}
  ]
}
Response Codes
Status Code Description
200 A JSON serialized list of ruleset names and their associated Git commit history.

GET /api/ruleset-git/files/ curl example

curl "https://<your-datamasque-host>/api/ruleset-git/files/" \
     -H "Authorization: Token <your-api-token>"

GET /api/library-git/

Authorization: User token only.

Pull the content of a specific ruleset library given its commit ID. The current user's Git SSH key is used for authentication.

How Library File Paths Are Built

Library files are stored under a libraries/ subdirectory within the configured git_directory_path. Libraries with a namespace are stored in subdirectories matching the namespace.

For example, with a git_directory_path of masking/rulesets:

Library Name Namespace File Path
my_library (none) masking/rulesets/libraries/my_library.yml
emails pii masking/rulesets/libraries/pii/emails.yml
addresses org/team masking/rulesets/libraries/org/team/addresses.yml

GET /api/library-git/ Parameters

Field Type Required Location Description
commit_id string Yes Query Parameter The Git commit ID for the library.
library_name string Yes Query Parameter The name of the library (without extension). Must not contain / characters.
namespace string No Query Parameter The namespace (subdirectory) for the library. May contain / for nested namespaces (e.g., org/team).
extension string No Query Parameter The file extension. Must be .yml or .yaml. Defaults to .yml.

GET /api/library-git/ Responses

Status Code Description
200 A JSON object with a single key, config_yaml, that contains the library content
404 The library file was not found at the specified commit

GET /api/library-git/ curl example

curl "https://<your-datamasque-host>/api/library-git/?commit_id=<your-full-commit-id>&library_name=<your-library-name>&namespace=pii&extension=.yml" \
     -H "Authorization: Token <your-api-token>"

POST /api/library-git/

Authorization: User token only.

Commit then push changes upstream for a specific ruleset library.

POST /api/library-git/ Parameters

Field Type Required Location Description
commit_message string Yes Request Body The Git commit message for the library changes.
library_name string Yes Request Body The name of the library (without extension). Must not contain / characters.
namespace string No Request Body The namespace (subdirectory) for the library. May contain / for nested namespaces (e.g., org/team).
extension string No Request Body The file extension. Must be .yml or .yaml. Defaults to .yml.
library_content string Yes Request Body The YAML contents of the library.

POST /api/library-git/ Responses

Status Code Description
204 Operation succeeded.
400 No changes to push, or validation error in request body.

POST /api/library-git/ curl example

curl -X POST "https://<your-datamasque-host>/api/library-git/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "commit_message": "Update library",
           "library_name": "my_library",
           "namespace": "pii",
           "extension": ".yml",
           "library_content": "version: \"1.0\"\nrules:\n  - mask_type: from_fixed"
         }'

GET /api/library-git/files/

Authorization: User token only.

This endpoint lists all library files in the libraries/ subdirectory of the configured git_directory_path. It recursively searches for files ending in .yml or .yaml and fetches the list of commits for each. Namespaced libraries are included with their relative path from the libraries/ directory.

GET /api/library-git/files/ Parameters

No parameters.

GET /api/library-git/files/ Responses

Example Response

The response is a JSON object with each key being the relative path of a library file from the libraries/ directory. Each file has an array of commit objects showing its history.

{
  "my_library.yml": [
    {
      "commit": "abc123abc123abc123abc123abc123abc123abcd",
      "date": "2026-03-15T10:30:00+00:00",
      "message": "Add my_library"
    }
  ],
  "pii/emails.yml": [
    {
      "commit": "def456def456def456def456def456def4560123",
      "date": "2026-03-14T09:00:00+00:00",
      "message": "Update email masking rules"
    }
  ],
  "org/team/addresses.yml": [
    {
      "commit": "789abc789abc789abc789abc789abc789abc0123",
      "date": "2026-03-13T14:00:00+00:00",
      "message": "Initial commit"
    }
  ]
}
Status Code Description
200 A JSON object mapping library file paths to their associated Git commit history.

GET /api/library-git/files/ curl example

curl "https://<your-datamasque-host>/api/library-git/files/" \
     -H "Authorization: Token <your-api-token>"

Exporting DataMasque Configuration

To keep a backup of the data stored in DataMasque, you can export it to a Zip file. This is done by making a GET request to /api/export/v1/. Optionally, you can also specify the export_type query parameter to select which data to include in the export. The parameter may be specified multiple times to specify different types of data to include in the same Zip file.

The Zip file will have the following structure, but please note that some files/directories may be missing if those files were not included in the export, due to setting an export_type.

Path Type Description
manifest.json File A JSON file containing metadata about the export and other files in the Zip.
rulesets/database/ Directory A directory containing database masking rulesets in YAML format.
rulesets/file/ Directory A directory containing file masking rulesets in YAML format.
libraries/ Directory A directory containing ruleset libraries in YAML format, organized by namespace.
seed_files/ Directory A directory containing custom seed files referenced by rulesets (only if include_seed_files=true).
errors.txt File A text file listing any errors encountered during export (e.g., missing seed files). Only if errors exist.

Export Types

The following export types may be used to control the data included in the export archive:

Currently, only the export of Rulesets is supported, therefore this is no difference in specifying rulesets as the export_type or omitting the export_type parameter completely.

Export Type Description
all Include all data described in this table. This is the default if no export_type is selected.
rulesets Include only rulesets.

manifest.json format

The manifest.json file contains the following information:

  • metadata: Metadata about the export archive.
    • version: The version format of the export file.
    • exported_at: The UTC date and time the export was created, in ISO format.
  • data: Information about the files included in the export archive.
    • rulesets: A list of metadata about the exported ruleset. Each object in the list contains the id, name and type (database or file) for each exported ruleset.
    • libraries: A list of metadata about the exported libraries (only present if the export contains libraries). Each object in the list contains the id, name, and namespace for each exported library.

Ruleset Export Naming

When rulesets are exported to a Zip archive, they are stored in either the rulesets/database/ directory, (for database rulesets) or rulesets/file/ directory (for file rulesets).

The name of the file is built by appending .yml to the ruleset name. For example:

  • The database masking ruleset named Ruleset 01 would be exported to rulesets/database/Ruleset 01.yml.
  • The file masking ruleset named Ruleset F would be exported to rulesets/file/Ruleset F.yml.

Note: Rulesets that have been deleted from DataMasque are not visible in the ruleset list in the DataMasque dashboard, but are still retained in the DataMasque database because runs reference them. These "archived" rulesets are not including the Zip export.

Library Export Naming

Libraries are stored in the libraries/ directory, organized by namespace.

The path of each library file is built from its namespace and name. For example:

  • A library named pii_masks in namespace acme would be exported to libraries/acme/pii_masks.yml.
  • A library named common_masks with no namespace would be exported to libraries/common_masks.yml.

When exporting specific rulesets, only the libraries used by those rulesets are included. When exporting all rulesets, all active libraries are included.

GET /api/export/v1/

Authorization: User token only.

Export DataMasque data to a Zip archive in the Version 1 format. The filename of the archive will be based on the export type selected, and contain the current UTC date and time. For example: datamasque_export_rulesets_20240211-091507.zip.

GET /api/export/v1/ Parameters

Field Type Required Location Description
export_type string No Query Parameter The type of data to export (see Export Types for a full list). Defaults to all.
ruleset_ids string No Query Parameter Comma-separated list of ruleset UUIDs to export. If omitted, all rulesets are exported.
include_seed_files string No Query Parameter Set to true to include custom seed files referenced by the exported rulesets. Default is false.

Multiple export types may be specified by using multiple export_type query parameters. For example, /api/export/v1/?export_type=type_a&export_type=type_b.

Selective Ruleset Export

To export only specific rulesets, provide a comma-separated list of ruleset UUIDs in the ruleset_ids parameter. For example: /api/export/v1/?ruleset_ids=uuid1,uuid2,uuid3.

Ruleset UUIDs can be found in the ruleset detail page URL or via the GET /api/rulesets/ endpoint.

Note: When exporting a large number of rulesets, the ruleset_ids query parameter may exceed the maximum URL length supported by your server or HTTP client. If you experience issues exporting many rulesets at once, use the POST /api/export/v1/ endpoint instead, which accepts the same parameters in the request body.

Including Seed Files

When include_seed_files=true is specified, the export will include any custom seed files referenced by the exported rulesets. Default DataMasque seed files (those prefixed with DataMasque_) are not included, as they are already available on all DataMasque installations.

If a ruleset references a seed file that does not exist, the export will still succeed, but an errors.txt file will be included in the Zip archive listing the missing seed files and which rulesets reference them.

GET /api/export/v1/ curl examples

When using curl, specify the -O flag to output the response to disk, and the -J flag to allow the response to specify the name (as per the example above).

Export all rulesets:

curl "https://<your-datamasque-host>/api/export/v1/" \
     -H "Authorization: Token <your-api-token>" \
     -J -O

A Zip file named like datamasque_export_all_20240211-091507.zip will be saved to the current directory.

Export specific rulesets by UUID:

curl "https://<your-datamasque-host>/api/export/v1/?ruleset_ids=9d641e97-adf7-4f22-9089-afc3711bf222,04ea20f0-ad4c-498e-881f-b0bc79d83ba7" \
     -H "Authorization: Token <your-api-token>" \
     -J -O

Export all rulesets with their seed files:

curl "https://<your-datamasque-host>/api/export/v1/?include_seed_files=true" \
     -H "Authorization: Token <your-api-token>" \
     -J -O

Export specific rulesets with seed files:

curl "https://<your-datamasque-host>/api/export/v1/?ruleset_ids=9d641e97-adf7-4f22-9089-afc3711bf222&include_seed_files=true" \
     -H "Authorization: Token <your-api-token>" \
     -J -O

POST /api/export/v1/

Authorization: User token only.

Export DataMasque data to a Zip archive in the Version 1 format, with parameters supplied in a JSON request body. This is the preferred method when exporting large numbers of rulesets, as it avoids URL length limits that may occur when using the GET variant with many ruleset UUIDs.

The GET /api/export/v1/ endpoint remains fully supported for backwards compatibility.

POST /api/export/v1/ Parameters

Field Type Required Location Description
export_type string No JSON body The type of data to export (see Export Types for a full list). Defaults to all.
ruleset_ids array[string] No JSON body Array of ruleset UUID strings to export. If omitted, all rulesets are exported.
include_seed_files boolean No JSON body Set to true to include custom seed files referenced by the exported rulesets. Default is false.

POST /api/export/v1/ curl examples

Export specific rulesets by UUID:

curl -X POST "https://<your-datamasque-host>/api/export/v1/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{"ruleset_ids": ["9d641e97-adf7-4f22-9089-afc3711bf222", "04ea20f0-ad4c-498e-881f-b0bc79d83ba7"]}' \
     -J -O

Export specific rulesets with their seed files:

curl -X POST "https://<your-datamasque-host>/api/export/v1/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{"ruleset_ids": ["9d641e97-adf7-4f22-9089-afc3711bf222"], "include_seed_files": true}' \
     -J -O

Importing DataMasque Configuration

A DataMasque export Zip can be imported to a DataMasque install using the /api/import/v1/ API endpoint.

For the best import experience, a Zip that has been exported from DataMasque than contains a manifest.json file should be used. However, a Zip with the correct folder structure may also be created, even if missing manifest.json. DataMasque will import the information, but automatic conflict resolution of duplicate rulesets will not work as well. The difference between inclusion/exclusion of manifest.json is explained below.

Zip Exports From DataMasque With manifest.json

Since Zip exports created by DataMasque include the UUID of each exported item, this can be used to determine which items already exist.

The behaviour when importing rulesets depends on the overwrite_rulesets parameter. When overwrite_rulesets is not specified (default/legacy behaviour):

  • If a ruleset with a given ID exists during import:
    • If ruleset is archived, then it will be restored and its name and content are updated with the imported ruleset.
    • If ruleset is not archived, then no action is taken with that ruleset. The content in the DataMasque instance is unchanged.
  • If a ruleset is found with a matching name, and the contents are identical, then no action is taken. The content in the DataMasque instance is unchanged.
  • If a ruleset is found with a matching name, but the contents are different, then a new ruleset is created by appending Copy to the name. For example, if Ruleset A exists, then the content will be uploaded to a ruleset Ruleset A Copy. An incrementing number will be added until an unused name is found, for example, Copy 2, Copy 3, etc.
  • If no ruleset with the given ID or name exists, then it is created.

When overwrite_rulesets=true:

  • Archived rulesets are restored and updated.
  • Existing rulesets are updated with the imported content.
  • New rulesets are created.

When overwrite_rulesets=false:

  • Archived rulesets are restored and updated.
  • Existing rulesets are skipped (left unchanged).
  • New rulesets are created.

Because of these rules, imports of the same Zip archive may be repeated multiple times without duplicating content.

When libraries are present in the import archive, the overwrite_libraries parameter controls behaviour:

  • When overwrite_libraries=false (default): existing libraries are skipped, new libraries are created.
  • When overwrite_libraries=true: existing libraries are updated, new libraries are created.

Libraries are matched by ID first (from the manifest), then by namespace and name. Archived libraries are always restored regardless of the overwrite setting.

When seed files are present in the import archive, the overwrite_seed_files parameter controls behaviour:

  • When overwrite_seed_files=false (default): existing seed files are skipped, new seed files are created.
  • When overwrite_seed_files=true: existing seed files are updated, new seed files are created.

Zip Exports Created Without manifest.json

A Zip export archive may be created manually, provided the file structure is correct. That is, it matches the structure outlined in Exporting DataMasque Configuration. Without a manifest.json, the ID of rulesets is not known, so matching is done based on the name, using the following rules (when overwrite_rulesets is not specified):

  • If a ruleset is found with a matching name, and the contents are identical, then no action is taken. The content in the DataMasque instance is unchanged.
  • If a ruleset is found with a matching name, but the contents are different, then a new ruleset is created by appending Copy to the name. For example, if Ruleset A exists, then the content will be uploaded to a ruleset Ruleset A Copy. An incrementing number will be added until an unused name is found, for example, Copy 2, Copy 3, etc.
  • If no ruleset with the given name exists, then it is created.

When overwrite_rulesets=true, existing rulesets with a matching name are updated instead of duplicated. When overwrite_rulesets=false, existing rulesets with a matching name are skipped.

Because the imported IDs of rulesets is not known, re-running an import without a manifest.json may result in duplicated rulesets with identical content (when using the default legacy behaviour).

Libraries without a manifest.json are matched by namespace and name only.

POST /api/import/v1/

Authorization: User token only.

Import a DataMasque export Zip file. The response will contain a list of actions taken for each included object.

POST /api/import/v1/ Parameters

Field Type Required Location Description
zip_archive file Yes Form Field The exported Zip archive file.
overwrite_rulesets boolean No Form Field If unset (default), existing rulesets are left unchanged and name collisions create duplicates. If true, existing rulesets will be updated. If false, existing rulesets will be skipped.
overwrite_libraries boolean No Form Field If true, existing libraries will be updated. If false (default), existing libraries will be skipped.
overwrite_seed_files boolean No Form Field If true, existing seed files will be updated. If false (default), existing seed files will be skipped.

POST /api/import/v1/ Responses

The response of an import request contains information about the resources that were imported, grouped by resource type. An example response is shown below.

The default example below shows the response when overwrite_rulesets is not specified (legacy behaviour):

{
  "rulesets": {
    "metadata": {"processed": 6, "created": 2, "restored": 1, "error": 1},
    "data": [
      {
        "exported_name": "Ruleset A",
        "exported_id": "9d641e97-adf7-4f22-9089-afc3711bf222",
        "imported_name": "Ruleset A",
        "imported_id": "9d641e97-adf7-4f22-9089-afc3711bf222",
        "ruleset_type": "database",
        "status": "NOT_CREATED",
        "message": "A ruleset with ID \"9d641e97-adf7-4f22-9089-afc3711bf222\" already exists, and was not changed."
      },
      {
        "exported_name": "Ruleset B",
        "exported_id": null,
        "imported_name": "Ruleset B Copy",
        "imported_id": "04ea20f0-ad4c-498e-881f-b0bc79d83ba7",
        "ruleset_type": "file",
        "status": "CREATED_DUPLICATE",
        "message": "A ruleset named \"Ruleset B\" already exists, so ruleset \"Ruleset B Copy\" was created."
      },
      {
        "exported_name": "Ruleset C",
        "exported_id": null,
        "imported_name": "Ruleset C",
        "imported_id": "7d731d55-68c9-400e-a790-e052afe789cc",
        "ruleset_type": "database",
        "status": "NOT_CREATED",
        "message": "A ruleset named \"Ruleset C\" exists with identical content."
      },
      {
        "exported_name": "Ruleset D",
        "exported_id": null,
        "imported_name": "Ruleset D",
        "imported_id": "99eeffd3-3f65-4ed7-8ad1-a31a539b7b2c",
        "ruleset_type": "file",
        "status": "CREATED",
        "message": "Ruleset named \"Ruleset D\" did not exist, and was created."
      },
      {
        "exported_name": "Ruleset E",
        "exported_id": "c0f5b5bb-a2ce-4cea-9248-1b8ef6539a0e",
        "imported_name": "Ruleset E",
        "imported_id": "c0f5b5bb-a2ce-4cea-9248-1b8ef6539a0e",
        "ruleset_type": "database",
        "status": "RESTORED",
        "message": "An archived ruleset with ID \"c0f5b5bb-a2ce-4cea-9248-1b8ef6539a0e\" has been restored and overwritten with the new name and content."
      },
      {
        "exported_name": "Ruleset F",
        "exported_id": "abc123",
        "imported_name": null,
        "imported_id": null,
        "ruleset_type": "database",
        "status": "ERROR",
        "message": "Import of ruleset with ID \"abc123\" due to error: invalid ID."
      }
    ]
  },
  "libraries": {
    "metadata": {"processed": 2, "created": 1, "restored": 0, "error": 0, "updated": 0, "skipped": 1},
    "data": [
      {
        "exported_name": "pii_masks",
        "exported_namespace": "acme",
        "exported_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
        "imported_name": "pii_masks",
        "imported_namespace": "acme",
        "imported_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
        "status": "CREATED",
        "message": "Library \"acme/pii_masks\" did not exist, and was created."
      },
      {
        "exported_name": "common_masks",
        "exported_namespace": "",
        "exported_id": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
        "imported_name": "common_masks",
        "imported_namespace": "",
        "imported_id": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
        "status": "SKIPPED",
        "message": "A library with ID \"b2c3d4e5-f6a7-8901-bcde-f12345678901\" already exists, and was not changed."
      }
    ]
  },
  "seed_files": {
    "metadata": {"processed": 3, "created": 1, "restored": 0, "error": 0, "updated": 0, "skipped": 2},
    "data": [
      {
        "exported_name": "names.csv",
        "imported_name": "names.csv",
        "status": "CREATED",
        "message": "Seed file \"names.csv\" was created."
      },
      {
        "exported_name": "addresses.csv",
        "imported_name": "addresses.csv",
        "status": "SKIPPED",
        "message": "A seed file named \"addresses.csv\" already exists and was not overwritten."
      },
      {
        "exported_name": "emails.csv",
        "imported_name": "emails.csv",
        "status": "SKIPPED",
        "message": "A seed file named \"emails.csv\" already exists and was not overwritten."
      }
    ]
  }
}

Note: The libraries key is only present in the response when the import archive contains libraries. The seed_files key is only present in the response when the import archive contains seed files.

The metadata for each item type shows the number of items of that type processed, and how many of each one were created, restored, or had an error. For rulesets, the metadata will also include updated and skipped counts when overwrite_rulesets is explicitly set to true or false. The libraries and seed_files metadata always includes updated and skipped counts.

Each data object in rulesets contains information about the import of that ruleset. The fields are:

  • exported_name: The name of the ruleset in the export Zip archive.
  • exported_id: The ID of the ruleset from the export Zip archive. Only available if a manifest.json files is present, otherwise this will be null.
  • imported_name: The name that the ruleset was imported to. Usually this will match exported_name. This will only be null on error. If the ruleset was not imported due to it already existing, this will still match exported_name.
  • imported_id: The ID that the ruleset was imported to. This will be generated if exported_id was null, otherwise it will be expected to match exported_id (even if the data was not changed). imported_id will be null on error.
  • ruleset_type: One of database or file.
  • status: The status of the import of this ruleset. One of:
    • NOT_CREATED: Ruleset was not created due to the ID existing or content being identical.
    • CREATED_DUPLICATE: A ruleset with that name existed, so it was imported with a new name (in imported_name).
    • CREATED: A ruleset with that ID or name did not exist, so was created.
    • RESTORED: An archived ruleset has been restored and overwritten with the new name and content from an imported ruleset.
    • UPDATED: An existing ruleset was updated with new content. Only returned when overwrite_rulesets=true.
    • SKIPPED: An existing ruleset was not modified. Only returned when overwrite_rulesets=false.
    • ERROR: There was an error creating the ruleset. Check message for details.
  • message: A human-readable message describing the action taken or error that occurred. Messages may change between DataMasque versions, so they should not be relied on to determine the outcome of an import. Instead, refer to the status field.

Each data object in libraries contains information about the import of that library. The fields are:

  • exported_name: The name of the library in the export Zip archive.
  • exported_namespace: The namespace of the library in the export Zip archive.
  • exported_id: The ID of the library from the export Zip archive. Only available if a manifest.json file is present, otherwise this will be null.
  • imported_name: The name that the library was imported to. This will only be null on error.
  • imported_namespace: The namespace that the library was imported to. This will only be null on error.
  • imported_id: The ID that the library was imported to. This will be null on error.
  • status: The status of the import of this library. One of:
    • NOT_CREATED: Library was not created due to the ID existing or content being identical.
    • CREATED: A library with that ID or namespace/name did not exist, so was created.
    • RESTORED: An archived library has been restored and overwritten with the new content.
    • UPDATED: An existing library was updated with new content. Only returned when overwrite_libraries=true.
    • SKIPPED: An existing library was not modified. Only returned when overwrite_libraries=false.
    • ERROR: There was an error importing the library. Check message for details.
  • message: A human-readable message describing the action taken or error that occurred.

Each data object in seed_files contains information about the import of that seed file. The fields are:

  • exported_name: The name of the seed file in the export Zip archive.
  • imported_name: The name that the seed file was imported to. Usually this will match exported_name. This will only be null on error.
  • status: The status of the import of this seed file. One of:
    • CREATED: A seed file with that name did not exist, so was created.
    • UPDATED: An existing seed file was updated with new content (when overwrite_seed_files=true).
    • SKIPPED: An existing seed file was not modified because overwrite_seed_files=false.
    • ERROR: There was an error importing the seed file. Check message for details.
  • message: A human-readable message describing the action taken or error that occurred.

The status code of the response, as shown in the table below, gives a quick overview of if any resources were created or not.

Status Code Description
200 The import was successful, indicating either no changes (e.g. the uploaded items already existed) or the successful restoration of archived rulesets or libraries.
201 The import was successful, and one or more rulesets, libraries, or seed files were created.

POST /api/import/v1/ curl example

Basic import (uses legacy behaviour by default: name collisions create duplicates, existing seed files are skipped):

curl -X POST "https://<your-datamasque-host>/api/import/v1/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: multipart/form-data" \
     -F "zip_archive=@</path/to/your/datamasque_export_all_20240211-091507.zip>"

Import without overwriting existing rulesets (skip rulesets that already exist):

curl -X POST "https://<your-datamasque-host>/api/import/v1/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: multipart/form-data" \
     -F "zip_archive=@</path/to/your/datamasque_export_all_20240211-091507.zip>" \
     -F "overwrite_rulesets=false"

Import with seed file overwriting enabled (update existing seed files):

curl -X POST "https://<your-datamasque-host>/api/import/v1/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: multipart/form-data" \
     -F "zip_archive=@</path/to/your/datamasque_export_all_20240211-091507.zip>" \
     -F "overwrite_seed_files=true"

Import with library overwriting enabled (update existing libraries):

curl -X POST "https://<your-datamasque-host>/api/import/v1/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: multipart/form-data" \
     -F "zip_archive=@</path/to/your/datamasque_export_all_20240211-091507.zip>" \
     -F "overwrite_libraries=true"

Import with overwrite enabled for rulesets, libraries, and seed files:

curl -X POST "https://<your-datamasque-host>/api/import/v1/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: multipart/form-data" \
     -F "zip_archive=@</path/to/your/datamasque_export_all_20240211-091507.zip>" \
     -F "overwrite_rulesets=true" \
     -F "overwrite_libraries=true" \
     -F "overwrite_seed_files=true"

Settings

GET /api/settings/

Authorization: User token (Admin users only, API tokens not permitted).

Get the configurable settings for the DataMasque instance, including SMTP, Git, SAML, locality, and AI Engine settings.

GET /api/settings/ Parameters

No parameters.

GET /api/settings/ Responses

Status Code Description
200 A JSON object containing all configurable settings fields.
403 User is not an admin or attempted to use an API token.

GET /api/settings/ Response Fields

Field Type Description
SENDER string Email address used as the sender for SMTP notifications.
EMAIL_HOST string SMTP server hostname.
EMAIL_PORT integer|null SMTP server port (1-65535).
EMAIL_HOST_USER string Username for SMTP authentication.
has_password boolean Indicates whether an SMTP password has been set (password itself is never returned).
EMAIL_TLS_SSL boolean Whether to use TLS/SSL for SMTP connection.
RECEIVE_CC_EMAIL string Email address to receive copies of notifications.
allowed_hosts array List of allowed hostnames for accessing DataMasque.
locality string Locality identifier (letters, numbers, hyphens, and underscores only).
allowed_cidr_nets array List of allowed CIDR networks for IP-based access control.
saml_idp_metadata_file string Path to the SAML IdP metadata file.
disable_local_logins boolean Whether local username/password logins are disabled (requires SAML SSO configured).
sdd_customised_keywords array List of custom keywords for sensitive data discovery.
sdd_ignored_keywords array List of keywords to ignore during sensitive data discovery.
git_repository_url string URL of the Git repository for ruleset version control.
git_branch string Git branch to use for ruleset storage.
git_directory_path string Directory path within the Git repository for rulesets.
dm_ai_engine_url string The configured DataMasque AI Engine URL for unstructured data masking. Empty string if not configured.

GET /api/settings/ curl example

curl "https://<your-datamasque-host>/api/settings/" \
     -H "Authorization: Token <your-user-token>"

PATCH /api/settings/

Authorization: User token (Admin users only, API tokens not permitted).

Update configurable settings for the DataMasque instance. All fields are optional (partial update supported).

PATCH /api/settings/ Parameters

Field Type Required Location Description
SENDER string No Request Body Email address used as the sender for SMTP notifications.
EMAIL_HOST string No Request Body SMTP server hostname.
EMAIL_PORT integer|null No Request Body SMTP server port (1-65535).
EMAIL_HOST_USER string No Request Body Username for SMTP authentication.
EMAIL_HOST_PASSWORD string No Request Body Password for SMTP authentication (write-only, never returned in responses).
EMAIL_TLS_SSL boolean No Request Body Whether to use TLS/SSL for SMTP connection.
RECEIVE_CC_EMAIL string No Request Body Email address to receive copies of notifications.
allowed_hosts array No Request Body List of allowed hostnames for accessing DataMasque (at least one required if provided).
locality string No Request Body Locality identifier (letters, numbers, hyphens, and underscores only, max 100 characters).
allowed_cidr_nets array No Request Body List of allowed CIDR networks for IP-based access control.
saml_idp_metadata_file file No Multipart SAML IdP metadata XML file upload.
disable_local_logins boolean No Request Body Whether to disable local username/password logins (requires SAML SSO configured).
sdd_customised_keywords array No Request Body List of custom keywords for sensitive data discovery (newline-separated).
sdd_ignored_keywords array No Request Body List of keywords to ignore during sensitive data discovery (newline-separated).
git_repository_url string No Request Body URL of the Git repository for ruleset version control.
git_branch string No Request Body Git branch to use for ruleset storage.
git_directory_path string No Request Body Directory path within the Git repository for rulesets.
dm_ai_engine_url string No Request Body The configured DataMasque AI Engine URL for unstructured data masking. Empty string if not configured.

For example, it is possible to set the DataMasque AI Engine URL and port like so:

docker exec -ti datamasque-admin-server-1 sh -c 'bash django_manage.sh set_config DM_AI_ENGINE_URL "http://myurl.com:1234"'

PATCH /api/settings/ Responses

Status Code Description
200 Settings updated successfully. Returns the updated settings object.
400 Validation error (e.g., invalid email format, invalid CIDR network, etc.).
403 User is not an admin or attempted to use an API token.

PATCH /api/settings/ Response Fields

The response includes all the same fields as the GET endpoint (see GET /api/settings/ Response Fields), plus:

Field Type Description
gitError string Optional. Present only if Git repository URL validation failed. Contains the error message.

PATCH /api/settings/ curl example

curl -X PATCH "https://<your-datamasque-host>/api/settings/" \
     -H "Authorization: Token <your-user-token>" \
     -H "Content-Type: application/json" \
     -d '{
       "EMAIL_HOST": "smtp.example.com",
       "EMAIL_PORT": 587,
       "EMAIL_TLS_SSL": true,
       "locality": "us-west"
     }'

Other API Requests

POST /api/users/admin-install/

Authorization: Anonymous, Only when no user has been created.

Verify the DataMasque installation, and set up an admin account.

POST /api/users/admin-install/ Parameters

Field Type Required Location Description
email string Yes Request Body The email of the user you are logging in as.
username string Yes Request Body The username of the user you are logging in as.
password string Yes Request Body The password for the user.
re_password string Yes Request Body The password for the user again, to confirm the password entered above.
allowed_hosts array[string] Yes Request Body A list of hostnames, IP addresses or CIDR networks that will be allowed to access DataMasque upon installation.
contract_license_type string Required only for AWS Contract Product installations. Request Body For contract products, the type of product to check out. Must be either business_db or enterprise.

POST /api/users/admin-install/ Responses

Status Code Description
201 A JSON serialised User object, with an extra warnings* item.

* Any non-critical warnings that were generated during installation are included in the warnings item of the response. This is an array of strings.

POST /api/users/admin-install/ curl example

curl -X POST "https://<your-datamasque-host>/api/users/admin-install/" \
     -H "Authorization: Token <your-api-token>" \
     -d '{
           "email": "<your-admin-email>",
           "username": "<your-username>",
           "password": "<your-admin-password>",
           "re_password": "<your-admin-password>",
           "allowed_hosts": ["masque.local"]
         }'

Installation Info Object

A JSON object showing the state of the current installation with the following data:

Field Type Description
is_aws_marketplace boolean Whether the current installation has been installed from the AWS marketplace.
installed boolean If the current installation has been successfully installed.
is_smtp_configured boolean If SMTP has been configured on the DataMasque instance.
is_saml_sso_configured boolean Is SSO has been enabled on the DataMasque instance.

GET /api/app/check/

Authorization: User token or API token.

Checks to verify if DataMasque has successfully been installed.

GET /api/app/check/ Parameters

No parameters.

GET /api/app/check/ Response

Code 200

Description:

Status Code Description
200 A JSON serialised Installation Info Object object.

GET /api/app/check/ curl example

curl "https://<your-datamasque-host>/api/app/check/" \
     -H "Authorization: Token <your-api-token>"

POST /api/license-upload/

Authorization: Admin user token only.

Uploads a license file to DataMasque.

POST /api/license-upload/ Parameters

No parameters.

POST /api/license-upload/ Responses

Status Code Description
200 Operation succeeded.

POST /api/license-upload/ curl example

curl -X POST "https://<your-datamasque-host>/api/license-upload/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -F "license_file=@</path/to/your/license_file.lic>"

License Object

Fields of the current license:

Field Type Description
version string The version of the license.
name string true if the masking agent worker processes are healthy, false if there are no available workers.
type string The type of the current license. Must be one of: Free Trial, Premium Free Trial, Consumption, Business DB, Business File, Enterprise, Dynamic, or Restricted
raw_type string The enum type of the current license. Must be one of: FREE_TRIAL, PREMIUM_FREE_TRIAL, CONSUMPTION, BUSINESS_DB, BUSINESS_FILE, ENTERPRISE, DYNAMIC, or RESTRICTED
expiry_date datetime The date when the license expires.
quota_tb float The amount of data that can be masked with the current license (in terabytes). null means unlimited.
maximum_node_count int The amount of EKS nodes that can be concurrently running (if applicable).
row_limit int The maximum amount of rows that can be masked in any given masking run.
platform_name string The name which the license is valid for. Must be one of: AWS, AWS Contract, AWS BYOL, Azure, Azure BYOL, Docker, Cohesity
platform_code string The marketplace code for the current platform (if applicable).
is_expired boolean true if the license has expired.
days_until_expiry int Days until the license expires (if applicable).
uuid string Unique identifier for the current license.
uploadable boolean true if the license can be uploaded, either via API or UI.
is_contract_product boolean true if the license is Dynamic.
contract_license_type string The type of Dynamic license. Must be one of business_db or enterprise.
switchable_license_metadata JSON Structure See Switchable License Metadata Object for details.

Switchable License Metadata Object:

Field Type Description
can_switch_license_source boolean true if the license source can be switched between file and cloud.
license_source string The source of the licensing information. Either file for the licensing file or cloud for Dynamic license.
license_select_time date Date when the license source was switched to cloud (ISO Format). null if using file as the license source.
aws_account_number string The AWS account number associated with the license.
last_checkout_success_time date Date of the last successful checkout (ISO Format).
last_checkout_success_type string The license type of the last successful checkout. Must be either business_db, or enterprise.
last_checkout_error string The exception message if there was an exception raised during the last checkout.
last_checkout_license_arn string AWS License Manager ARN from the last successful checkout. null if not yet checked out.
last_checkout_product_name string Product name from the last successful checkout. null if license-manager:GetLicense permission not granted.
last_checkout_contract_expiry date Contract end date from the last successful checkout (ISO Format). This is the subscription expiry, not the 1-hour checkout token expiry. null if license-manager:GetLicense permission not granted.
last_checkout_agreement_id string AWS Marketplace Agreement ID from the last successful checkout. null if aws-marketplace:SearchAgreements permission not granted or no matching agreement found.
last_checkout_agreement_url string Partition-aware AWS Marketplace console URL for the agreement. null if the agreement ID is unavailable.
checkout_mode string The checkout mode for Dynamic licenses: automatic (default), specific (use a specific SKU), or disabled (checkout disabled — masking runs blocked until re-enabled).
selected_product_sku string The specific AWS Marketplace product SKU to use when checkout_mode is specific. null when not set or when checkout_mode is automatic.
allow_fallback boolean When checkout_mode is specific, whether to fall back to other available SKUs if the selected SKU cannot be checked out. Defaults to true.
last_checkout_success_license_count integer Number of license units checked out in the last successful checkout. null if not yet checked out.
iam_role_arn string IAM role ARN DataMasque assumes before calling AWS License Manager APIs. Used for cross-account setups where the license is in a different AWS account from the DataMasque deployment. null if not configured (default credentials are used).

GET /api/license/

Authorization: Authenticated user.

Retrieve information about the current license.

GET /api/license/ Parameters

No parameters.

GET /api/license/ Responses

Status Code Description
200 License information is returned.
400 License is invalid. Response includes message and license_upload_allowed (boolean).
401 Authentication credentials were not provided or are invalid.
404 No license file has been uploaded.

An example response is shown below.

{
    "name": "DataMasque Limited",
    "version": "4.0",
    "type": "Consumption",
    "raw_type": "CONSUMPTION",
    "platform_name": "Docker",
    "marketplace_product_code": null,
    "row_limit": 0,
    "expiry_date": "2026-07-22",
    "days_until_expiry": 157,
    "uuid": "3e90a512-236b-498f-bc11-ee6b4eed6d41",
    "is_expired": false,
    "quota_tb": "1",
    "uploadable": true,
    "is_contract_product": false,
    "contract_license_type": null,
    "max_service_count": {
        "in-flight": null,
        "agent": null
    },
    "switchable_license_metadata": {
        "can_switch_license_source": true,
        "license_source": "cloud",
        "license_select_time": "2026-02-10T08:21:47.290186Z",
        "aws_account_number": "123456789012",
        "last_checkout_success_time": "2026-02-10T09:00:12.451230Z",
        "last_checkout_success_type": "business_db",
        "last_checkout_success_license_count": 2,
        "last_checkout_error": null,
        "checkout_mode": "automatic",
        "selected_product_sku": null,
        "allow_fallback": true,
        "iam_role_arn": null
    }
}

GET /api/license/ curl example

curl "https://<your-datamasque-host>/api/license/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json"

GET /api/license/summary/

Authorization: Authenticated user.

Retrieve a summary of the current license state, including license details, expiry information, contract metadata, and capabilities (allowed run types).

GET /api/license/summary/ Parameters

No parameters.

GET /api/license/summary/ Responses

Status Code Description
200 License summary is returned.
400 License is invalid. Response includes message and license_upload_allowed (boolean).
401 Authentication credentials were not provided or are invalid.

GET /api/license/summary/ Response Fields

Field Type Description
license_type string The effective license type. null if no license is present.
license_source string The source of the license: file or cloud. null if no license is present.
is_dynamic boolean true if the license is a Dynamic license.
is_contract_product boolean true if the license is a cloud contract product (Dynamic with business_db or enterprise type).
can_switch_license_source boolean true if the license source can be switched between file and cloud.
checkout_mode string The checkout mode for Dynamic licenses: automatic (default), specific (use a specific SKU), or disabled (checkout disabled — masking runs blocked until re-enabled).
expiry_date string The license expiry date (YYYY-MM-DD format). null if no expiry.
days_until_expiry integer Days until the license expires. null if no expiry.
is_expired boolean true if the license has expired.
is_license_missing boolean true if no valid license file exists.
license_ui_mode string Which license UI to display: file_only, switchable, or contract_checkout.
contract object Contract metadata from SwitchableLicenseInfo. null if not available. See Switchable License Metadata Object.
license object Full license details (same structure as GET /api/license/ response). null if no license.
capabilities object License capabilities. Contains allowed_run_types: a list of run types the license permits (database, file, file_data_discovery).

An example response is shown below.

{
    "license_type": "Enterprise",
    "license_source": "cloud",
    "is_dynamic": true,
    "is_contract_product": true,
    "can_switch_license_source": true,
    "checkout_mode": "automatic",
    "expiry_date": "2026-12-31",
    "days_until_expiry": 320,
    "is_expired": false,
    "is_license_missing": false,
    "license_ui_mode": "contract_checkout",
    "contract": {
        "license_type": "enterprise",
        "checkout_mode": "automatic",
        "selected_product_sku": null,
        "allow_fallback": true,
        "iam_role_arn": null,
        "license_select_time": "2026-01-15T08:21:47.290186Z",
        "aws_account_number": "123456789012",
        "last_checkout_success_time": "2026-02-13T14:32:05.812340Z",
        "last_checkout_success_type": "enterprise",
        "last_checkout_success_license_count": 2,
        "last_checkout_error": null,
        "last_checkout_license_arn": "arn:aws:license-manager::123456789012:license:l-abc1234567890",
        "last_checkout_product_name": "DataMasque - Contract Plan",
        "last_checkout_contract_expiry": "2027-01-15T00:00:00Z",
        "last_checkout_agreement_id": "agmt-abc1234567890",
        "last_checkout_agreement_url": "https://console.aws.amazon.com/marketplace/home#/agreements/agmt-abc1234567890"
    },
    "license": {
        "name": "DataMasque Limited",
        "version": "4.0",
        "type": "Dynamic",
        "raw_type": "DYNAMIC",
        "platform_name": "Docker",
        "marketplace_product_code": null,
        "row_limit": 0,
        "expiry_date": "2026-12-31",
        "days_until_expiry": 320,
        "uuid": "3e90a512-236b-498f-bc11-ee6b4eed6d41",
        "is_expired": false,
        "quota_tb": "1",
        "uploadable": false,
        "is_contract_product": true,
        "contract_license_type": "enterprise",
        "max_service_count": {
            "in-flight": null,
            "agent": null
        },
        "switchable_license_metadata": {
            "can_switch_license_source": true,
            "license_source": "cloud",
            "license_select_time": "2026-01-15T08:21:47.290186Z",
            "aws_account_number": "123456789012",
            "last_checkout_success_time": "2026-02-13T14:32:05.812340Z",
            "last_checkout_success_type": "enterprise",
            "last_checkout_success_license_count": 2,
            "last_checkout_error": null,
            "checkout_mode": "automatic",
            "selected_product_sku": null,
            "allow_fallback": true,
            "iam_role_arn": null
        }
    },
    "capabilities": {
        "allowed_run_types": ["database", "file", "file_data_discovery"]
    }
}

GET /api/license/summary/ curl example

curl "https://<your-datamasque-host>/api/license/summary/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json"

GET /api/license/contract-type/

Authorization: User token only.

For Dynamic licenses, retrieve the type of license that has been configured to be used.

GET /api/license/contract-type/ Parameters

No parameters.

GET /api/license/contract-type/ Responses

Status Code Description
200 License type retrieved.
400 The licensing method is not of Dynamic type, so setting the license type is not supported.
404 The license type has not yet been specified.

An example response is shown below.

{
  "license_source": "cloud",
  "contract_license_type": "business_db"
}

license_source must be one of:

  • cloud
  • file

contract_license_type must be one of:

  • business_db
  • enterprise

GET /api/license/contract-type/ curl example

curl "https://<your-datamasque-host>/api/license/contract-type/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json"

PUT /api/license/contract-type/

Authorization: Admin User token only.

For Dynamic Offer licenses, set the type of license to check out.

PUT /api/license/contract-type/ Parameters

Field Type Required Location Description
contract_license_type string No Request Body The type of license to check out. Must be one of business_db, business_file, or enterprise.
license_source string No Request Body The source of the license. Must be one of file or cloud.
checkout_mode string No Request Body The checkout mode. Must be one of automatic (default), specific, or disabled. When set to disabled, all active licenses are checked in and masking runs are blocked until re-enabled.
selected_product_sku string No Request Body The specific AWS Marketplace product SKU to check out. Required when checkout_mode is specific. When set, DataMasque will prefer this SKU over others.
allow_fallback boolean No Request Body When checkout_mode is specific, whether to fall back to other available SKUs if the selected SKU cannot be checked out. Defaults to true.
iam_role_arn string No Request Body IAM role ARN for DataMasque to assume before calling AWS License Manager APIs. Used when the DataMasque license is purchased in a different AWS account from where DataMasque is deployed. Pass null or "" to clear.

PUT /api/license/contract-type/ Responses

Status Code Description
201 License type updated.
400 The licensing method is not of Dynamic type, so setting the license type is not supported, or the specified license type is invalid.

PUT /api/license/contract-type/ curl example

# Switch to Dynamic license (business_db type) with automatic checkout
curl -X PUT "https://<your-datamasque-host>/api/license/contract-type/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{"contract_license_type": "business_db", "license_source": "cloud", "checkout_mode": "automatic"}'

# Disable checkout while remaining on Dynamic license
curl -X PUT "https://<your-datamasque-host>/api/license/contract-type/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{"checkout_mode": "disabled"}'

# Use a specific product SKU with fallback enabled
curl -X PUT "https://<your-datamasque-host>/api/license/contract-type/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{"checkout_mode": "specific", "selected_product_sku": "prod-abc1234567890", "allow_fallback": true}'

# Configure a cross-account IAM role for license operations
curl -X PUT "https://<your-datamasque-host>/api/license/contract-type/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{"iam_role_arn": "arn:aws:iam::123456789012:role/CrossAccountLicenseRole"}'

# Clear a previously configured IAM role (revert to default credentials)
curl -X PUT "https://<your-datamasque-host>/api/license/contract-type/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json" \
     -d '{"iam_role_arn": null}'

PUT /api/license/contract-type/ Example response

This shows a successful license type update to a Dynamic Business DB license with checkout disabled.

{
  "message": "License type updated successfully"
}

Error responses return a JSON object with an error field containing the error message, such as "The licensing method is not of Dynamic type" or "Error checking in license: <details>".

GET /api/license/checkouts/

Authorization: User token only.

Shows the current license checkouts from AWS.

GET /api/license/checkouts/ Parameters

No parameters.

GET /api/license/checkouts/ Responses

Status Code Description
200 License history is returned.
400 The licensing method is not of Dynamic type, so setting the license cannot be checked out.

An example response is shown below.

[
  {
    "checkout_type": "PROVISIONAL",
    "license_consumption_token": "eyJhbGciOiJIUzI1NiIsInR5cCI...",
    "entitlements_allowed": [
      {
        "name": "enterprise_lic",
        "unit": "Count",
        "value": "1"
      }
    ],
    "issued_at": "2026-02-13T14:32:05Z",
    "expiration": "2026-02-13T15:32:05Z",
    "license_arn": "arn:aws:license-manager::123456789012:license:l-abc1234567890",
    "node_id": "i-0abcdef1234567890",
    "product_name": "DataMasque Synthetically Identical Masked Data - Contract Plan",
    "contract_expiry": "2027-01-15T00:00:00Z",
    "agreement_id": "agmt-abc1234567890",
    "agreement_url": "https://console.aws.amazon.com/marketplace/home#/agreements/agmt-abc1234567890"
  }
]

Note: expiration is the ~1 hour checkout token expiry, not the contract end date. contract_expiry is the actual subscription end date. product_name, contract_expiry, agreement_id, and agreement_url are null if the corresponding IAM permissions (license-manager:GetLicense and aws-marketplace:SearchAgreements) are not granted. See AWS permissions in the Dynamic Licenses section for details.

GET /api/license/checkouts/ curl example

curl "https://<your-datamasque-host>/api/license/checkouts/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json"

GET /api/license/aws-discovery/

Authorization: User token only.

Discovers DataMasque licenses available in AWS License Manager for the configured AWS account.

This endpoint queries AWS License Manager to find all DataMasque product licenses, enriches them with product names and usage information, and returns consolidated license and entitlement details.

GET /api/license/aws-discovery/ Parameters

No parameters.

GET /api/license/aws-discovery/ Responses

Status Code Description
200 Request completed. Check discovery_error field in response body for any errors that occurred during license discovery.

Note: The endpoint always returns HTTP 200. Discovery errors are reported in the discovery_error field of the response body.

Cross-account discovery: If iam_role_arn is configured via PUT /api/license/contract-type/, this endpoint will assume that role before querying AWS License Manager. This allows discovery of licenses held in a different AWS account from where DataMasque is deployed. If the configured role ARN is invalid or the assume-role call fails, the error is reported in discovery_error with no fallback to default credentials.

Response Format

The response contains:

  • products: Array of DataMasque license products found in AWS License Manager
  • discovery_error: Error message if discovery failed, otherwise null

Each product contains:

  • product_sku: AWS Marketplace product SKU (UUID)
  • license_arn: AWS License Manager ARN for the license
  • product_name: Human-readable product name (e.g., "DataMasque Virtually Identical Masked Data")
  • status: License status (e.g., AVAILABLE, SUSPENDED, EXPIRED)
  • validity_end: ISO 8601 timestamp when the license expires, or null if not available
  • entitlements: Array of entitlement objects with:
    • name: Entitlement name (e.g., enterprise_lic, ai_workloads)
    • max_count: Maximum count for this entitlement, or null for unlimited
    • consumed: Number of units currently consumed, or null if usage data unavailable
    • available: Number of units available (max_count - consumed), or null if usage data unavailable

AWS Permissions Required

The IAM role or instance profile must have the following permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "license-manager:ListReceivedLicenses",
        "license-manager:GetLicense",
        "license-manager:GetLicenseUsage"
      ],
      "Resource": "*"
    }
  ]
}

Note: GetLicense and GetLicenseUsage are optional - discovery will succeed without them but will not include product names or consumed/available counts.

GET /api/license/aws-discovery/ Example Response

{
  "products": [
    {
      "product_sku": "e76ff3e0-aa88-4c66-bb1b-1e6185a6b7dd",
      "license_arn": "arn:aws:license-manager::123456789012:license:l-abc123",
      "product_name": "DataMasque Virtually Identical Masked Data",
      "status": "AVAILABLE",
      "validity_end": "2027-01-01T00:00:00Z",
      "entitlements": [
        {
          "name": "enterprise_lic",
          "max_count": 5,
          "consumed": 2,
          "available": 3
        },
        {
          "name": "ai_workloads",
          "max_count": null,
          "consumed": null,
          "available": null
        }
      ]
    }
  ],
  "discovery_error": null
}

If discovery fails, the response will have an empty products array and a non-null discovery_error:

{
  "products": [],
  "discovery_error": "License discovery failed. Ensure IAM permissions include license-manager:ListReceivedLicenses. Error: AccessDeniedException"
}

GET /api/license/aws-discovery/ curl example

curl "https://<your-datamasque-host>/api/license/aws-discovery/" \
     -H "Authorization: Token <your-api-token>" \
     -H "Content-Type: application/json"

Health Check Object

Various health statistics about the DataMasque instance:

Field Type Description
worker_running boolean true if the masking agent worker processes are healthy, false if there are no available workers.
license_expired boolean true if the license is expired, false if the license is not expired.
license_renewal_in_days integer Remaining days until license expiry.
license_limit_breach object An object describing any license breaches that have occurred. Each property on the object is the type of breach that has occurred. Each property value is an object containing breach_type, message, and created_date properties.

GET /api/health-check/

Authorization: User token or API token.

Get the basic health-check status of DataMasque.

GET /api/health-check/ Parameters

No parameters.

GET /api/health-check/ Responses

Status Code Description
200 A JSON serialised Health Check Object.
500 A server error has occurred, such as an invalid license file exists. The known error will be returned.

GET /api/health-check/ curl example

curl "https://<your-datamasque-host>/api/health-check/" \
     -H "Authorization: Token <your-api-token>"