DataMasque Portal

Azure Cosmos DB connections

DataMasque supports Azure Cosmos DB for MongoDB request unit (RU) accounts, whose hostnames end in .mongo.cosmos.azure.com. Cosmos DB for MongoDB vCore accounts are not supported, and neither are Cosmos DB's other APIs (NoSQL, Cassandra, Gremlin and Table), which are unrelated wire protocols.

Required privileges

A DataMasque connection for Cosmos DB requires the account name and one of its read-write keys. A read-only key connects and discovers, but cannot write masked values back. Both are on the account's Connection strings blade in the Azure portal.

Connection parameters

Cosmos DB connections use the same parameters as MongoDB connections, with the following considerations:

  • Host: The account host, <account>.mongo.cosmos.azure.com.
  • Port: The Cosmos DB port (default: 10255).
  • User / Password: The account name, and one of its read-write keys.
  • Enable TLS/SSL: On by default, and needs to stay on. Cosmos DB only accepts TLS connections. Its certificate chains to a public certificate authority, so no CA certificate needs to be uploaded.

Retryable writes are turned off for a Cosmos DB connection, and need to stay off. Cosmos DB rejects retryable inserts while accepting retryable updates, and masking only updates, so turning them back on does not fail the run. What it loses is the run's entry in the DATAMASQUE_RUN_HISTORY collection, which is written as an insert: Cosmos DB rejects it, DataMasque records the failure in the agent log and the run otherwise completes normally.

Sorting on fields other than _id

A Cosmos DB account created with the default indexing policy indexes only the _id field, and rejects a query that sorts on an unindexed field with The index path corresponding to the specified order-by item is excluded.

Masking sorts each batch by _id, so this does not arise on a default run. It does arise for a run started through the API with the order_batch run option, which makes each batch read in the order of the masking task's key columns instead of _id. Index those fields before such a run:

db.<collection>.createIndex({ <field>: 1 })

Sharded collections

A Cosmos DB collection larger than the unsharded storage limit has to be sharded on a shard key. DataMasque masks a sharded collection with no extra configuration and no mention of the shard key in the ruleset: it updates each document by _id, and Cosmos DB routes that to the right partition itself.

Schema discovery samples the oldest documents

Cosmos DB's $sample aggregation stage returns the first documents in a collection by _id rather than a random selection. Schema discovery builds a collection's field list from that sample, so on Cosmos it only ever inspects one end of the collection. Where _id is left to the driver to generate, that is the oldest documents, because a generated ObjectId increases over time.

A field that appears only in documents at the other end of that order, in the common case the most recently inserted ones, is therefore not discovered. It is not reported, not profiled by in-data discovery, and will not be masked unless the ruleset names it explicitly.

In-data discovery is not restricted to the oldest documents in the same way, because it reads its own sample rather than using $sample. It can only profile fields that schema discovery already found, though, so a field missed above is missed here too.

Masking itself is unaffected: a masking run pages through the whole collection and does not sample. When running discovery against Cosmos, review the generated ruleset against the fields you expect rather than relying on discovery alone.

For more information, see Microsoft's documentation: https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/.

Encrypted connections

Cosmos DB uses the same encrypted connection settings as MongoDB. See MongoDB connections -> Encrypted connections.