Regular expressions

• Common regular expression patterns
• Example patterns
  • Phone Number
  • Social Security Number
  • Time
  • Email Address

Common regular expression patterns

A regular expression (or "regex") is a sequence of characters that acts as a search pattern to filter or select substrings of text strings. They can be used in replace_regex masks and matches conditions of if and skip blocks. Note that it is best practice to wrap the regular expression in quotes to avoid special characters being misinterpreted as YAML syntax:

version: '1.0'
tasks:
  - type: mask_table
    table: driversLicence
    key: ID
    rules:
      - column: driversLicence
        masks:
          - type: replace_regex
            regex: "[0-9]"
            masks:
              - type: from_fixed
                value: "#"

Notes: DataMasque uses the Python regular expression syntax, which you can learn more about in the official tutorial.

Regular expressions allow you to match various characters by specifying a character or character set. Below are a list of characters sets that can be used in regular expressions.

Characters can be specified as character classes.

• "[0-9]" can be used to match any numerical character.
• "[a-z]" and "[A-Z]" will match lower case and upper case alphabetical characters respectively.
• Classes can also be combined; "[a-zA-Z]" will match any upper or lower case character.
• You may also specify a list of characters to match within square brackets. If you wish to only match vowels, you can use "[aeiouAEIOU]".
• If a caret character is added before any character within the character class, it will instead match something other than the specified character. "^[0-9]" will match any character other than a numerical character.

The table below detail other ways to denote certain character type without the use of character classes.

Character	Description
\d	One digit character, ranging from 0 to 9.
\w	One 'word' character; an ascii letter, a digit or an underscore.
\s	One whitespace character. This can be a space, a line break, or a tab.
\D	One character that is NOT a digit.
\W	One character that is NOT a word character.
\S	One character that is NOT a whitespace character.

Note that when used in double-quoted strings in YAML, literal backslashes must be escaped by backslashes, for example, "\\d" will be interpreted as \d. We recommend quoting YAML strings with single quotes to avoid the need for this escaping, as '\d' will be interpreted as \d.

You can also choose to add a quantifier to any character or character class, which will specify how many characters will be matched.

Quantifier	Description
+	One or more characters will be matched.
{n}	Exactly n characters will be matched. For example, \d{3} will match strings of exactly 3 numerical digits.
{a,b}	Any string of characters between x and y length will be matched. For example, \w{1,3} will match a string of word characters between 1 and 3 length.
{c,}	Any string of characters with c or more characters. For example, \d{2,} will match a string of numerals of length 2 or more.
*	Any instance of the characters appearing zero or more times.
?	Matches if the character or character class appears once, or no times.

Below are some common regular expression patterns that can be used to match typical database strings.

Example patterns

Phone Number

The pattern below will match a phone number consisting of an international code in parentheses followed by a 7-digit phone number that may be hyphenated after the 3rd digit.

'\(\+\d{1,4}\)\s?\d{3}-?\d{4}'

Some example of matching patterns are shown below:

• (+64)123-4567
• (+1234) 5678910
• (+61) 987-5432

Social Security Number

The pattern below will match a Social Security Number consisting of hyphen-separated digits:

'\d{3}-\d{2}-\d{4}'

Some example of matching patterns are shown below:

• 111-22-3333
• 987-65-4321
• 112-35-8132

Time

The pattern below will match a time consisting of 1 or 2 hour digits and 2 minute digits separated by a colon:

'[0-2]?\d:\d{2}'

Some example of matching patterns are shown below:

• 7:30
• 23:50
• 05:45

Email Address

The pattern below will match an email address consisting of an alphanumeric username and an alphanumeric domain name separated by an @ symbol. For a more comprehensive email regular expression, see: emailregex.com.

'[a-zA-Z0-9]+@[a-zA-Z0-9\.]+'

Some example of matching patterns are shown below:

• janedoe@gmail.com
• bobsmith@hotmail.com
• thomas@yahoo.com

DISCLAIMER: The above examples are simplistic and general examples to demonstrate how regular expressions can be constructed - you should ensure these regular expressions are appropriate for your particular data before using them in your masking rulesets.