Secure Personally Identifiable Information
Information security has been a constant challenge for enterprises. Especially in a software test environment, enterprises face the threat of exposing the personally identifiable information (PII) of customers and commercially sensitive data (financial, operational, and strategic) to the test teams (external vendors).
Test data management (TDM) is one of the effective ways to tauten data security. It helps businesses to improve the security of sensitive information and prevent their exposure to the test environment. Some of the many features offered by TDM tools are:
- Data profiling
- Data sub-setting
- Data masking
- Synthetic test data creation
- Test data repository
These functionalities not only improve information security but also provide features such as on-demand test data and reusability of test data. A typical TDM implementation will include source data, TDM tool, and test environment. The standard source data is connected to a TDM tool to provide maximum security and filter out sensitive data before it is exposed to the test environment.
Challenges of Test data management (TDM) Tools
Although TDM is an effective method to manage data security, there are still certain issues that need to be addressed to enhance data security and process agility. Some of them are:
Integration with source system: A lot of source systems do not use any industry standard databases (DBs) like Oracle, SQL etc. Accessing such No-SQL DBs and understanding the table structure, relationship etc. is a challenge for the TDM tool. (For e.g. Salesforce has NoSQL DB and TDM tools face integration challenges with Salesforce).
Sensitive data exposure: TDMs provide data security to a certain level. However, sensitive data (Customer PII information, financial information, strategic moves etc.) is still exposed to the TDM users/teams.
Delays in test data creation: TDM can create synthetic data rather quickly. However, extracting huge data from production and processing it is often time consuming.
Adding a data security layer to enhance security and efficiency in TDM
In order to overcome the challenges discussed above, it is recommended to add a data security layer on top of the TDM tool. This data security layer should be custom built with data security & integration agent and automated data loader to enhance the security and performance.
Data security and integration agent
One of the effective ways to enhance the security and ease the integration challenges is to introduce an agent between the source and TDM tools. This agent provides quicker integration APIs to integrate a TDM system with any of the source systems, irrespective of the database management systems.
Data security and integration agent analyzes the source database schema, tables, and relationships and create tables, triggers, and sequences in accordance with the TDM tool’s DB structure.
These agents not only solve integration issues but also provide an additional layer of security through automated data-masking scripts that minimize data exposure.
Data security and integration agent should be capable of extracting only relevant and minimum possible data (tables and relationship) from the source thus enabling faster processing. Agents also provide configurations for sensitive data/personally identifiable information (PII), data access and masking policies.
Automated data loader
Automated scripts play a crucial role in enhancing data security and expediting the extraction process. Automated scripts securely extract the data such as a table, relationship, and schema information from the source system without manual intervention and load these data from source to TDM and further from TDM to test environment in a secured (password-protected) and effective way. This helps avoid exposing sensitive personally identifiable information (PII) to test teams.
Benefits of data security layer on TDM:
- PII compliance, as no production data is exposed to testing or TDM environment
- Increase in productivity through agent-based slicing
- Faster data extraction through automated data loading scripts
- Faster data masking through an agent and automated scripts
- Customized test data management can help enterprises secure their data in a test environment and generate test data covering all the scenarios in less time
- Test data repository enables enterprises to access test data on demand
By Jagadeesh G, Technical Lead, Prodapt
Jagadeesh has 12+ years of automation testing and test lifecycle management experience in integrating complex technologies such as Salesforce Cloud, Netezza DWH and Hadoop big data for test data management.