Data Retention Policies
The Informatics Core Services (ICS) follows a structured approach to data retention, archiving, and deletion to ensure compliance, manage costs, and support research needs. This policy outlines when and why data is archived or deleted, and describes the use of RIS for long-term cold storage.
Data Lifecycle and Retention
Data assets in the WUSM Data Lake progress through several lifecycle stages: development, review, promotion, maintenance, and removal. During the maintenance phase, ICS and data stewards periodically review assets to determine if they are still needed, properly tagged, and compliant with institutional and regulatory requirements.
Archiving Data
- When is data archived?
- Data is archived when it is no longer actively used but must be retained for compliance, audit, or institutional policy reasons (e.g., data from completed studies, superseded datasets, or assets required for record-keeping).
- Archiving is also used to manage storage costs by moving infrequently accessed data out of high-cost environments.
- How is data archived?
- Archived data is moved from the active data lake environment to RIS (Research Infrastructure Services) cold storage. RIS provides a secure, scalable, and cost-effective platform for long-term data retention.
- The process for transferring data to RIS is documented in Transferring Cloud Data To and From RIS Storage.
Data Deletion
- When is data deleted?
- Data is deleted when retention requirements have been met, and there is no further business, legal, or compliance need to retain the asset.
- Examples include data from projects that have reached the end of their required retention period, or assets that have been superseded and are no longer referenced.
- How is data deleted?
- Deletion is performed in accordance with institutional and regulatory guidelines. All deletions are logged for auditing purposes.
- In some cases, data may first be archived to RIS before final deletion from the data lake.
Special Considerations
- Retention Periods: Retention periods are determined by institutional policy, regulatory requirements, and the needs of the data owner. See Research Data Record Retention Requirements for more information.
- Cold Storage: RIS cold storage is the designated location for long-term archival of ICS data. This ensures data is preserved securely and can be retrieved if needed for compliance or future research.
- Auditing: All archiving and deletion actions are logged. Regular audits are conducted to ensure compliance with retention policies.