Access Historical Data

Databricks provides the ability to look at previous versions of data within a given table. This can be helpful if the record has been removed from a table or the information has been updated and you need to find a previous value.

Below is some example code showing how to find historical data in the REDCap data table.

-- See changes to table

desc history cleansed.redcap_prod.redcap_appp60m1_redcap_data;

-- Pull specific version

select * from cleansed.redcap_prod.redcap_appp60m1_redcap_data 
version as of 10
where project_id = 123
and record = 'example1';

-- Or Range

select * from cleansed.redcap_prod.redcap_appp60m1_redcap_data 
timestamp as of '2024-07-27'
where project_id = 123

Similar code is used in the Restore REDcap Data from History notebook. Please refer to that notebook for a real-world example of using this feature.

See Also

Work with Delta Lake Table History