Databasin Changelog

2025-1

General Enhancement

• Added Data Warehouse vs Data Lake ingestion pattern

2024-12

General Enhancements

• Add "Stop Job" feature for pipelines

Automation UI (Alpha)

• Generic cron scheduler
  • Run notebooks
  • Trigger Databricks Jobs
  • Docker container runs (coming soon)
• Provides three stages/steps
  • Setup step
  • Execution step
  • Completion step
• Supports executing notebooks from Databricks workspace or an Azure DevOps git repository.
• Supports Databasin File Drop to send CSV files to a target location

Automation Audit Templates (Beta)

• Can be used enforce auditing policies
• Templates can be customized to support different policies
• Templates surface when create an automation in the UI
  • Can be selected when an automation is created
  • Will require the user to provide the configuration to be used for auditing

SMB Connector (Beta)

• Need users to test various network share locations
• Share name is the network share (root folder) to the SMB connection
• Remote path is the path below the root folder
• Basic Authentication is the WUSTL key credentials
• Domain can be set to accounts for WUSTL key account

Data Warehouse Ingestion Type (In Progress)

• Adding a feature to bypass the medallion pattern in a pipeline target

Bug Fixes

• Private connectors now work in pipelines
• Failed jobs no longer show as running in the UI

2024-11

Onboard New Team

• New teams can now be created in the Databasin UI
• Creates baseline resources for a team
  • Databricks "team" group
  • Shared cluster
  • Sandbox schema
  • ...