Git Workflow

Diagram

Outline

1. Clone Your Git Repository: In your Databricks workspace, clone the remote repository to create a local version in your user folder.

2. Create a Feature Branch: Use the Repos UI to create a new feature branch from the main branch. This is where you'll make your modifications.

3. Regular Development: Make changes to Azure Databricks notebooks and other files in your feature branch. Regularly commit these changes.

4. Pull Latest Changes from Main Branch: Regularly pull the latest changes from the main branch to your feature branch to stay updated. This helps in reducing the chances of conflicts.

   1. In Azure Databricks, checkout your project branch
   2. Select Rebase from the menu
   3. Choose the main branch
   4. Click Begin Rebase

5. Resolve Conflicts: If a merge conflict occurs, manually resolve it by editing the conflicting files. Databricks provides UI tools to help identify and resolve these conflicts.

   1. Databricks will display all files with conflicting changes.
   2. Review these changes and determine the best way to resolve the conflict.
   3. Either manually edit the files and mark the conflicts as resolved, or use the drop down to accept all existing or all incoming changes if appropriate.
   4. Click Continue Rebase

6. Push Changes: Once your feature is complete and all conflicts are resolved, push your changes back to your branch in the remote repository.

7. Merge to Main Branch: Once your code is complete, create a pull request to merge your feature branch into the main branch. Resolve any conflicts that might arise during this process.

8. Final Merge: After resolving all conflicts and getting the necessary approvals, merge your feature branch into the main branch.

Additional Resources

• Databricks Repos Documentation
• Atlassian Documentation on Merge vs Rebase
• CI/CD techniques with Git and Databricks Repos