Always run your production Databricks Workflows in the name of a Service Principal.
When working with Databricks Workflows, it is often convenient to use the personal account of the creator as the “Run as” user.
However, this approach comes with significant downsides. That is why I strongly recommend establishing a rule to always run production workflows in the name of a Service Principal.
Here is why this is crucial, particularly for governance and security:
- Decoupling from individual users: A Service Principal remains active even if an employee leaves the organization, avoiding disruptions.
- Expanded permissions: Service Principals can perform operations beyond the scope of individual user permissions.
- Non-interactive authentication: They enable seamless authentication, perfect for automated pipelines and scheduled jobs.
- Auditable actions: Actions are logged under the Service Principal’s identity, simplifying change tracking and audit compliance.
This list is far from exhaustive. What would you add to this? Let me know what you think!
