Prints to Std-Out
Every print statement in your Databricks code goes straight to stdout in the cluster logs.
Create a DataFrame
Creating a DataFrame in Pandas and PySpark may seem almost identical at first glance, but there's a key difference.
The Lakehouse Path: Databricks Logger Part 2
Last Thursday I published the second part of my three part series on logging in Databricks in my blog The Lakehouse Path.
Spark Session not needed
Did you know that you can easily spot whether someone is new to Databricks or has copied and pasted code from an LLM when they create a Spark Session in a Notebook?
The Lakehouse Path: Databricks Logger Part 1
Over the past few weeks, I’ve been exploring Substack as a platform for publishing long-form technical content.
Databricks Runtime 17 Pandas Version
Databricks Runtime 17 is set to make some Pandas users happy.
Preinstalled Python Libraries
Before you install Python packages on your Databricks Cluster, do this: Did you know that every Databricks Runtime comes with a suite of preinstalled Python libraries?
Never Use Pandas in Databricks for Data Transformation
Here’s a bold statement: I never use Pandas for data transformation in Databricks.
Luciano Ramalho: Fluent Python
After years of working with Python, I recently gained a deeper understanding of its object model, thanks to _Fluent Python_ by Luciano Ramalho.
Let's talk about your lakehouse.
Thirty minutes, no slides. Bring the question that keeps coming back.