Running Python Wheel Tasks in Custom Docker Containers in Databricks

A step-by-step tutorial to build and run Python Wheel Tasks on custom Docker images in Databricks (feat. Poetry and Typer CLI)

Johannes Schmidt
Towards Data Science
13 min readJun 29, 2023

--

Photo by Lluvia Morales on Unsplash

Data engineers design and build pipelines to run ETL workloads so that data can be used downstream to solve business problems. In…

--

--

Software & Data Engineer at Datamesh GmbH. Sharing knowledge and code around software (cloud) development, data engineering & data science!