Cloudera

FEBRUARY 8, 2023

How Bad is Bad Code: The ROI of Fixing Broken Spark Code Once in a while I stumble upon Spark code that looks like it has been written by a Java developer and it never fails to make me wince because it is a missed opportunity to write elegant and efficient code: it is verbose, difficult to read, and full of distributed processing anti-patterns.

Java 57

Java Datasets Coding Python 57

ProjectPro

NOVEMBER 22, 2021

PySpark has exploded in popularity in recent years, and many businesses are capitalizing on its advantages by producing plenty of employment opportunities for PySpark professionals. One of the examples of giants embracing PySpark is Trivago. Trivago has been employing PySpark to fulfill its team's tech demands.

Hadoop 52

Hadoop Python Datasets Metadata 52

Snowflake

JUNE 28, 2023

Flexible Runtime Constructs – Snowpark provides flexible compute and runtime constructs that allow users to bring in and run custom logic on warehouses or Snowpark Container Services (private preview). Vectorized UDTFs enable seamless partition-by-partition processing vs the row-by-row processing of scalar UDTFs.

Python 52

Python Accessible Accessibility Pipeline-centric 52