Data Engineering Podcast

JUNE 16, 2024

Summary Stripe is a company that relies on data to power their products and business. To support that functionality they have invested in Trino and Iceberg for their analytical workloads. In this episode Kevin Liu shares some of the interesting features that they have built by combining those technologies, as well as the challenges that they face in supporting the myriad workloads that are thrown at this layer of their data platform.

Data Lake 130

Data Lake High Quality Data Metadata Python 130

Jesse Anderson

JUNE 12, 2024

This video covers the latest announcements from StreamNative, Confluent, and WarpStream. We discuss communication protocols, how they’re used, and what they mean for you. We also discuss the various systems using Kafka’s protocol. Finally, we discuss the announcements about writing to Iceberg and DeltaLake directly from the broker and what that means for costs and operational ease.

Kafka 147

Kafka Data Lake Architecture Cloud 147

Christophe Blefari

JUNE 15, 2024

hey ( credits ) 🥹It's been a long time since I've put words down on paper or hit the keyboard to send bytes across the network. We're in the age of AI, and my lords computer science have evolved over the last 30 years. I'm writing this edition from my child's home, and it brings back memories. I got my first computer at the age of 6 and spent my days installing Windows 98 over and over again, getting lost between the BIOS and the Windows installation pages, playi

Data 100

Data Bytes Metadata SQL 100

Confessions of a Data Guy

JUNE 11, 2024

One of the things I love about Python is its flexibility and huge community, a community that puts out a never-ending stream of useful packages for the average Software Engineer. In a show of solidarity to the open-source community, I thought I would publish a PYPI package that will probably be used by 5 people […] The post Building Open-Source Python Packages – SparklePop appeared first on Confessions of a Data Guy.

Python 100

Python Building Software Engineer Software Engineering 100

Speaker: Maher Hanafi, VP of Engineering at Betterworks & Tony Karrer, CTO at Aggregage

Executive leaders and board members are pushing their teams to adopt Generative AI to gain a competitive edge, save money, and otherwise take advantage of the promise of this new era of artificial intelligence. There's no question that it is challenging to figure out where to focus and how to advance when it’s a new field that is evolving everyday. 💡 This new webinar featuring Maher Hanafi, VP of Engineering at Betterworks, will explore a practical framework to transform Generative AI pr

Data Collection

databricks

JUNE 13, 2024

Today, we are excited to announce Databricks LakeFlow, a new solution that contains everything you need to build and operate production data pipelines.

Data Engineering 126

Data Engineering Data Engineer Engineering Data Pipeline 126

Engineering at Meta

JUNE 12, 2024

As we continue to focus our AI research and development on solving increasingly complex problems, one of the most significant and challenging shifts we’ve experienced is the sheer scale of computation required to train large language models (LLMs). Traditionally, our AI model training has involved a training massive number of models that required a comparatively smaller number of GPUs.

Algorithm 114

Algorithm Data Storage Technology Building 114

Data Engineering Weekly

JUNE 10, 2024

Experience Enterprise-Grade Apache Airflow Astro augments Airflow with enterprise-grade features to enhance productivity, meet scalability and availability demands across your data pipelines, and more. Learn More → Cube Research: Crystallizing Snowflake Summit 2024 We should officially call the first week of June the data engineering week, as two major data companies are running their developer conference.

Data Engineering 103

Data Engineering Data Engineer Engineering Kafka 103

ArcGIS

JUNE 11, 2024

The Living Atlas team publishes a lot of web layers. Here's some of our favorite tips and tricks for customizing your layers and maps.

114

114

databricks

JUNE 12, 2024

“FactSet’s mission is to empower clients to make data-driven decisions and supercharge their workflows and productivity. To deliver AI-driven solutions across our entire.

Machine Learning 117

Machine Learning Data 117

Snowflake

JUNE 10, 2024

Discovering and surfacing telemetry traditionally can be a tedious and challenging process, especially when it comes to pinpointing specific issues for debugging. However, as applications and pipelines grow in complexity, understanding what’s happening beneath the surface becomes increasingly crucial. A lack of visibility hinders the development and maintenance of high-quality applications and pipelines, ultimately impacting customer experience.

Python 99

Python Java Hadoop Coding 99

Speaker: Jason Tanner

While growth of software-enabled solutions generates momentum, growth alone is not enough to ensure sustainability. The probability of success dramatically improves with early planning for profitability. A sustainable business model contains a system of interrelated choices made not once but over time. Join this webinar for an iterative approach to ensuring solution, economic and relationship sustainability.

Certification

KDnuggets

JUNE 12, 2024

A simple tutorial on how to connect to databases, execute SQL queries, and analyze and visualize data.

SQL 124

SQL Python Database Data 124

Engineering at Meta

JUNE 10, 2024

At Meta, Bento , our internal Jupyter notebooks platform, is a popular tool that allows our engineers to mix code, text, and multimedia in a single document. Use cases run the entire spectrum from what we call “lite” workloads that involve simple prototyping to heavier and more complex machine learning workflows. However, even though the lite workflows require limited compute, users still have to go through the same process of reserving and provisioning remote compute – a process that takes time

SQL 91

SQL Python Coding Machine Learning 91

Data Engineering Weekly

JUNE 16, 2024

Experience Enterprise-Grade Apache Airflow Astro augments Airflow with enterprise-grade features to enhance productivity, meet scalability and availability demands across your data pipelines, and more. Learn More → Databricks: Open Sourcing Unity Catalog This week brought many exciting developments, with Snowflake and Databricks announcing open-source catalogs.

Data Engineering 70

Data Engineering Data Engineer Engineering Kafka 70

databricks

JUNE 11, 2024

Today, we are excited to announce Databricks AI/BI , a new type of business intelligence product built from the ground up to deeply.

BI 133

BI Business Intelligence Data 133

Advertisement

Generative AI is upending the way product developers & end-users alike are interacting with data. Despite the potential of AI, many are left with questions about the future of product development: How will AI impact my business and contribute to its success? What can product managers and developers expect in the future with the widespread adoption of AI?

Data

Snowflake

JUNE 13, 2024

The journey toward achieving a robust data platform that secures all your data in one place can seem like a daunting one. But at Snowflake, we’re committed to making the first step the easiest — with seamless, cost-effective data ingestion to help bring your workloads into the AI Data Cloud with ease. Snowflake is launching native integrations with some of the most popular databases, including PostgreSQL and MySQL.

Data Ingestion 86

Data Ingestion MySQL PostgreSQL Data Pipeline 86

KDnuggets

JUNE 13, 2024

Data privacy has been a long-standing issue that continues to challenge the data industry. Let’s understand how rapid developments in the world of AI have elevated data privacy concerns.

Data 88

Data 88

ArcGIS

JUNE 10, 2024

Sketch an extent on your map and choose between more new features with the Processing Extent control in ArcGIS Pro 3.3!

Process 108

Process 108

Robinhood

JUNE 10, 2024

Robinhood was founded on a simple idea: that our financial markets should be accessible to all. With customers at the heart of our decisions, Robinhood is lowering barriers and providing greater access to financial information and investing. Together, we are building products and services that help create a financial system everyone can participate in. … The Robinhood team is incredibly excited to welcome Katelyn Perna as Crypto Chief Information Security Officer.

Finance 80

Finance Portfolio Accessible Accessibility 80

Advertisement

Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.

Database

databricks

JUNE 12, 2024

Over the last year, we have seen a surge of commercial and open-source foundation models showing strong reasoning abilities on general knowledge tasks.

Systems 117

Systems Building Data Science Data 117

Snowflake

JUNE 10, 2024

Today’s data-driven world requires an agile approach. Modern data teams are constantly under pressure to deliver innovative solutions faster than ever before. Fragmented tooling across data engineering, application development and AI/ML development creates a significant bottleneck, hindering the speed of value delivery required to stay competitive.

Python 91

Python Data Pipeline SQL Database 91

KDnuggets

JUNE 12, 2024

Learn programming for free from top-tier universities like Harvard and MIT.

Data Science 138

Data Science Coding Programming Data 138

ArcGIS

JUNE 11, 2024

Revolutionizing GIS: Streamlining Change Detection for Mapping Agencies.

Building 121

Building Deep Learning Machine Learning 121

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

Data Science

Cloudera

JUNE 14, 2024

At a time when artificial intelligence (AI) and tools like generative AI (GenAI) and large language models (LLMs) have exploded in popularity, getting the most out of organizational data is critical to driving business value and carving out a competitive market advantage. To reach that goal, more businesses are turning toward hybrid cloud infrastructure – with data on-premises, in the cloud, or both – as a means to tap into valuable data.

Data Governance 70

Data Governance Government Cloud Insurance 70

databricks

JUNE 13, 2024

We are excited to announce that we are open sourcing Unity Catalog, the industry’s first open source catalog for data and AI governance.

Government 92

Government Data 92

Snowflake

JUNE 11, 2024

Bringing machine learning (ML) models into production is often hindered by fragmented MLOps processes that are difficult to scale with the underlying data. Many enterprises stitch together a complex mix of various MLOps tools to build an end-to-end ML pipeline. The friction of having to set up and manage separate environments for features and models creates operational complexity that can be costly to maintain and difficult to use.

Management 87

Management Government Metadata Python 87

KDnuggets

JUNE 10, 2024

Machine Learning model is an exciting project. Learn how to develop your first model that the company would want to use.

Machine Learning 117

Machine Learning Building Project 117

Advertisement

“Reimagined: Building Products with Generative AI” is an extensive guide for integrating generative AI into product strategy and careers featuring over 150 real-world examples, 30 case studies, and 20+ frameworks, and endorsed by over 20 leading AI and product executives, inventors, entrepreneurs, and researchers.

Building

Engineering at Meta

JUNE 11, 2024

MobileConfig enables developers to centrally manage a mobile app’s configuration parameters in our data centers. Once a parameter value is changed on our central server, billions of app devices automatically fetch and apply the new value without app updates. These remotely managed configuration parameters serve various purposes such as A/B testing, feature rollout, and app personalization.

Java 74

Java Designing Technology Engineering 74

Robinhood

JUNE 12, 2024

Sessions kicked off in Las Vegas on April 29th and Chicago on May 1st Robinhood Markets, Inc. has partnered with Syracuse University’s D’Aniello Institute for Veterans and Military Families (IVMF) to bring retirement education workshops to entrepreneurs across the U.S. We’re honored to partner with an organization helping veterans and veteran family members launch and grow their own businesses.

Education 71

Education Programming Accessible Accessibility 71

databricks

JUNE 12, 2024

In an era marked by rapid advancements in artificial intelligence and an explosion of data and Gen AI tools, enterprises face fragmented data.

Data 100

Data 100

Snowflake

JUNE 12, 2024

It is estimated that between 80% and 90% of the world’s data is unstructured 1 , with text files and documents making up a significant portion. Every day, countless text-based documents, like contracts and insurance claims, are stored for safekeeping. Despite containing a wealth of insights, this vast trove of information often remains untapped, as the process of extracting relevant data from these documents is challenging, tedious and time-consuming.

Unstructured Data 75

Unstructured Data Finance Insurance Manufacturing 75

Speaker: John Mansour

If you're looking to advance your career in product management, there are more options than just climbing the management ladder. Join our upcoming webinar to learn about highly rewarding career paths that don't involve management responsibilities. We'll cover both career tracks and provide tips on how to position yourself for success in the one that's right for you.

Management