Data Engineering Podcast

AUGUST 14, 2021

What do you do when you need to manage unstructured information, or build a computer vision model? In this episode Davit Buniatyan, founder and CEO of Activeloop, explains why he is spending his time and energy on building a platform to simplify the work of getting your unstructured data ready for machine learning.

Unstructured Data 100

Unstructured Data Machine Learning Data Lake SQL 100

Snowflake

APRIL 20, 2023

Generative AI and large language models (LLMs) are revolutionizing many aspects of both developer and non-coder productivity with automation of repetitive tasks and fast generation of insights from large amounts of data. Figure 1: Visual Question Answering Challenge data types and results.

Building 117

Building Unstructured Data Government Coding 117

DoorDash Engineering

APRIL 23, 2024

Product attributes allow DoorDash to group products based on commonalities, building a product profile for each customer around their affinities to certain attributes. These are the building blocks for providing highly relevant and personalized shopping recommendations. Better personalization.

Building 91

Building Retail Manufacturing Unstructured Data 91

Data Engineering Podcast

FEBRUARY 27, 2022

In this episode she explains the difficulties that everyone faces as they scale beyond a single operating environment, and how the Komprise platform reduces the burden of managing large and heterogeneous collections of unstructured files.

Unstructured Data 100

Unstructured Data Cloud Management Metadata 100

Towards Data Science

FEBRUARY 9, 2024

How to build a modern, scalable data platform to power your analytics and data science projects (updated) Table of Contents: What’s changed? The Platform Integration Data Store Transformation Orchestration Presentation Transportation Observability Closing What’s changed?

Building 79

Building Transportation Data Lake Metadata 79

Snowflake

FEBRUARY 5, 2024

Financial services organizations need a modern data platform that allows them to anonymize data and share it without moving or copying it or risking the exposure of PII. Increasingly, financial institutions will monetize their data through apps and data marketplaces.

Unstructured Data 100

Unstructured Data Banking Government Insurance 100

Monte Carlo

JULY 19, 2023

Building a data stack doesn’t have to be complicated. Here’s what data leaders say are the 5 must-have layers of your data platform to drive data adoption – and ROI – across your business. Like bean dip and ogres , layers are the building blocks of the modern data stack. Makes sense.

Building 96

Building Business Intelligence Cloud Storage BI 96

Snowflake

FEBRUARY 6, 2024

To further encourage data use and reuse, adopt data product thinking , processes to facilitate their design and delivery , and teams to build and deploy them. An end-user-facing data catalog or marketplace can improve discoverability and access. Transform unstructured data to expand available internal data.

Unstructured Data 103

Unstructured Data Retail Data Manufacturing 103

Towards Data Science

JULY 21, 2023

Here are the 5 must-have layers to drive data product adoption at scale. Like bean dip and ogres , layers are the building blocks of the modern data stack. So, with infinitely expanding integrations and the opportunity to add new layers for every feature and function of your data motion, the question arises — where do you start?

Building 79

Building Business Intelligence BI Cloud Storage 79

Data Engineering Podcast

JUNE 26, 2022

In this episode Isaac Brodsky explains how the Unfolded platform is architected, their experience joining the team at Foursquare, and how you can start using it for analyzing your spatial data today. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows.

Datasets 130

Datasets Unstructured Data Metadata MongoDB 130

Cloudera

APRIL 1, 2024

We scored the highest in hybrid, intercloud, and multi-cloud capabilities because we are the only vendor in the market with a true hybrid data platform that can run on any cloud including private cloud to deliver a seamless, unified experience for all data, wherever it lies.

Cloud 104

Cloud Unstructured Data Metadata Datasets 104

Rockset

MARCH 28, 2024

Overview In this guide, you will: Gain a high-level understanding of vectors, embeddings, vector search, and vector databases, which will clarify the concepts we will build upon. Build a dynamic web application using vanilla CSS, HTML, JavaScript, and Flask, seamlessly integrating with the Rockset API and the OpenAI API.

Systems 52

Systems Building Database Utilities 52

Monte Carlo

AUGUST 19, 2023

A comprehensive data platform solution powers data acquisition, storage, preparation, delivery, governance, and even the robust security needs of users and applications. In today’s data-driven landscape, building a data platform is no longer a nice-to-have, but a necessity for most organizations. It depends.

Building 52

Building BI Data Lake Data Governance 52

Rockset

APRIL 18, 2023

Join me and Rockset VP of Engineering Louis Brandy for a tech talk, From Spam Fighting at Facebook to Vector Search at Rockset: How to Build Real-Time Machine Learning at Scale , on May 17th at 9am PT/ 12pm ET. Due to these difficulties, unstructured data has remained largely underutilized. Why use vector search?

Unstructured Data 105

Unstructured Data Metadata Machine Learning SQL 105

DareData

JULY 5, 2023

Learn how we build data lake infrastructures and help organizations all around the world achieving their data goals. In today's data-driven world, organizations are faced with the challenge of managing and processing large volumes of data efficiently. And what is the reason for that?

Data Lake 52

Data Lake Building Raw Data ETL Tools 52

Snowflake

APRIL 8, 2024

In 2020, Snowflake announced a new global competition to recognize the work of early-stage startups building their apps — and their businesses — on Snowflake, offering up to $250,000 in investment as the top prize. It deploys gen AI components as containers on Snowpark Container Services, close to the customer’s data. SignalFlare.ai

Pipeline-centric 126

Pipeline-centric Food Healthcare Unstructured Data 126

Monte Carlo

JANUARY 27, 2024

Building a modern data stack doesn’t have to be complicated. Here’s what data leaders say are the 5 must-have layers of your data platform to drive data adoption – and ROI – across your business. Like bean dip and ogres , layers are the building blocks of the modern data stack.

Building 52

Building Business Intelligence Cloud Storage BI 52

Knowledge Hut

APRIL 25, 2024

Big data vs machine learning is indispensable, and it is crucial to effectively discern their dissimilarities to harness their potential. Big data revolves around extensive volumes of structured and unstructured data originating from diverse sources. It focuses on collecting, storing, and processing extensive datasets.

Machine Learning 52

Machine Learning Big Data Unstructured Data Data Mining 52

Snowflake

MARCH 18, 2024

By 2025 it’s estimated that there will be 7 petabytes of data generated every day compared with “just” 2.3 And it’s not just any type of data. The majority of it (80%) is now estimated to be unstructured data such as images, videos, and documents — a resource from which enterprises are still not getting much value.

Unstructured Data 87

Unstructured Data Government Data Pipeline Cloud 87

Cloudera

MAY 14, 2024

This form of hybrid also goes a level deeper than one may find in a standard hybrid cloud, accounting for the entirety of the data lifecycle, whether that’s the point of ingestion, warehousing, or machine learning—even when that end-to-end data lifecycle is split between entirely different environments. Data comes in many forms.

Cloud 96

Cloud Data Governance Unstructured Data Data Architecture 96

Snowflake

NOVEMBER 1, 2023

That’s why we created Snowflake Cortex (in private preview), Snowflake’s new, intelligent, fully managed service that enables organizations to quickly analyze data and build AI applications — all within Snowflake. This means anyone who knows Python can securely build powerful LLM apps in minutes or hours, not days or weeks.

Unstructured Data 106

Unstructured Data SQL Python Accessible 106

AltexSoft

DECEMBER 23, 2022

Whether your goal is data analytics or machine learning , success relies on what data pipelines you build and how you do it. But even for experienced data engineers, designing a new data pipeline is a unique journey each time. Data engineering in 14 minutes. Tools to build an ELT pipeline.

Process 52

Process Building Raw Data Data Lake 52

Snowflake

MARCH 19, 2024

But before a company can be successful with generative AI, LLMs and other innovative technologies, it has to build a strong data foundation. This suggests that even as organizations increase the granularity of their data governance practices, they’re able to do more, not less, with the data.

Data Governance 117

Data Governance Government Unstructured Data Programming Language 117

Knowledge Hut

APRIL 23, 2024

They also facilitate historical analysis, as they store long-term data records that can be used for trend analysis, forecasting, and decision-making. Big Data In contrast, big data encompasses the vast amounts of both structured and unstructured data that organizations generate on a daily basis.

Data Warehouse 52

Data Warehouse Big Data Unstructured Data Hadoop 52

Monte Carlo

MAY 31, 2022

onFailure(funcName, qe, exception)) } } Spline provided a head-start in building Spark lineage. Instead of building a series of listeners from scratch, we decided to take advantage of that open source technology, Spline. A good engineer can build solutions for hard problems. .”)) new SparkLineageInitializer(sparkSession).createListener(isCodelessInit

Data Lake 52

Data Lake Building Scala Metadata 52

Snowflake

SEPTEMBER 19, 2023

AI unlocks new data use cases. With the ability to handle unstructured data types and larger volumes of data, AI gives us the tools to tackle more complex, exciting problems. But now this enables a newer kind of insights from all this unstructured data that has been untapped so far. Some takeaways?

Machine Learning 95

Machine Learning Unstructured Data Data Analytics Government 95

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Snowflake

MARCH 7, 2024

Compared to traditional AI models, generative AI/LLMs provide a significant uplift in tackling the wealth of unstructured data that make up things such as loan agreements, claims agreements, underwriting documents and the like. How do Gen AI/LLMs “democratize” access to insights?

Unstructured Data 83

Unstructured Data Government Data Architecture Architecture 83

AltexSoft

MAY 12, 2022

Audio data file formats. Similar to texts and images, audio is unstructured data meaning that it’s not arranged in tables with connected rows and columns. Building an app for snore and teeth grinding detection. AltexSoft & SleepScore Labs: Building an iOS App for Snoring and Teeth Grinding Detection.

Machine Learning 122

Machine Learning Building Datasets Healthcare 122

Snowflake

OCTOBER 26, 2023

Gen AI can also analyze unstructured data sets, such as clinical notes, diagnostic imaging and recordings and provide evidence-based recommendations. In addition, hiring for AI-related roles such as AI data scientists, data engineers and AI product owners remains a challenge.

Healthcare 84

Healthcare Unstructured Data High Quality Data Medical 84

Data Engineering Podcast

JUNE 12, 2022

Summary Building a well rounded and effective data team is an iterative process, and the first hire can set the stage for future success or failure. Trupti Natu has been the first data hire multiple times and gone through the process of building teams across the different stages of growth.

Metadata 100

Metadata Unstructured Data Business Intelligence MongoDB 100

Snowflake

JANUARY 8, 2024

Additionally, the use of synthetic data generated by AI can speed up stress testing and the evaluation and prediction of exposure risks, including those related to fluctuating interest rates and potential defaults. This ensures the models can be built and operated efficiently and can scale as the data volume increases.

Unstructured Data 83

Unstructured Data Government Portfolio Insurance 83

Monte Carlo

JANUARY 25, 2023

In an evolving data landscape, the explosion of new tooling solutions—from cloud-based transforms to data observability —has made the question of “build versus buy” increasingly important for data leaders. Hint: it’s not a one-size-fits-all answer, but answering a few critical questions can help. Let’s jump in!

Data Warehouse 40

Data Warehouse Building Data Lake Data Storage 40

Knowledge Hut

APRIL 23, 2024

Data Usage It stores the data in a sorted manner for future use. It uses data from the past and present to make decisions related to future growth. Data Type Data science deals with both structured and unstructured data. Business Intelligence only deals with structured data.

Business Intelligence 52

Business Intelligence Data Science BI Unstructured Data 52

Cloudera

MARCH 17, 2023

I love the enthusiasm surrounding ChatGPT and the eagerness to learn about modern data architectures such as data lakehouses, data meshes, and data fabrics. ChatGPT is an excellent resource for gaining high-level insights and building awareness of any technology.

Education 62

Education Unstructured Data Data Lake Data Warehouse 62

Data Engineering Weekly

APRIL 7, 2024

[link] Matt Turck: Full Steam Ahead: The 2024 MAD (Machine Learning, AI & Data) Landscape Coninue the week of insights into the world of data & AI landscape, the 2024 MAD landscape is out. EvalPlus builds a leadership board to demonstrate the efficiency of leading AI coder models.

Data Engineering 70

Data Engineering Data Engineer Engineering Unstructured Data 70

Snowflake

OCTOBER 11, 2023

The growing role of data science in the modern business Today’s businesses are facing an unprecedented expansion of unstructured data that can permeate every department in an organization. Here are five trends that startups should keep an eye on in the months ahead.

Unstructured Data 103

Unstructured Data Cloud Data Science Finance 103