Data Engineering Podcast

NOVEMBER 20, 2022

In this episode CEO and founder Salma Bakouk shares her views on the causes and impacts of "data entropy" and how you can tame it before it leads to failures. The Ascend Data Automation Cloud provides a unified platform for data ingestion, transformation, orchestration, and observability.

Data Lake 130

Data Lake Data Ingestion MongoDB Scala 130

Data Engineering Podcast

JUNE 12, 2022

Another category of unstructured data that every business deals with is PDFs, Word documents, workstation backups, and countless other types of information. In this episode Rod Christensen shares the story behind Aparavi and how you can use it to cut costs and gain value for the long tail of your unstructured data.

Unstructured Data 130

Unstructured Data MongoDB Scala MySQL 130

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Metadata 130

Metadata MongoDB Scala MySQL 130

Data Engineering Weekly

MARCH 31, 2024

[link] Databricks: PySpark in 2023 - A Year in Review Can we safely say PySpark killed Scala-based data pipelines? I’m looking forward to playing around with Testing API and Arrow-optimized UDF since UDF is the only reason I write Scala nowadays.

Data Engineering 62

Data Engineering Data Engineer Engineering Scala 62

Data Engineering Podcast

NOVEMBER 6, 2022

In this episode she shares the story behind the project, the details of how it is implemented, and how you can use it for your own data projects. The Ascend Data Automation Cloud provides a unified platform for data ingestion, transformation, orchestration, and observability. That’s where our friends at Ascend.io

MongoDB 130

MongoDB Scala MySQL Data Lake 130

Data Engineering Podcast

NOVEMBER 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

MongoDB 100

MongoDB Scala Google Cloud MySQL 100

Data Engineering Podcast

JULY 24, 2022

Summary The current stage of evolution in the data management ecosystem has resulted in domain and use case specific orchestration capabilities being incorporated into various tools. In this episode Nick Schrock discusses the importance of orchestration and a central location for managing data systems, the road to Dagster’s 1.0

MongoDB 100

MongoDB Scala MySQL Data Lake 100

Knowledge Hut

DECEMBER 26, 2023

Some of the best books that will guide you in Scala are:- Scala Cookbook: Recipes for Object-Oriented and Functional Programming (Author: Alvin Alexander) Scala for the Impatient (Author: Cay S. Horstmann) Programming Scala: Scalability = Functional Programming + Objects (Author: Alex Payne and Dean Wampler) 2.

Data Science 52

Data Science Scala Programming Language R (Programming) 52

Ascend.io

SEPTEMBER 5, 2023

In the fast-evolving landscape of cloud data solutions, Snowflake has consistently been at the forefront of innovation, offering enterprises sophisticated tools to optimize their data management.

IT 59

IT Scala Java Programming Language 59

Data Engineering Podcast

JULY 17, 2022

In this episode Joe Reis, founder of Ternary Data and co-author of "Fundamentals of Data Engineering", turns the tables and interviews the host, Tobias Macey, about his journey into podcasting, how he runs the show behind the scenes, and the other things that occupy his time. That’s where our friends at Ascend.io Links Podcast.__init__

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Snowflake

AUGUST 30, 2023

In this blog post, we will delve into six such capabilities – comprehensive cross-cloud replication, zero copy database and schema clone, collation support, stored procedures, multi-table transactions, and transparent online upgrade – that every enterprise must consider while choosing their data platforms.

Scala 90

Scala Government Database Cloud 90

Data Engineering Podcast

SEPTEMBER 25, 2022

In this episode Mark Van de Wiel explains how they integrated CDC functionality into their existing product, discusses the nuances of different approaches to change data capture from various sources. Sign up free… or just get the free t-shirt for being a listener of the Data Engineering Podcast at dataengineeringpodcast.com/rudder.

Food 100

Food MongoDB Scala MySQL 100

Knowledge Hut

NOVEMBER 17, 2023

Additionally, they convert data into formats that can be used and store it effectively and securely in the Azure cloud. The data engineers are responsible for creating conversational chatbots with the Azure Bot Service and automating metric calculations using the Azure Metrics Advisor.

Data Engineering 52

Data Engineering Data Engineer Engineering Scala 52

Data Engineering Podcast

OCTOBER 9, 2022

In this episode Purvi Shah, the VP of Enterprise Big Data Platforms at American Express, explains how they have invested in the cloud to power this visibility and the complex suite of integrations they have built and maintained across legacy and modern systems to make it possible. Data teams are increasingly under pressure to deliver.

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

MAY 29, 2022

In this episode SVP of engineering Shireesh Thota describes the impact on your overall system architecture that Singlestore can have and the benefits of using a cloud-native database engine for your next application. That’s where our friends at Ascend.io Can you describe what SingleStore is and the story behind it?

Architecture 100

Architecture Database Data Architecture PostgreSQL 100

Data Engineering Podcast

OCTOBER 16, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Lake 100

Data Lake Food MongoDB Scala 100

Data Engineering Podcast

SEPTEMBER 11, 2022

In this episode he shares his journey from building a consumer product to launching a data pipeline service and how his frustrations as a product owner have informed his work at Hevo Data. Sign up free… or just get the free t-shirt for being a listener of the Data Engineering Podcast at dataengineeringpodcast.com/rudder.

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Data Engineering Podcast

SEPTEMBER 18, 2022

He discusses the challenges of making data collection purposeful and efficient without distracting employees from their primary duties and how business owners can use the provided analytics to support their staff in their duties. Data teams are increasingly under pressure to deliver. That’s where our friends at Ascend.io

Hospitality 100

Hospitality Food MongoDB Scala 100

Data Engineering Podcast

AUGUST 21, 2022

In this episode Shruti Bhat gives her view on the state of the ecosystem for real-time data and the work that she and her team at Rockset is doing to make it easier for engineers to build those experiences. The Ascend Data Automation Cloud provides a unified platform for data ingestion, transformation, orchestration, and observability.

Lambda Architecture 100

Lambda Architecture MongoDB Scala MySQL 100

Data Engineering Podcast

AUGUST 6, 2022

In this episode Frank Liu explains how the open source Milvus vector database is implemented to speed up machine learning development cycles, how to think about proper storage and scaling of these vectors, and how data engineering and machine learning teams can collaborate on the creation and maintenance of these data sets.

Machine Learning 100

Machine Learning Database MySQL PostgreSQL 100

Data Engineering Podcast

JULY 31, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Analysis 100

Data Analysis MongoDB Scala Algorithm 100

Data Engineering Podcast

JULY 10, 2022

In this episode Chris Riccomini shares his experiences building and scaling data operations at WePay and LinkedIn, as well as the lessons he has learned working with other teams as they automated their own systems. Data teams are increasingly under pressure to deliver. What does the future of data engineering look like?

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Data Engineering Podcast

JULY 3, 2022

In this episode they explain how the utility is implemented to run quickly and how you can start using it in your own data workflows to ensure that your data warehouse isn’t missing any records from your source systems. Can you describe what the data diff tool is and the story behind it? That’s where our friends at Ascend.io

Data Integration 100

Data Integration MongoDB Scala MySQL 100

AltexSoft

MARCH 30, 2023

What is Databricks Databricks is an analytics platform with a unified set of tools for data engineering, data management , data science, and machine learning. It combines the best elements of a data warehouse, a centralized repository for structured data, and a data lake used to host large amounts of raw data.

Scala 64

Scala Data Lake BI Google Cloud 64

Data Engineering Podcast

OCTOBER 2, 2022

In this episode co-founder Martin Sahlen explains the impact that easy access to lineage information can have on the work of data engineers and analysts, and how he and his team have designed their platform to offer that information to engineers and stakeholders in the places that they interact with data.

IT 100

IT Food PostgreSQL MongoDB 100

Netflix Tech

FEBRUARY 9, 2022

Let’s define some requirements that we are interested in delivering to the Netflix data engineers or anyone who would like to schedule a workflow with some external assets in it. It could be a JAR compiled from Scala, a Python script or module, or a simple SQL file. scala-workflow ??? setup.py ??? build.gradle ???

Data Pipeline 80

Data Pipeline Scala Management Python 80

Data Engineering Podcast

AUGUST 13, 2022

In this episode Shinji Kim discusses the challenges of data discovery and how to collect and preserve additional context about each piece of information so that you can find what you need when you don’t even know what you’re looking for yet. That’s where our friends at Ascend.io Closing Announcements Thank you for listening!

Metadata 100

Metadata MongoDB Scala MySQL 100

Data Engineering Podcast

OCTOBER 30, 2022

In this episode he shares his experiences working with organizations to adopt analytics engineering patterns and the ways that Optimus and dbt were combined to let data analysts deliver insights without the roadblocks of complex pipeline management. Interview Introduction How did you get involved in the area of data management?

Engineering 100

Engineering MongoDB Scala MySQL 100

Snowflake

NOVEMBER 29, 2023

Data cloud technology can accelerate FAIRification of the world’s biomedical patient data. Flexibility to ensure that the data itself is interoperable and reusable across varied research, clinical and real-world use cases. The principles emphasize machine-actionability (i.e.,

Metadata 84

Metadata Healthcare Medical Data Storage 84

Knowledge Hut

APRIL 25, 2024

If you pursue the MSc big data technologies course, you will be able to specialize in topics such as Big Data Analytics, Business Analytics, Machine Learning, Hadoop and Spark technologies, Cloud Systems etc. Look for a suitable big data technologies company online to launch your career in the field.

Big Data 98

Big Data Technology NoSQL Hadoop 98

Data Engineering Podcast

AUGUST 28, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Building 100

Building MongoDB Scala MySQL 100

Data Engineering Podcast

JUNE 5, 2022

In this episode Sean Falconer explains the idea of a data privacy vault and how this new architectural element can drastically reduce the potential for making a mistake with how you manage regulated or personally identifiable information. That’s where our friends at Ascend.io Can you describe what Skyflow is and the story behind it?

Data Security 100

Data Security Metadata MongoDB Scala 100

Knowledge Hut

NOVEMBER 28, 2023

The contemporary world experiences a huge growth in cloud implementations, consequently leading to a rise in demand for data engineers and IT professionals who are well-equipped with a wide range of application and process expertise. Work closely with software engineers and data scientists.

Data Engineering 52

Data Engineering Data Engineer Engineering Generalist 52

Knowledge Hut

DECEMBER 27, 2023

Data management skills Data management involves gathering, arranging, safeguarding, and archiving an organization's data. This data is further used for analysis and business decision-making. Data management skills are crucial in managing enormous amounts of data enterprises produce.

Programming Language 52

Programming Language Scala Software Engineer Software Engineering 52

Data Engineering Podcast

JUNE 28, 2021

This is a fascinating exploration of the technical and organizational evolution of the Influx Data platform, with some promising glimpses of where they are headed in the near future. It builds your customer data warehouse and your identity graph on your data warehouse, with support for Snowflake, Google BigQuery, Amazon Redshift, and more.

Data Analysis 100

Data Analysis Scala Data Warehouse Kafka 100

ProjectPro

FEBRUARY 8, 2023

For e.g., Finaccel, a leading tech company in Indonesia, leverages AWS Glue to easily load, process, and transform their enterprise data for further processing. Another leading European company, Claranet, has adopted Glue to migrate their data load from their existing on-premise solution to the cloud. How Does AWS Glue Work?

AWS 98

AWS Scala Metadata Data Lake 98

Data Engineering Podcast

AUGUST 13, 2022

In this episode founder Shayan Mohanty explains how he and his team are bringing software best practices and automation to the world of machine learning data preparation and how it allows data engineers to be involved in the process. That’s where our friends at Ascend.io Can you describe what Watchful is and the story behind it?

Machine Learning 100

Machine Learning Pipeline-centric Database-centric MongoDB 100