Data Engineering Podcast

NOVEMBER 20, 2022

Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. RudderStack helps you build a customer data platform on your warehouse or data lake. runs natively on data lakes and warehouses and in AWS, Google Cloud and Microsoft Azure.

Data Lake 130

Data Lake Data Ingestion MongoDB Scala 130

ProjectPro

FEBRUARY 16, 2023

ETL is a critical component of success for most data engineering teams, and with teams harnessing it with the power of AWS, the stakes are higher than ever. Data Engineers and Data Scientists require efficient methods for managing large databases, which is why centralized data warehouses are in high demand.

AWS 52

AWS Data Management ETL Tools Management 52

Data Engineering Podcast

OCTOBER 16, 2022

Summary The "data lakehouse" architecture balances the scalability and flexibility of data lakes with the ease of use and transaction support of data warehouses. Mention the podcast to get a free "In Data We Trust World Tour" t-shirt.

Data Lake 100

Data Lake Food MongoDB Scala 100

Zalando Engineering

APRIL 12, 2023

Some teams use tools like dependabot , scala-steward that create pull requests in repositories when new library versions are available. We publish a curated data set containing dependency data from the SBOM for every application we deploy, based on its Container image. maven-versions-plugin , gradle-versions-plugin ).

Java 98

Java Scala Python Metadata 98

AltexSoft

MARCH 30, 2023

What is Databricks Databricks is an analytics platform with a unified set of tools for data engineering, data management , data science, and machine learning. It combines the best elements of a data warehouse, a centralized repository for structured data, and a data lake used to host large amounts of raw data.

Scala 64

Scala Data Lake BI Google Cloud 64

Data Engineering Podcast

NOVEMBER 6, 2022

RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control.

MongoDB 130

MongoDB Scala MySQL Data Lake 130

Data Engineering Podcast

JUNE 19, 2022

RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control.

Metadata 130

Metadata MongoDB Scala MySQL 130

Data Engineering Podcast

JUNE 12, 2022

Acryl Data provides DataHub as an easy to consume SaaS product which has been adopted by several companies. Signup for the SaaS product at dataengineeringpodcast.com/acryl RudderStack helps you build a customer data platform on your warehouse or data lake.

Unstructured Data 130

Unstructured Data MongoDB Scala MySQL 130

Data Engineering Podcast

OCTOBER 30, 2022

Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. RudderStack helps you build a customer data platform on your warehouse or data lake. Datafold]([link] Datafold helps you deal with data quality in your pull request.

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Podcast

JULY 17, 2022

RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control.

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Data Engineering Podcast

NOVEMBER 13, 2022

Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. RudderStack helps you build a customer data platform on your warehouse or data lake. RudderStack helps you build a customer data platform on your warehouse or data lake.

MongoDB 100

MongoDB Scala Google Cloud MySQL 100

Data Engineering Podcast

OCTOBER 23, 2022

Mention the podcast to get a free "In Data We Trust World Tour" t-shirt. RudderStack helps you build a customer data platform on your warehouse or data lake. Ascend automates workloads on Snowflake, Databricks, BigQuery, and open source Spark, and can be deployed in AWS, Azure, or GCP.

Database 100

Database MySQL Cloud MongoDB 100

Data Engineering Podcast

SEPTEMBER 25, 2022

Mention the podcast to get a free "In Data We Trust World Tour" t-shirt. RudderStack helps you build a customer data platform on your warehouse or data lake. Ascend automates workloads on Snowflake, Databricks, BigQuery, and open source Spark, and can be deployed in AWS, Azure, or GCP.

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 18, 2022

RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control.

Hospitality 100

Hospitality Food MongoDB Scala 100

Data Engineering Podcast

JULY 31, 2022

RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control.

Data Analysis 100

Data Analysis MongoDB Scala Algorithm 100

Data Engineering Podcast

JULY 24, 2022

RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control.

MongoDB 100

MongoDB Scala MySQL Data Lake 100

Data Engineering Podcast

JULY 10, 2022

RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control.

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Data Engineering Podcast

JULY 3, 2022

Ascend users love its declarative pipelines, powerful SDK, elegant UI, and extensible plug-in architecture, as well as its support for Python, SQL, Scala, and Java. Ascend automates workloads on Snowflake, Databricks, BigQuery, and open source Spark, and can be deployed in AWS, Azure, or GCP.

Data Integration 100

Data Integration MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 11, 2022

RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control.

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Data Engineering Podcast

OCTOBER 9, 2022

Mention the podcast to get a free "In Data We Trust World Tour" t-shirt. RudderStack helps you build a customer data platform on your warehouse or data lake. Ascend automates workloads on Snowflake, Databricks, BigQuery, and open source Spark, and can be deployed in AWS, Azure, or GCP.

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

OCTOBER 2, 2022

Mention the podcast to get a free "In Data We Trust World Tour" t-shirt. RudderStack helps you build a customer data platform on your warehouse or data lake. Ascend automates workloads on Snowflake, Databricks, BigQuery, and open source Spark, and can be deployed in AWS, Azure, or GCP.

IT 100

IT Food PostgreSQL MongoDB 100

Knowledge Hut

SEPTEMBER 26, 2023

Language Compatibility: Databricks provides extensive language compatibility, catering to data professionals with diverse skill sets. Some of the prominent languages supported include: Scala: Ideal for developers who want to leverage the full power of Apache Spark.

Data Lake 52

Data Lake Database-centric Pipeline-centric Machine Learning 52

Data Engineering Podcast

SEPTEMBER 9, 2018

What are the benefits of implementing the Elasticsearch API on top of your data in S3 as opposed to using systems such as Presto or Drill to interact with the same information via SQL? What mechanisms do you use to allow for such drastic space savings of indexed data in S3 versus in an Elasticsearch cluster?

IT 100

IT PostgreSQL Scala AWS 100

Knowledge Hut

APRIL 25, 2023

Here, we'll take a look at the top data engineer tools in 2023 that are essential for data professionals to succeed in their roles. These tools include both open-source and commercial options, as well as offerings from major cloud providers like AWS, Azure, and Google Cloud. What are Data Engineering Tools?

Data Engineering 52

Data Engineering Data Engineer Engineering Google Cloud 52

Data Engineering Podcast

AUGUST 6, 2022

RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control.

Machine Learning 100

Machine Learning Database MySQL PostgreSQL 100

Data Engineering Podcast

JUNE 26, 2022

Data Engineering Podcast listeners can sign up for a free 2-week sandbox account, go to dataengineeringpodcast.com/tonic today to give it a try! RudderStack helps you build a customer data platform on your warehouse or data lake. RudderStack helps you build a customer data platform on your warehouse or data lake.

Data Management 100

Data Management Management MongoDB Scala 100

Snowflake

AUGUST 16, 2023

Applications Snowflake Native App Framework now available in AWS – public preview Snowflake Native Apps are an entirely new way to put data to work. Developers can now start building and testing Snowflake Native Apps in their accounts in AWS. Learn more here. Learn more here.

Scala 52

Scala Transportation Kafka Data Lake 52

Data Engineering Podcast

NOVEMBER 18, 2019

Summary With the constant evolution of technology for data management it can seem impossible to make an informed decision about whether to build a data warehouse, or a data lake, or just leave your data wherever it currently rests. How does it influence the relevancy of data warehouses or data lakes?

Data Lake 100

Data Lake Scala Data Warehouse Hadoop 100

Knowledge Hut

DECEMBER 26, 2023

Some of the best books that will guide you in Scala are:- Scala Cookbook: Recipes for Object-Oriented and Functional Programming (Author: Alvin Alexander) Scala for the Impatient (Author: Cay S. Horstmann) Programming Scala: Scalability = Functional Programming + Objects (Author: Alex Payne and Dean Wampler) 2.

Data Science 52

Data Science Scala Programming Language R (Programming) 52

Knowledge Hut

SEPTEMBER 26, 2023

We as Azure Data Engineers should have extensive knowledge of data modelling and ETL (extract, transform, load) procedures in addition to extensive expertise in creating and managing data pipelines, data lakes, and data warehouses. Different techniques are employed to store various kinds of data.

Certification 52

Certification Data Engineering Data Engineer Engineering 52

AltexSoft

FEBRUARY 11, 2023

It serves as a foundation for the entire data management strategy and consists of multiple components including data pipelines; , on-premises and cloud storage facilities – data lakes , data warehouses , data hubs ;, data streaming and Big Data analytics solutions ( Hadoop , Spark , Kafka , etc.);

Data Architect 87

Data Architect Certification Generalist Big Data 87

Cloudera

JUNE 16, 2022

Laila wants to use CSP but doesn’t have time to brush up on her Java or learn Scala, but she knows SQL really well. . SSB provides a comprehensive interactive user interface for developers, data analysts, and data scientists to write streaming applications with industry standard SQL. Without context, streaming data is useless.”

Kafka 86

Kafka Manufacturing Data Lake SQL 86

ProjectPro

FEBRUARY 21, 2023

AWS or Azure? With so many data engineering certifications available , choosing the right one can be a daunting task. This section mainly focuses on the three most valuable and popular vendor-specific data engineering certifications- AWS, Azure , and GCP. Cloudera or Databricks?

Certification 52

Certification Data Engineering Data Engineer Engineering 52

Edureka

FEBRUARY 9, 2023

Some of the top skills to include are: Experience with Azure data storage solutions: Azure Data Engineers should have hands-on experience with various Azure data storage solutions such as Azure Cosmos DB, Azure Data Lake Storage, and Azure Blob Storage.

Data Engineering 52

Data Engineering Data Engineer Engineering Amazon Web Services 52

Knowledge Hut

NOVEMBER 28, 2023

Python is ubiquitous, which you can use in the backends, streamline data processing, learn how to build effective data architectures, and maintain large data systems. Java can be used to build APIs and move them to destinations in the appropriate logistics of data landscapes.

Data Engineering 52

Data Engineering Data Engineer Engineering Generalist 52

Confluent

JUNE 7, 2019

At TokenAnalyst, we’re using Kafka for ingestion of blockchain data—which is directly pushed from our cluster of Bitcoin and Ethereum nodes—to different streams of transformation and loading processes. Complementary data types such as transaction receipts, event logs, and state diffs are also extracted.

Accessible 90

Accessible Accessibility Kafka Scala 90

Knowledge Hut

NOVEMBER 7, 2023

You can swiftly provision infrastructure services like computation, storage, and databases, as well as machine learning, the internet of things, data lakes and analytics, and much more. " Instead of relying on nearby hard drives and personal data centers, it requires storing and accessing data on distant servers.

Cloud Computing 98

Cloud Computing Cloud Amazon Web Services Entertainment 98