Data Engineering Podcast

JUNE 12, 2022

Summary Unstructured data takes many forms in an organization. From a data engineering perspective that often means things like JSON files, audio or video recordings, images, etc. Sign up free… or just get the free t-shirt for being a listener of the Data Engineering Podcast at dataengineeringpodcast.com/rudder.

Unstructured Data 130

Unstructured Data MongoDB Scala MySQL 130

Data Engineering Podcast

OCTOBER 30, 2022

Summary One of the most impactful technologies for data analytics in recent years has been dbt. It’s hard to have a conversation about data engineering or analysis without mentioning it. Despite its widespread adoption there are still rough edges in its workflow that cause friction for data analysts.

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Podcast

OCTOBER 16, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Lake 100

Data Lake Food MongoDB Scala 100

Data Engineering Podcast

NOVEMBER 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

MongoDB 100

MongoDB Scala Google Cloud MySQL 100

Data Engineering Podcast

AUGUST 21, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Lambda Architecture 100

Lambda Architecture MongoDB Scala MySQL 100

Knowledge Hut

DECEMBER 27, 2023

Scala Scala is a functional programming language that many people find fairly simple to learn and is another essential language for software engineering experts. SQL The computer language SQL, or Structured Query Language, is used to store, manipulate, and retrieve data from relational databases.

Software Engineer 98

Software Engineer Software Engineering Engineering Programming Language 98

Data Engineering Podcast

AUGUST 6, 2022

In this episode Frank Liu explains how the open source Milvus vector database is implemented to speed up machine learning development cycles, how to think about proper storage and scaling of these vectors, and how data engineering and machine learning teams can collaborate on the creation and maintenance of these data sets.

Machine Learning 100

Machine Learning Database MySQL PostgreSQL 100

Data Engineering Podcast

OCTOBER 2, 2022

Alvin is a platform that aims to provide a low effort solution for data lineage capabilities focused on simplifying the work of data engineers. Their SDKs make event streaming from any app or website easy, and their state-of-the-art reverse ETL pipelines enable you to send enriched data to any cloud tool.

IT 100

IT Food PostgreSQL MongoDB 100

Data Engineering Podcast

JULY 3, 2022

Summary The perennial challenge of data engineers is ensuring that information is integrated reliably. In order to quickly identify if and how two data systems are out of sync Gleb Mezhanskiy and Simon Eskildsen partnered to create the open source data-diff utility. Data teams are increasingly under pressure to deliver.

Data Integration 100

Data Integration MongoDB Scala MySQL 100

Data Engineering Podcast

JULY 24, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

MongoDB 100

MongoDB Scala MySQL Data Lake 100

Knowledge Hut

MAY 2, 2024

As per Apache, “ Apache Spark is a unified analytics engine for large-scale data processing ” Spark is a cluster computing framework, somewhat similar to MapReduce but has a lot more capabilities, features, speed and provides APIs for developers in many languages like Scala, Python, Java and R.

Scala 52

Scala Hospitality Healthcare Retail 52

Data Engineering Podcast

JUNE 5, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Security 100

Data Security Metadata MongoDB Scala 100

Data Engineering Podcast

SEPTEMBER 25, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

AUGUST 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 100

Metadata MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 18, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Hospitality 100

Hospitality Food MongoDB Scala 100

Data Engineering Podcast

SEPTEMBER 11, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. We also offer a 2-week free trial.

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Databand.ai

DECEMBER 13, 2022

The Top 25 Data Engineering Influencers and Content Creators on LinkedIn Ryan Yackel 2022-12-13 10:23:19 Interested in data engineering? LinkedIn is full of influencers sharing new ideas and sparking conversations on all kinds of topics, and data engineering is no exception. You’ve come to the right place.

Data Engineering 52

Data Engineering Data Engineer Engineering AWS 52

Data Engineering Podcast

AUGUST 28, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Building 100

Building MongoDB Scala MySQL 100

Snowflake

SEPTEMBER 7, 2023

Snowpark is the set of libraries and runtimes that enables data engineers, data scientists and developers to build data engineering pipelines, ML workflows, and data applications in Python, Java, and Scala. For example, Maps API can be used to get location data, which can optimize supply chain routes.

Amazon Web Services 118

Amazon Web Services AWS Government Python 118

Data Engineering Podcast

OCTOBER 9, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

AUGUST 13, 2022

Summary Data engineers have typically left the process of data labeling to data scientists or other roles because of its nature as a manual and process heavy undertaking, focusing instead on building automation and repeatable systems. Data labeling is a large and competitive market.

Machine Learning 100

Machine Learning Pipeline-centric Database-centric MongoDB 100

Data Engineering Podcast

JUNE 26, 2022

Summary The most complicated part of data engineering is the effort involved in making the raw data fit into the narrative of the business. Your newly mimicked datasets are safe to share with developers, QA, data scientists—heck, even distributed teams around the world.

Data Management 100

Data Management Management MongoDB Scala 100

Data Engineering Podcast

SEPTEMBER 4, 2022

In this episode Gopal Erinjippurath discusses the data engineering challenges of building and serving those data sets, and how they are distilling complex climate information into consumable facts so you don’t have to be an expert to understand it. Go to dataengineeringpodcast.com/ascend and sign up for a free trial.

MongoDB 100

MongoDB Scala MySQL Data Ingestion 100

Knowledge Hut

DECEMBER 26, 2023

Data Science is a combination of several disciplines including Mathematics and Statistics, Data Analysis, Machine Learning, and Computer Science. Data Science is a huge umbrella with a plethora of roles available in the field such as a Data Scientist, Data Engineer, BI Developer, Data and Analytics Manager, etc.

Data Science 97

Data Science BI Business Intelligence Data Mining 97

ProjectPro

FEBRUARY 21, 2023

They must also understand how to build a data processing pipeline that can support the five Vs of big data- volume, velocity, variety, veracity, and value- as well as how to transform this data into maintainable code. Python, Java, and Scala knowledge are essential for Apache Spark developers.

Scala 52

Scala Programming Language Java Hadoop 52

Data Engineering Podcast

DECEMBER 9, 2018

Jean George Perrin has been so impressed by the versatility of Spark that he is writing a book for data engineers to hit the ground running. He also discusses what you need to know to get it deployed and keep it running in a production environment and how it fits into the overall data ecosystem.

Scala 100

Scala MySQL Kafka Hadoop 100

Knowledge Hut

SEPTEMBER 6, 2023

These certifications have big data training courses where tutors help you gain all the knowledge required for the certification exam. Programming Languages : Good command on programming languages like Python, Java, or Scala is important as it enables you to handle data and derive insights from it. It is a 13-course series.

Big Data 52

Big Data Certification Hadoop Scala 52

Data Engineering Podcast

NOVEMBER 20, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Lake 130

Data Lake Data Ingestion MongoDB Scala 130

ProjectPro

NOVEMBER 14, 2015

It plays a key role in streaming in the form of Spark Streaming libraries, interactive analytics in the form of SparkSQL and also provides libraries for machine learning that can be imported using Python or Scala. From Data Engineering Fundamentals to full hands-on example projects , check out data engineering projects by ProjectPro 2.

Hadoop 52

Hadoop Project Big Data Healthcare 52

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 130

Metadata MongoDB Scala MySQL 130

Data Engineering Podcast

JULY 31, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Analysis 100

Data Analysis MongoDB Scala Algorithm 100

Data Engineering Podcast

MAY 29, 2022

Summary A large fraction of data engineering work involves moving data from one storage location to another in order to support different access and query patterns. Singlestore aims to cut down on the number of database engines that you need to run so that you can reduce the amount of copying that is required.

Architecture 100

Architecture Database Data Architecture PostgreSQL 100

Knowledge Hut

MARCH 20, 2024

Average Salary: $93,438 Required skills: The different types of software engineers must have expertise in coding automated tests using various automation tools such as Cucumber, Watir, Selenium, Microsoft Coded UI, Katalone, UFT/QTP, and others. To work with a range of databases, you must learn SQL Scripts and be able to write SQL.

Software Engineer 52

Software Engineer Software Engineering Engineering Java 52

ProjectPro

FEBRUARY 8, 2023

Another leading European company, Claranet, has adopted Glue to migrate their data load from their existing on-premise solution to the cloud. The popular data integration tool, AWS Glue, enables data analytics users to quickly acquire, analyze, migrate, and integrate data from multiple sources. How Does AWS Glue Work?

AWS 98

AWS Scala Metadata Data Lake 98

ProjectPro

JULY 27, 2021

This blog is your one-stop solution for the top 100+ Data Engineer Interview Questions and Answers. In this blog, we have collated the frequently asked data engineer interview questions based on tools and technologies that are highly useful for a data engineer in the Big Data industry.

Data Engineering 40

Data Engineering Data Engineer Engineering Hadoop 40

ProjectPro

JANUARY 31, 2023

Big Data Interview Questions and Answers Based on Job Role With the help of ProjectPro experts, we have compiled a list of interview questions on big data based on several job roles, including big data tester, big data developer, big data architect, and big data engineer. What are its benefits?

Big Data 52

Big Data Hadoop AWS Relational Database 52

AltexSoft

OCTOBER 21, 2022

It offers high throughput, low latency, and scalability that meets the requirements of Big Data. The technology was written in Java and Scala in LinkedIn to solve the internal problem of managing continuous data flows. cloud data warehouses — for example, Snowflake , Google BigQuery, and Amazon Redshift.

Kafka 93

Kafka Hadoop ETL Tools Big Data 93