Knowledge Hut

APRIL 25, 2023

The tremendous growth in data generation, then the rise in data engineer jobs - there’s no arguing the fact that the big data industry is at its best pace and you, as an aspiring data engineer, have a lot to learn and make out of it - including some tools! What are Data Engineering Tools?

Data Engineering 52

Data Engineering Data Engineer Engineering Google Cloud 52

Data Engineering Podcast

NOVEMBER 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Building 130

Building Metadata MongoDB MySQL 130

Knowledge Hut

NOVEMBER 16, 2023

This is nothing but a data analytics course that can give you global exposure. The demand for SAS – data analytics is growing day-by-day and the business intelligence domain has emerged as one of the most trusted and lucrative options for science graduates.

Technology 98

Technology MySQL MongoDB Google Cloud 98

Data Engineering Podcast

NOVEMBER 6, 2022

Summary A lot of the work that goes into data engineering is trying to make sense of the "data exhaust" from other applications and services. Trusted by thousands of organizations and supported by over 20,000 community members, Prefect powers over 100MM business critical tasks a month.

Building 130

Building IT Metadata MongoDB 130

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Knowledge Hut

DECEMBER 26, 2023

Data Science is a combination of several disciplines including Mathematics and Statistics, Data Analysis, Machine Learning, and Computer Science. Data Science is a huge umbrella with a plethora of roles available in the field such as a Data Scientist, Data Engineer, BI Developer, Data and Analytics Manager, etc.

Data Science 97

Data Science BI Business Intelligence Data Mining 97

Data Engineering Podcast

AUGUST 21, 2022

In this episode Tracy Daniels, CDO of Truist, shares her journey into the position, her responsibilities, and her relationship to the data professionals in her organization. Their SDKs make event streaming from any app or website easy, and their state-of-the-art reverse ETL pipelines enable you to send enriched data to any cloud tool.

Metadata 100

Metadata MongoDB MySQL Data Lake 100

Data Engineering Podcast

SEPTEMBER 11, 2022

Summary Data engineering systems are complex and interconnected with myriad and often opaque chains of dependencies. In order to turn this into a tractable problem one approach is to define and enforce contracts between producers and consumers of data. Email hosts@dataengineeringpodcast.com ) with your story.

Systems 100

Systems Metadata Building MongoDB 100

Data Engineering Podcast

OCTOBER 16, 2022

In this episode Adrian Kosowski explains how the Pathway product got started, how its design simplifies the creation of data products that support supply chain operations, and how developers can help to build an ecosystem of applications that allow businesses to accelerate their time to insight.

Database 100

Database Metadata MongoDB MySQL 100

Data Engineering Podcast

OCTOBER 2, 2022

In this episode Roambee CEO, Sanjay Sharma, shares the types of questions that companies are asking about their logistics, the technical work that they do to provide ways to answer those questions, and how they approach the challenge of data quality in its many forms. For more information on Prefect, visit dataengineeringpodcast.com/prefect.

Metadata 100

Metadata Electronics MongoDB MySQL 100

Data Engineering Podcast

SEPTEMBER 25, 2022

In this episode Tom Baeyens explains their reasons for creating a new syntax for expressing and validating checks for data assets and processes, as well as how to incorporate it into your own projects. Data engineers don’t enjoy writing, maintaining, and modifying ETL pipelines all day, every day.

Building 100

Building Metadata MongoDB MySQL 100

Data Engineering Podcast

JUNE 12, 2022

Trupti Natu has been the first data hire multiple times and gone through the process of building teams across the different stages of growth. In this episode she shares her thoughts and insights on how to be intentional about establishing your own data team.

Metadata 100

Metadata Unstructured Data Business Intelligence MongoDB 100

Data Engineering Podcast

JULY 17, 2022

In this episode Crux CTO Mark Etherington discusses the different costs involved in managing external data, how to think about the total return on investment for your data, and how the Crux platform is architected to reduce the toil involved in managing third party data.

Data Management 130

Data Management Management Metadata MongoDB 130

Data Engineering Podcast

SEPTEMBER 4, 2022

In this episode founders Barr Moses and Lior Gavish rejoin the show to reflect on the evolution and adoption of data observability technologies and the capabilities that are being introduced as the broader ecosystem adopts the practices. The only thing worse than having bad data is not knowing that you have it.

IT 100

IT Metadata MongoDB MySQL 100

Data Engineering Podcast

JULY 10, 2022

In this episode Inbar Yogev and Lior Winner share the journey that they and their teams at Riskified have been on for their data platform. They also discuss how they have established a guild system for training and supporting data professionals in the organization. What is the composition of your data roles?

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Data Engineering Podcast

OCTOBER 9, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 100

Metadata AWS MongoDB MySQL 100

Data Engineering Podcast

JUNE 5, 2022

In this episode Sean Falconer explains the idea of a data privacy vault and how this new architectural element can drastically reduce the potential for making a mistake with how you manage regulated or personally identifiable information. If you’re a data engineering podcast listener, you get credits worth $5,000 when you become a customer.

Data Security 100

Data Security Metadata MongoDB Scala 100

Databand.ai

DECEMBER 13, 2022

The Top 25 Data Engineering Influencers and Content Creators on LinkedIn Ryan Yackel 2022-12-13 10:23:19 Interested in data engineering? LinkedIn is full of influencers sharing new ideas and sparking conversations on all kinds of topics, and data engineering is no exception. You’ve come to the right place.

Data Engineering 52

Data Engineering Data Engineer Engineering AWS 52

Data Engineering Podcast

AUGUST 6, 2022

In this episode Paolo Platter shares the lessons they have learned in that process, the Data Mesh Boost platform that they have built to reduce some of the boilerplate required to make it successful, and some of the considerations to make when deciding if a data mesh is the right choice for you.

Metadata 100

Metadata MongoDB MySQL Business Intelligence 100

Data Engineering Podcast

JULY 31, 2022

In this episode Ernie Ostic shares the approach that he and his team at Manta are taking to build a complete view of data lineage across the various data systems in your organization and the useful applications of that information in the work of every data stakeholder.

IT 100

IT Metadata MongoDB MySQL 100

Data Engineering Podcast

JULY 3, 2022

She also discusses her views on the role of the data lakehouse as a building block for these architectures and the ongoing influence that it will have as the technology matures. These technological shifts have brought about corresponding changes in data and platform architectures for managing data and analytical workflows.

Architecture 100

Architecture Metadata MongoDB MySQL 100

Knowledge Hut

DECEMBER 21, 2023

Data Science is the world's most rapidly growing sector and data engineers are at the forefront. In this article, we will understand the promising data engineer career outlook and what it takes to succeed in this role. What is Data Engineering? What are the Data Engineer Career Opportunities?

Data Engineering 52

Data Engineering Data Engineer Engineering MongoDB 52

Rockset

MAY 17, 2022

Complex SQL queries have long been commonplace in business intelligence (BI). And when systems such as Hadoop and Hive arrived, it married complex queries with big data for the first time. Limitations of NoSQL SQL supports complex queries because it is a very expressive, mature language. This is intentionally not their forte.

SQL 52

SQL NoSQL Hadoop MongoDB 52

Zalando Engineering

JUNE 6, 2016

Big Data Engineer The term Big Data was first coined in the 1990s by John Mashey , referring to a large set of data that is almost impossible to manage using traditional business intelligence tools.

Big Data 40

Big Data Programming Language MongoDB NoSQL 40

Data Engineering Podcast

DECEMBER 29, 2022

In this episode she shares the strategic and tactical elements of how to make more effective use of the technical and organizational resources that are available to you for getting work done with data. Missing data? Trusted by the data teams at Fox, JetBlue, and PagerDuty, Monte Carlo solves the costly problem of broken data pipelines.

Management 130

Management Metadata Business Intelligence Data Lake 130

ProjectPro

DECEMBER 7, 2021

This process enables quick data analysis and consistent data quality, crucial for generating quality insights through data analytics or building machine learning models. Build a Job Winning Data Engineer Portfolio with Solved End-to-End Big Data Projects What is an ETL Data Pipeline?

Data Pipeline 52

Data Pipeline Architecture Kafka AWS 52

Knowledge Hut

SEPTEMBER 6, 2023

Once the data is tailored to your requirements, it then should be stored in a warehouse system, where it can be easily used by applying queries. Some of the most popular database management tools in the industry are NoSql, MongoDB and oracle. Career Prospect - After completing this course you can take up a role as a Data Engineer.

Data Science 97

Data Science Certification Business Analyst Machine Learning 97

Data Engineering Podcast

DECEMBER 18, 2022

In this episode Abe Gong brings his experiences with the Great Expectations project and community to discuss the technical and organizational considerations involved in implementing these constraints to your data workflows. Missing data? Missing data? Struggling with broken pipelines? Stale dashboards? Stale dashboards?

Metadata 130

Metadata Business Intelligence Data Lake BI 130

Data Engineering Podcast

DECEMBER 25, 2022

Summary Five years of hosting the Data Engineering Podcast has provided Tobias Macey with a wealth of insight into the work of building and operating data systems at a variety of scales and for myriad purposes. Missing data? Struggling with broken pipelines? Stale dashboards? If this resonates with you, you’re not alone.

Building 100

Building Metadata Business Intelligence Data Lake 100

Data Engineering Podcast

NOVEMBER 20, 2022

Summary The majority of blog posts and presentations about data engineering and analytics assume that the consumers of those efforts are internal business users accessing an environment controlled by the business. Data engineers don’t enjoy writing, maintaining, and modifying ETL pipelines all day, every day.

Systems 130

Systems Metadata Data Pipeline MongoDB 130

Data Engineering Podcast

OCTOBER 23, 2022

In this episode Shane Gibson shares practical advice and insights from his years of experience as a consultant and engineer working in data about how to adopt agile principles in your data work so that you can move faster and provide more value to the business, while building systems that are maintainable and adaptable.

Project 130

Project Metadata MongoDB MySQL 130

Knowledge Hut

SEPTEMBER 6, 2023

They highlight competence in data management, a pivotal requirement in today's business landscape, making certified individuals a sought-after asset for employers aiming to efficiently handle, safeguard, and optimize data operations. Skills acquired : Core data concepts. MongoDB aggregation. MongoDB performance.

Database 52

Database Certification MongoDB MySQL 52

ProjectPro

AUGUST 24, 2021

Nevertheless, that is not the only job in the data world. Data professionals who work with raw data like data engineers, data analysts, machine learning scientists , and machine learning engineers also play a crucial role in any data science project.

Data Engineering 52

Data Engineering Data Engineer Coding Project 52

Data Engineering Podcast

NOVEMBER 27, 2022

In this episode Wes McKinney shares the ways that Arrow and its related projects are improving the efficiency of data systems and driving their next stage of evolution. Missing data? Data engineers struggling with unreliable data need look no further than Monte Carlo, the leading end-to-end Data Observability Platform!

Data Process 100

Data Process Process Metadata Business Intelligence 100

Data Engineering Podcast

JUNE 26, 2022

In this episode Isaac Brodsky explains how the Unfolded platform is architected, their experience joining the team at Foursquare, and how you can start using it for analyzing your spatial data today. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities.

Datasets 130

Datasets Unstructured Data Metadata MongoDB 130

Data Engineering Podcast

OCTOBER 5, 2021

This is an interesting deep dive into the internals of a high performance streaming engine and the details that are involved in building distributed systems. Missing data? Data engineers struggling with unreliable data need look no further than Monte Carlo, the world’s first end-to-end, fully automated Data Observability Platform!

Engineering 100

Engineering MongoDB Kafka Data Lake 100

ProjectPro

MAY 19, 2015

5 Reasons to Learn Hadoop With 2015 bringing in a larger list of big data use cases for analysing information, here are 5 reasons to learn Hadoop so that professionals can exploit these lucrative career opportunities in the big data market.

Hadoop 40

Hadoop Big Data NoSQL Database-centric 40