Data Engineering Podcast

JUNE 17, 2023

By incorporating column-level lineage in the data modeling process it encourages a more robust and well-informed design. In this episode Satish Jayanthi explores the benefits of incorporating column-aware tooling in the data modeling process. Closing Announcements Thank you for listening!

Data Lake 130

Data Lake Metadata Data Architecture Machine Learning 130

Knowledge Hut

MARCH 28, 2024

Data Engineer Career: Overview Currently, with the enormous growth in the volume, variety, and veracity of data generated and the will of large firms to store and analyze their data, data management is a critical aspect of data science. That’s where data engineers are on the go.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

The Modern Data Company

FEBRUARY 26, 2024

With each new product launch and market expansion, the data architecture that once supported its growth now threatened to be its Achilles’ heel. Mixing metaphors aside, this startup knows that to sustain growth and outmaneuver competitors, it needs to evolve its approach to data management.

Architecture 52

Architecture Data Architecture Data Technology 52

AltexSoft

FEBRUARY 11, 2023

In the article, we explore the role of a data architect, discuss the responsibilities and required skills, and share what kind of companies may need such a specialist. What is a data architect? To get a better understanding of a data architect’s role, let’s clear up what data architecture is.

Data Architect 87

Data Architect Certification Generalist Big Data 87

Knowledge Hut

MARCH 26, 2024

An essential position in IT is that of a solution architect who develops and deploys technological solutions that are customized to satisfy certain business requirements. They have a thorough understanding of a variety of technologies.

Amazon Web Services 52

Amazon Web Services Google Cloud Computer Science AWS 52

Data Engineering Podcast

JULY 3, 2022

Summary The ecosystem for data tools has been going through rapid and constant evolution over the past several years. These technological shifts have brought about corresponding changes in data and platform architectures for managing data and analytical workflows. When is a lakehouse the wrong choice?

Architecture 100

Architecture Metadata MongoDB MySQL 100

Cloudera

JUNE 7, 2022

Companies can now capitalize on the value in all their data, by delivering a hybrid data platform for modern data architectures with data anywhere. Cloudera Data Platform (CDP) is designed to address the critical requirements for modern data architectures today and tomorrow.

IT 108

IT Unstructured Data Data Architecture Government 108

Data Engineering Podcast

OCTOBER 2, 2021

Summary Aerospike is a database engine that is designed to provide millisecond response times for queries across terabytes or petabytes. He also discusses the technical implementation that allows for such extreme performance and how the data model contributes to the scalability of the system.

Building 100

Building BI Data Architecture Architecture 100

Snowflake

NOVEMBER 2, 2023

Over the years, the technology landscape for data management has given rise to various architecture patterns, each thoughtfully designed to cater to specific use cases and requirements. Each of these architectures has its own unique strengths and tradeoffs.

Data Lake 89

Data Lake Data Warehouse Cloud Unstructured Data 89

Data Engineering Podcast

APRIL 3, 2022

Summary The flexibility of software oriented data workflows is useful for fulfilling complex requirements, but for simple and repetitious use cases it adds significant complexity. Coalesce is a platform designed to reduce repetitive work for common workflows by adopting a visual pipeline builder to support your data warehouse transformations.

Data Warehouse 100

Data Warehouse Data Workflow Data Architecture SQL 100

Databand.ai

AUGUST 30, 2023

DataOps Architecture: 5 Key Components and How to Get Started Ryan Yackel August 30, 2023 What Is DataOps Architecture? DataOps is a collaborative approach to data management that combines the agility of DevOps with the power of data analytics. As a result, they can be slow, inefficient, and prone to errors.

Architecture 52

Architecture Data Ingestion Data Governance Data Cleanse 52

Striim

FEBRUARY 19, 2024

Central to unlocking the potential of these frameworks is Striim, a platform that facilitates real-time data integration, analytics, and event processing, providing businesses can fully leverage their data. Data Fabric uses advanced technology, such as AI and machine learning, to automatically find, manage, and combine data.

Data Integration 52

Data Integration Government Data Data Governance 52

The Modern Data Company

APRIL 15, 2024

There’s a common saying about not wanting to know how the sausage is made, suggesting that the process behind many things we enjoy—be it our favorite food or the conveniences of modern technology—might not be as appealing as the end product itself. This analogy rings especially true in the world of data.

Building 52

Building Raw Data Food Data 52

Cloudera

JANUARY 30, 2024

As we navigate the fourth and fifth industrial revolution, AI technologies are catalyzing a paradigm shift in how products are designed, produced, and optimized. Quality: Use cases like visual inspection, yield optimization, fault detection, and classification are enhanced with AI technologies. That is a very low number.

Manufacturing 77

Manufacturing Data Lake Data Science Cloud 77

Knowledge Hut

JANUARY 18, 2024

But here's the fascinating part - it's estimated that by 2025, a whopping 463 exabytes of data will be created globally every single day. To put that into perspective, that's equivalent to 212,765,957 DVDs worth of data! The data analytics future is brimming with exciting possibilities. Are Data Analysts in Demand?

Data Analytics 52

Data Analytics Healthcare Machine Learning Algorithm 52

Knowledge Hut

DECEMBER 21, 2023

Data engineering involves a lot of technical skills like Python, Java, and SQL (Structured Query Language). For a data engineer career, you must have knowledge of data storage and processing technologies like Hadoop, Spark, and NoSQL databases. What are the Data Engineer Career Opportunities?

Data Engineering 52

Data Engineering Data Engineer Engineering MongoDB 52

Snowflake

FEBRUARY 12, 2024

With a projected attendance of around 90,000 people, MWC is a global networking hub that attracts key players, decision-makers and innovators in multiple telecom areas (including mobile service providers, device manufacturers, technology providers and digital services) and almost every industry imaginable. We’re at the heart of it.

Python 81

Python Manufacturing Cloud Government 81

Monte Carlo

JANUARY 24, 2024

More data, more access to data, and more regulations mean data governance has become a higher-stakes game. Data Governance Trends The biggest data governance trend isn’t really a trend at all—rather, it’s a state of mind. At the same time, data governance technologies are growing more intelligent.

Data Governance 52

Data Governance Government Data Lake Data Architecture 52

Precisely

MARCH 14, 2023

New technologies are making it easier for customers to process increasingly large datasets more rapidly. And the desire to leverage those technologies for analytics, machine learning, or business intelligence (BI) has grown exponentially as well. Simply design data pipelines, point them to the cloud environment, and execute.

Data Integration 52

Data Integration High Quality Data BI Data 52

Data Engineering Podcast

SEPTEMBER 23, 2018

He also explains which layers are useful for the different members of the business, and which pitfalls to look out for along the path to a mature and flexible data platform. How do you define data curation? How does the size and maturity of a company affect the ways that they architect and interact with their data systems?

Data Lake 100

Data Lake Data Warehouse Data Architecture Architecture 100

Data Engineering Weekly

MARCH 3, 2024

The article explains the challenges with frequent updates in the vanilla data lake setup and how Hudi allows for faster writes and guarantees atomicity, consistency, isolation, and durability (ACID). Part 1: Why did we need to build our own SIEM?

Data Engineering 67

Data Engineering Data Engineer Pipeline-centric Engineering 67

Knowledge Hut

DECEMBER 26, 2023

Columbia University-Financial Engineering and Risk Management Part Overview: This specialization is designed for ambitious learners and professionals who want to develop quantitative finance skills. Create a systematic, data-driven strategy for calculating predicted returns and risks for major asset classes and optimal portfolios.

Certification 95

Certification Java Google Cloud Education 95

Data Engineering Podcast

AUGUST 20, 2021

Summary Data lakes have been gaining popularity alongside an increase in their sophistication and usability. Despite improvements in performance and data architecture they still require significant knowledge and experience to deploy and manage. Can you describe what Cuelake is and the story behind it?

Building 100

Building Data Lake Data Warehouse SQL 100

Cloudera

SEPTEMBER 24, 2021

The Dawn of Telco Big Data: 2007-2012. Reductions in the cost of compute and storage, with efficient appliance based architectures, presented options for understanding more deeply what was actually happening on the network historically, as the first phase of telecom network analytics took shape. Let’s examine how we got here.

Data Architect 91

Data Architect Government NoSQL Big Data 91

Data Engineering Podcast

JULY 1, 2019

In this interview he explains the system design that he tested, his findings for how these tools were able to work together, and how they behaved at different orders of scale. It was an interesting conversation about how he stress tested the Instaclustr managed service for benchmarking an application that has real-world utility.

Kafka 100

Kafka Finance Media Architecture 100

Knowledge Hut

MARCH 13, 2024

Every big company is either eager to implement big data analytics into their business strategies or has already incorporated it into their systems. These large volumes of data are helpful for companies in any sector as nowadays, user data shares equal importance in a company alongside its profits and market share.

Big Data 52

Big Data Data Engineering Data Engineer Engineering 52

Knowledge Hut

MARCH 13, 2024

Every big company is either eager to implement big data analytics into their business strategies or has already incorporated it into their systems. These large volumes of data are helpful for companies in any sector as nowadays, user data shares equal importance in a company alongside its profits and market share.

Big Data 52

Big Data Data Engineering Data Engineer Engineering 52

Data Engineering Podcast

SEPTEMBER 1, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Lake 100

Data Lake Cloud AWS SQL 100

Knowledge Hut

NOVEMBER 17, 2023

In the post, we will investigate how to become an Azure data engineer, the skills required, the roles and responsibilities of an Azure data engineer, and much more. Who is an Azure Data Engineer? They will work with other data specialists to ensure that data solutions are successfully integrated into business processes.

Data Engineering 52

Data Engineering Data Engineer Engineering Scala 52

Data Engineering Podcast

AUGUST 26, 2019

He also explains useful patterns for collaboration between data engineers and data analysts, and what they can learn from each other. We have partnered with organizations such as O’Reilly Media, Dataversity, Corinium Global Intelligence, and Data Counsil. Closing Announcements Thank you for listening!

Building 100

Building Data Analytics Media Data Engineering 100

Data Engineering Podcast

SEPTEMBER 30, 2019

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Pipeline 100

Data Pipeline Machine Learning Media Data 100

Ascend.io

FEBRUARY 28, 2024

The rise of generative AI is changing more than just technology; it’s reshaping our professional landscapes — and yes, data engineering is directly experiencing the impact. How does AI recalibrate the workload and priorities of data teams? ChatGPT screenshot of AI-generated Python code and an explanation of what it means.

Data Engineering 52

Data Engineering Data Engineer Engineering Metadata 52

Data Engineering Podcast

AUGUST 12, 2019

This is a great conversation to listen to for a better understanding of the challenges inherent in synchronizing your data. Upcoming events include the O’Reilly AI Conference, the Strata Data Conference, and the combined events of the Data Architecture Summit and Graphorum. Integration of multiple data sources (e.g.

Media 100

Media Data Warehouse Data Big Data 100

Data Engineering Podcast

SEPTEMBER 18, 2019

This was an eye opening conversation about how stateful computation of data streams from edge devices can reduce cost and complexity as compared to batch oriented workflows. We have partnered with organizations such as O’Reilly Media, Dataversity, Corinium Global Intelligence, and Data Council.

Data Lake 100

Data Lake Hadoop BI Kafka 100

Data Engineering Podcast

SEPTEMBER 22, 2019

He also explains the technical implementation, innovative design, and broad vision for the project. We have partnered with organizations such as O’Reilly Media, Dataversity, Corinium Global Intelligence, and Data Council. Interview Introduction How did you get involved in the area of data management?

AWS 100

AWS Google Cloud Cloud Storage Data Lake 100

Knowledge Hut

NOVEMBER 17, 2023

Their job entails Azure data engineer skills like using big data, databases, data lakes, and analytics to help firms make efficient data-driven decisions. The DP 203 syllabus is designed by incorporating real-world business case studies and activities, you can make sure that your abilities are applicable to the workplace.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Lake 52

Cloudera

DECEMBER 1, 2022

For analytic applications to properly leverage a hybrid, multi-cloud ecosystem to support modern data architectures, data observability has become even more important. RI is a global leader in the design and deployment of large-scale, production-level modern data platforms for the world’s largest enterprises.

Data Lake 81

Data Lake Data Pipeline Data Governance Analytics Application 81