Data Engineering Podcast

APRIL 28, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Go to dataengineeringpodcast.com/dagster today to get started. Your first 30 days are free! Data lakes are notoriously complex.

Building 147

Building Data Lake High Quality Data Machine Learning 147

Snowflake

JUNE 5, 2024

Enterprises can utilize gen AI to extract more value from their data and build conversational interfaces for customer and employee applications. Snowflake Cortex AI is a fully managed service designed to unlock the potential of the technology for everyone within an organization, regardless of their technical expertise.

Coding 85

Coding Building Management Government 85

Data Engineering Podcast

SEPTEMBER 17, 2023

Summary A significant amount of time in data engineering is dedicated to building connections and semantic meaning around pieces of information. Linked data technologies provide a means of tightly coupling metadata with raw information. Go to dataengineeringpodcast.com/materialize today to get 2 weeks free!

Building 189

Building BI SQL Python 189

Knowledge Hut

NOVEMBER 16, 2023

The best part to jump on the bandwagon of information technology or IT is, there is an enormous possibility for an individual if he or she starts studying for a diploma or a degree, does either a master's degree or a research course. JSP certification JSP is a Web page-building tool that allows for dynamic content.

Technology 98

Technology MySQL MongoDB Google Cloud 98

Knowledge Hut

MAY 6, 2024

Applying systems thinking views a system as a set of interconnected and interdependent components defined by its limits and more than the sum of their parts (subsystems). When one component of a system is altered, the effects frequently spread across the entire system. are the main objectives of systems thinking.

Systems 98

Systems Transportation Certification Healthcare 98

Data Engineering Podcast

OCTOBER 15, 2023

Summary Building streaming applications has gotten substantially easier over the past several years. Decodable was built with a mission of eliminating all of the painful aspects of developing and deploying stream processing systems for engineering teams. How can you get the best results for your use case?

Process 182

Process Building SQL BI 182

Data Engineering Podcast

JANUARY 21, 2019

Summary Building internal expertise around big data in a large organization is a major competitive advantage. They discuss the challenges of being at global scale from the start, hiring and training talented engineers, prototyping and deploying new systems in the cloud, and what they have learned in the process.

Big Data 100

Big Data Systems Building Hadoop 100

Data Engineering Podcast

DECEMBER 19, 2021

Summary Building a well managed data ecosystem for your organization requires a holistic view of all of the producers, consumers, and processors of information. The team at Metaphor are building a fully connected metadata layer to provide both technical and social intelligence about your data. No more scripts, just SQL.

Systems 100

Systems Building Metadata Data Warehouse 100

Data Engineering Podcast

JANUARY 28, 2024

Summary Monitoring and auditing IT systems for security events requires the ability to quickly analyze massive volumes of unstructured log data. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. When is Scanner the wrong choice?

Data Lake 147

Data Lake Building High Quality Data AWS 147

Data Engineering Podcast

FEBRUARY 26, 2023

Jean-Georges Perrin was tasked with designing a new data platform implementation at PayPal and wound up building a data mesh. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Are you tired of dealing with the headache that is the 'Modern Data Stack'? We feel your pain.

Building 147

Building Metadata Machine Learning Data Integration 147

Data Engineering Podcast

JULY 27, 2020

Summary A majority of the scalable data processing platforms that we rely on are built as distributed systems. Kyle Kingsbury created the Jepsen framework for testing the guarantees of distributed data processing systems and identifying when and why they break. This brings with it a vast number of subtle ways that errors can creep in.

Systems 100

Systems Building Scala Java 100

Data Engineering Podcast

SEPTEMBER 11, 2022

Summary Data engineering systems are complex and interconnected with myriad and often opaque chains of dependencies. As they scale, the problems of visibility and dependency management can increase at an exponential rate. Interview Introduction How did you get involved in the area of data management?

Systems 100

Systems Metadata Building MongoDB 100

Knowledge Hut

DECEMBER 28, 2023

The information system is a very vast concept that encompasses several aspects like database management, the communication system, various devices, several connections, the internet, collection, organization, and storing data and other information-related applications that are typically used in a business forum.

Systems 52

Systems Telecommunication Technology Certification 52

Knowledge Hut

MAY 20, 2024

Scala is now being used by many companies and individuals to build their own new ideas. Apache Kafka allows the user to publish and subscribe streams of data, just like a messaging system. Yahoo Kafka Manager is also written in Scala and the web console is built using Play Framework.

Scala 52

Scala Technology Kafka Hadoop 52

Knowledge Hut

JULY 7, 2023

Sun Microsystems, the original makers of Java which Oracle later acquired, have kept pace with the advancement in the field of technology and the developers' needs to release new versions and introduce new features. However, some believe some emerging technologies might replace Java in the future.

Java 97

Java Technology Scala Programming Language 97

Knowledge Hut

MARCH 22, 2024

I have comprehensively analyzed the area of physical security, particularly the ongoing discussion surrounding fail safe vs fail-safe secure electric strike locking systems. These two distinct approaches offer unique strategies for securing buildings and valuables. What are Fail-Safe Door Locks?

Systems 105

Systems Electronics Hospitality Architecture 105

Data Engineering Podcast

AUGUST 27, 2023

Summary Data persistence is one of the most challenging aspects of computer systems. In the era of the cloud most developers rely on hosted services to manage their databases, but what if you are a cloud service? What are the different methods that you have used for managing database instances? Why Postgres?

Database 130

Database Building PostgreSQL BI 130

Knowledge Hut

JANUARY 23, 2024

Blockchain technology in agriculture improves food safety by allowing information to be traced across the food supply chain. The capacity of blockchain to store and manage data enables traceability, which is utilized to aid in creating and implementing technologies for intelligent farming and index-based crop insurance.

Technology 52

Technology Food Insurance Data Collection 52

Data Engineering Podcast

JULY 5, 2021

Summary At the core of every data pipeline is an workflow manager (or several). Deploying, managing, and scaling that orchestration can consume a large fraction of a data team’s energy so it is important to pick something that provides the power and flexibility that you need.

Systems 100

Systems Management Data Warehouse Programming Language 100

Snowflake

OCTOBER 2, 2023

The case for business continuity is particularly compelling for a company such as The Depository Trust & Clearing Corporation (DTCC) , which is designated as a systemically important financial market utility (SIFMU), a U.S. We have conducted at least 15 disaster recovery exercises using Snowgrid technology for business continuity.

AWS 102

AWS Technology Data Cloud 102

Knowledge Hut

APRIL 16, 2024

Over several decades, projects have been initiated or undertaken due to market demands, business needs, at the behest of customer request, technological advancements and to comply with regulatory requirements. We will look at some of the key project management methods followed in today’s world. Know more about the project description.

Project 98

Project Management Programming Designing 98

Knowledge Hut

MAY 2, 2024

IT managers are often bound by limitations and reach a wall with remote working environments. While working remotely is beneficial with benefits such as increased employee productivity, satisfaction levels and is known to increase the company income it also comes with a few challenges to the IT managers. What is working remotely?

IT 98

IT Management Project Building 98

Data Engineering Podcast

JUNE 11, 2023

Summary Data engineering is all about building workflows, pipelines, systems, and interfaces to provide stable and reliable data. In this episode Gleb Mezhanskiy shares some valuable advice and insights into how you can build reliable and well-tested data assets with dbt and data-diff.

Project 130

Project Building Data Lake Machine Learning 130

Data Engineering Podcast

JANUARY 15, 2023

Summary The modern data stack has made it more economical to use enterprise grade technologies to power analytics at organizations of every scale. Unfortunately it has also introduced new overhead to manage the full experience as a single workflow. Go to dataengineeringpodcast.com/gartnerda today to find out more.

Coding 130

Coding Building PostgreSQL Data Lake 130

Data Engineering Podcast

NOVEMBER 20, 2022

In this episode Ian Schweer shares his experiences at Riot Games supporting player-focused features such as machine learning models and recommeder systems that are deployed as part of the game binary. The biggest challenge with modern data systems is understanding what data you have, where it is located, and who is using it.

Systems 130

Systems Metadata Data Pipeline MongoDB 130

Data Engineering Podcast

DECEMBER 25, 2022

Recognizing this shortcoming and the capabilities that could be unlocked by a robust solution Rishabh Poddar helped to create Opaque Systems as an outgrowth of his PhD studies. RudderStack helps you build a customer data platform on your warehouse or data lake. Sign up free at dataengineeringpodcast.com/rudder Build Data Pipelines.

Machine Learning 130

Machine Learning Systems Data Lake Data Warehouse 130

Data Engineering Podcast

APRIL 16, 2023

As the capabilities of these systems has improved and become more accessible, the target of what self-serve means changes. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake.

Business Intelligence 147

Business Intelligence Building Data Lake BI 147

Knowledge Hut

DECEMBER 27, 2023

The job of a business systems analyst is a uniquely cross-functional one that plays a critical role in improving a company’s processes for better operational efficiency. What is Business Systems Analyst? Business systems analyst duties also include writing instruction manuals and analyzing costs and benefits related to technology.

Systems 52

Systems Business Analyst Consulting Certification 52

Data Engineering Podcast

JUNE 2, 2024

Summary Modern businesses aspire to be data driven, and technologists enjoy working through the challenge of building data systems to support that goal. Nicola Askham found her way into data governance by accident, and stayed because of the benefit that she was able to provide by serving as a bridge between the technology and business.

Data Governance 130

Data Governance Government Data Lake High Quality Data 130

Knowledge Hut

DECEMBER 26, 2023

By facilitating collaboration and value exchange amongst users of the same platform but with no other connections, blockchain technology takes it to a new level. The main benefits and drawbacks of blockchain technology will be covered in this blog. Distributed ledger technology, which manages its decentralized database, is used (DLT).

Technology 52

Technology Food Manufacturing Banking 52

Data Engineering Podcast

FEBRUARY 20, 2022

Summary The life sciences as an industry has seen incredible growth in scale and sophistication, along with the advances in data technology that make it possible to analyze massive amounts of genomic information. Interview Introduction (see Guy’s bio below) How did you get involved in the area of data management?

Systems 100

Systems Software Engineer Software Engineering Data Warehouse 100

DoorDash Engineering

SEPTEMBER 19, 2023

This success has been heavily influenced by the strength and leadership of our Eng management team; great companies are built by great people who have great managers. First, we spent time talking to current Engineering managers and identifying which existing expectations were still relevant, and which ones were no longer serving us.

Management 81

Management Engineering Programming Technology 81

Edureka

MAY 5, 2023

MERN Stack Technology is one of the most important and latest technology stack that developers and programmers use today to build high end applications and websites. In this blog we are going to learn about MERN Stack, its benefits and why today developers require MERN Stack Technologies. MERN Stack MEAN Stack MEVN Stack.

Technology 52

Technology MongoDB Programming Language Database 52

Knowledge Hut

NOVEMBER 19, 2023

With advances in technology, wearable devices do provide some trace of your health. Furthermore, Data Scientists analyze this data, build reports and train ML models on top of it. Imagine an industrial system where Temperature and Humidity need to be within a range and any potential anomaly needs to be detected and flagged.

Data Science 52

Data Science Technology Machine Learning Big Data 52

Knowledge Hut

DECEMBER 5, 2023

If you have a passion for information technology (IT) and dream of turning it into a fulfilling career, ponder the path of a systems engineer. Join us on a detailed exploration of who can pursue a career as a systems engineer and the steps to become one in the year 2024. Who is a System Engineer, and What Do They Do?

Systems 52

Systems Engineering Recruitment Electronics 52

Data Engineering Podcast

JULY 27, 2021

The technology to make that possible has been around for a number of years, but the barriers to adoption have still been high due to the level of technical understanding and operational capacity that have been required to run at scale. Go to dataengineeringpodcast.com/97things today to get your copy!

Building 100

Building Management Kafka Data Warehouse 100

Cloudera

DECEMBER 1, 2023

Recent Government Initiatives on Public Sector AI Solutions In recent years, governments across the globe have recognized the transformative potential of artificial intelligence (AI) and have embarked on initiatives to harness this technology to drive innovation and serve their citizens more effectively. million), among others.

Building 112

Building Government Transportation Data Governance 112