Data Engineering Podcast

OCTOBER 15, 2023

Despite this, it is still operationally challenging to deploy and maintain your own stream processing infrastructure. Decodable was built with a mission of eliminating all of the painful aspects of developing and deploying stream processing systems for engineering teams. Check out the agenda and register today at Neo4j.com/NODES.

Process 182

Process Building SQL BI 182

Data Engineering Podcast

JANUARY 7, 2024

Summary Data processing technologies have dramatically improved in their sophistication and raw throughput. Unfortunately, the volumes of data that are being generated continue to double, requiring further advancements in the platform capabilities to keep up. What do you have planned for the future of your academic research?

Data Process 147

Data Process Process Data Lake High Quality Data 147

Striim

DECEMBER 12, 2023

Exciting news is on the horizon as Striim proudly announces its Technology Partnership with YugabyteDB, a collaboration set to reshape the landscape of data management. As we embark on this thrilling journey, we share a vision of empowering organizations with the tools they need to thrive in a data-driven world.

Data Management 59

Data Management Database-centric Management PostgreSQL 59

Data Engineering Podcast

JUNE 8, 2020

Summary The landscape of data management and processing is rapidly changing and evolving. This is a useful conversation to gain a macro perspective on where businesses are looking to improve their capabilities to work with data. If you hand a book to a new data engineer, what wisdom would you add to it?

Data Management 100

Data Management Management Machine Learning Portfolio 100

The Modern Data Company

MAY 10, 2023

In our previous post, The Pros and Cons of Leading Data Management and Storage Solutions , we untangled the differences among data lakes, data warehouses, data lakehouses, data hubs, and data operating systems. What factors are most important when building a data management ecosystem?

Data Management 52

Data Management Management Data Lake Data Warehouse 52

The Modern Data Company

MAY 10, 2023

In our previous post, The Pros and Cons of Leading Data Management and Storage Solutions , we untangled the differences among data lakes, data warehouses, data lakehouses, data hubs, and data operating systems. What factors are most important when building a data management ecosystem?

Data Management 52

Data Management Management Data Lake Data Warehouse 52

LinkedIn Engineering

JULY 19, 2023

End-users create relational entities in the form of Tables over structured or semi-structured data using compute engines, with the metadata for a Table stored in a catalog, and data stored in distributed storage. While functional, our current setup for managing tables is fragmented. retention, replication), optimal (e.g.,

Big Data 78

Big Data Data Management Management Metadata 78

The Modern Data Company

MAY 10, 2023

In our previous post, The Pros and Cons of Leading Data Management and Storage Solutions , we untangled the differences among data lakes, data warehouses, data lakehouses, data hubs, and data operating systems. What factors are most important when building a data management ecosystem?

Data Management 52

Data Management Data Lake Management Data Warehouse 52

The Modern Data Company

MAY 8, 2023

Data lakes, data warehouses, data hubs, data lakehouses, and data operating systems are data management and storage solutions designed to meet different needs in data analytics, integration, and processing. This feature allows for a more flexible exploration of data.

Data Management 52

Data Management Data Lake Management Data Governance 52

The Modern Data Company

MAY 8, 2023

Data lakes, data warehouses, data hubs, data lakehouses, and data operating systems are data management and storage solutions designed to meet different needs in data analytics, integration, and processing. This feature allows for a more flexible exploration of data.

Data Management 52

Data Management Data Lake Management Data Governance 52

The Modern Data Company

MAY 8, 2023

Data lakes, data warehouses, data hubs, data lakehouses, and data operating systems are data management and storage solutions designed to meet different needs in data analytics, integration, and processing. This feature allows for a more flexible exploration of data.

Data Management 52

Data Management Data Lake Management Data Governance 52

Data Engineering Podcast

NOVEMBER 16, 2020

In this episode CEO and co-founder of Isima Darshan Rawal explains how the biOS platform is architected to enable ease of use, the challenges that were involved in building an entirely new system from scratch, and how it can integrate with the rest of your data platform to allow for incremental adoption. What is the story behind the name?

Data Management 100

Data Management Management BI Business Intelligence 100

Ascend.io

AUGUST 15, 2023

This same principle holds true in data management. It’s no surprise, then, that the quest for Fivetran alternatives is on the rise as organizations set their sights on a more holistic data approach. Defense: Saving Money with Intelligent Data Refresh In football, a solid defense does more than just stop goals.

Data Management 52

Data Management Management Data Ingestion Data Pipeline 52

Robinhood

FEBRUARY 7, 2024

When dealing with large-scale data, we turn to batch processing with distributed systems to complete high-volume jobs. In this blog, we explore the evolution of our in-house batch processing infrastructure and how it helps Robinhood work smarter. Why Batch Processing is Integral to Robinhood Why is batch processing important?

Process 75

Process Hadoop Architecture Accessible 75

Precisely

OCTOBER 2, 2023

Manual, error-prone SAP data processes simply don’t cut it anymore. Automating the processes that create and maintain the vast amounts of interdependent data that support your SAP ERP business processes is key to gaining agility, speed, and improved data quality and integrity. Automation.

Process 64

Process Finance Government Data Management 64

ProjectPro

FEBRUARY 16, 2023

ETL is a critical component of success for most data engineering teams, and with teams harnessing it with the power of AWS, the stakes are higher than ever. Data Engineers and Data Scientists require efficient methods for managing large databases, which is why centralized data warehouses are in high demand.

AWS 52

AWS Data Management ETL Tools Management 52

Towards Data Science

NOVEMBER 16, 2023

Data Management A tutorial on how to use VDK to perform batch data processing Photo by Mika Baumeister on Unsplash Versatile Data Ki t (VDK) is an open-source data ingestion and processing framework designed to simplify data management complexities.

Data Process 79

Data Process Process Raw Data Data 79

Data Engineering Podcast

SEPTEMBER 3, 2018

Summary With the proliferation of data sources to give a more comprehensive view of the information critical to your business it is even more important to have a canonical view of the entities that you care about. Can you start by establishing a definition of data mastering that we can work from?

Data Management 100

Data Management Management Relational Database Business Intelligence 100

Cloudera

FEBRUARY 4, 2021

Financial services institutions need the ability to analyze and act on massive volumes of data from diverse sources in order to monitor, model, and manage risk across the enterprise. They need a comprehensive data and analytics platform to model risk exposures on-demand. End-to-end Data Lifecycle.

Technology 90

Technology Data Management Management Insurance 90

Precisely

DECEMBER 11, 2023

That has led to a wave of process automation initiatives aimed at streamlining business processes, improving operational efficiency, and increasing accuracy. Their leaders understand that effective data management is pivotal for efficiency, and that process automation can deliver a win-win on both cost and quality.

Process 59

Process Finance Data Validation Utilities 59

Knowledge Hut

MARCH 27, 2024

In the world of project management, adherence to PMP project compliance standards is not just a requirement but a cornerstone of success. Adhering to compliance in project management is a critical aspect that must not be overlooked. In my IT project management experience, ensuring project compliance was crucial.

Project 52

Project Process Government Management 52

Edureka

AUGUST 2, 2023

Raw data, however, is frequently disorganised, unstructured, and challenging to work with directly. Data processing analysts can be useful in this situation. Let’s take a deep dive into the subject and look at what we’re about to study in this blog: Table of Contents What Is Data Processing Analysis?

Data Process 52

Data Process Process Data Cleanse Data Mining 52

Precisely

JULY 25, 2023

Organizations that run SAP can use Excel-to-SAP automation to do more with less, while also increasing agility and improving their SAP master data management process automation. We bring automation closer to the business users who own the data and the day-to-day processes that drive the business.

Data Process 52

Data Process Process Data Data Integration 52

The Modern Data Company

APRIL 7, 2023

It is important to note that data debt has methodological and technological components. Data management technologies grow old and out of date. The longer an organization ignores its data debt, the worse it will become. Slowed processing also results in frustrated users who stop using the data.

Management 98

Management Government Data Data Management 98

Data Engineering Podcast

NOVEMBER 20, 2021

With the improvements in streaming engines it is now possible to perform all of your data integration in near real time, but it can be challenging to understand the proper processing patterns to make that performant. Can you start by giving an overview of the state of the market for data lakes today?

Data Lake 100

Data Lake Data Integration Lambda Architecture Process 100

Data Science Blog: Data Engineering

SEPTEMBER 9, 2020

The tool successfully adheres to the importance of keeping test-data centric in Automation Test solutions. To start with, organizing and managing test data is very easy in TestProject. We are aware that as an application gets bigger and more tests are added, test data management becomes more difficult.

Data Management 52

Data Management Database-centric Management PostgreSQL 52

Data Engineering Podcast

SEPTEMBER 24, 2021

In this episode Ehsan Totoni explains how he built the Bodo project to bring the speed and processing power of HPC techniques to the Python data ecosystem without requiring any re-work. What are the techniques/technologies that teams might use to optimize or scale out their data processing workflows?

Data Process 100

Data Process Python Process Data Lake 100

Knowledge Hut

APRIL 23, 2024

We live in a data-driven culture where familiarity with databases is crucial. Database management is crucial for businesses of all sizes to guarantee that their data is complete, safe, and easily available when needed. Why are Database Management Skills Important? In this section, we'll examine every skill in depth.

Database 52

Database Management Relational Database SQL 52

Data Engineering Podcast

MARCH 2, 2020

The ksqlDB project was created to address this state of affairs by building a unified layer on top of the Kafka ecosystem for stream processing. You listen to this show to learn and stay up to date with what’s happening in databases, streaming platforms, big data, and everything else you need to know about modern data management.

Kafka 100

Kafka Process PostgreSQL MySQL 100

Precisely

FEBRUARY 17, 2023

SAP does a great job managing all of that complexity, but it takes many resources to maintain master data, update SAP information on a timely basis, and keep everything running smoothly. As this drive toward increased efficiency and agility continues, here are the trends that we see unfolding in 2023 for automating SAP processes.

Process 52

Process Technology Programming Data Management 52

Knowledge Hut

JULY 24, 2023

The future of SQL (Structured Query Language) is a scalding subject among professionals in the data-driven world. As data generation continues to skyrocket, the demand for real-time decision-making, data processing, and analysis increases. According to recent studies, the global database market will grow from USD 63.4

SQL 52

SQL Database Process NoSQL 52

Data Engineering Podcast

NOVEMBER 27, 2022

Summary The data ecosystem has been growing rapidly, with new communities joining and bringing their preferred programming languages to the mix. This has led to inefficiencies in how data is stored, accessed, and shared across process and system boundaries. What do you have planned for the future of Arrow?

Data Process 100

Data Process Process Metadata Business Intelligence 100

Data Engineering Podcast

JULY 19, 2021

In this episode Gleb Mezhanskiy shares some strategies for adding quality checks at every stage of your development and deployment workflow to identify and fix problematic changes to your data before they get to production. What are the biggest factors that you see contributing to data quality issues?

Management 100

Management Data Warehouse Data Pipeline Data 100

Knowledge Hut

MARCH 22, 2024

Product management is a highly engaging and rewarding role - product managers are responsible for developing new products that not only delight your customers but also help with business growth. Who is a Product Manager? Who is a product manager and what is the role of a product manager?

Management 52

Management Education Certification Programming 52

Rockset

MARCH 21, 2019

When we started Rockset, we envisioned building a powerful cloud data management system that was really easy to use. Making the data stack simpler is fundamental to making data usable by developers and data scientists. The data management should feel limitless.

SQL 52

SQL Data Management Management Engineering 52

Data Engineering Podcast

MARCH 15, 2021

In this episode the head of data platform for DoorDash, Sudhir Tonse, discusses the technologies that they are using, the approach that they take to adding new systems, and how they think about priorities for what to support for the whole company vs what to leave as a specialized concern for a single team.

Management 100

Management Data Warehouse PostgreSQL Kafka 100

Hevo

DECEMBER 29, 2023

Increased reliance on data brings many responsibilities and challenges, like data management, security, and maximizing the data potential. This is where Data Lifecycle Management (DLM) comes into play. What is Data Lifecycle Management? During this process, […]

Management 40

Management Data Data Management Process 40