Christophe Blefari

FEBRUARY 23, 2024

My ideas these days ( credits ) Hey, fresh Data News edition. This week I've participated to a round table about data and did a cool presentation about Engines. — Apache Arrow is an awesome library that powers a lot of innovations in the data space in the recent years. Is the modern data stack disappearing?

Data Lake 130

Data Lake PostgreSQL MongoDB Scala 130

KDnuggets

AUGUST 22, 2019

Fast growing technology companies like Uber or LinkedIn have been forced to build their own in-house data lifecycle management solutions to power different groups of machine learning models.

Machine Learning 55

Machine Learning Management Data Management Datasets 55

Christophe Blefari

JULY 22, 2023

Also be ready for the Data News: Summer Edition. What if we conceive data as a game for our consumers and not as a linear tool to do boring actions. What if we conceive data as a game for our consumers and not as a linear tool to do boring actions. Classically boring stuff. It reminds me something.

Data 130

Data Database Big Data Data Engineering 130

Data Engineering Podcast

FEBRUARY 19, 2023

Summary Cloud data warehouses have unlocked a massive amount of innovation and investment in data applications, but they are still inherently limiting. Because of their complete ownership of your data they constrain the possibilities of what data you can store and how it can be used. We feel your pain.

IT 147

IT Data Lake Metadata Data Warehouse 147

DataKitchen

OCTOBER 20, 2023

In today’s data-driven landscape, Data and Analytics Teams i ncreasingly face a unique set of challenges presented by Demanding Data Consumers who require a personalized level of Data Observability. Data Observability platforms often need to deliver this level of customization.

Insurance 67

Insurance Pharmaceutical Data Data Ingestion 67

Data Engineering Podcast

AUGUST 6, 2021

Summary Every organization needs to be able to use data to answer questions about their business. The trouble is that the data is usually spread across a wide and shifting array of systems, from databases to dashboards. Are you bored with writing scripts to move data into SaaS tools like Salesforce, Marketo, or Facebook Ads?

Database 100

Database SQL BI Business Intelligence 100

Pinterest Engineering

NOVEMBER 7, 2023

Jeff Xiang | Software Engineer, Logging Platform Vahid Hashemian | Software Engineer, Logging Platform Jesus Zuniga | Software Engineer, Logging Platform At Pinterest, data is ingested and transported at petabyte scale every day, bringing inspiration for our users to create a life they love.

Kafka 99

Kafka Java Software Engineer Software Engineering 99

Precisely

MAY 13, 2024

Key Takeaways: Data integration is vital for real-time data delivery across diverse cloud models and applications, and for leveraging technologies like generative AI. The right data integration solution helps you streamline operations, enhance data quality, reduce costs, and make better data-driven decisions.

Data Integration 52

Data Integration Metadata Amazon Web Services Data Governance 52

Mutt Data

OCTOBER 25, 2023

Common Questions When it comes to discoveries many companies have questions for us. After reading this post you’ll be able to answer the following questions: What Is A Discovery Process? Why Should We Carry Out A Discovery Process? What Deliverables Can We Expect From Our Discovery? How Long Does A Discovery Take?

Architecture 45

Architecture Project Data Architecture Technology 45

Snowflake

JANUARY 26, 2024

A key benefit of the Snowflake Data Cloud is the elimination of data silos. To support this, the Data Cloud was designed to provide customers with the same product experience—including security and governance capabilities — across multiple cloud regions with the three major cloud providers: AWS, Azure, and Google Cloud.

Google Cloud 86

Google Cloud AWS Government Cloud 86

Snowflake

APRIL 29, 2024

Identification of potential threats through data analysis and risk assessment processes to shift toward more proactive risk management. This strategy must incorporate business impact analyses as well as backup and recovery plans in the event of a security incident or loss of access to data. or better) for data in transit.

Transportation 84

Transportation Data Governance Government Consulting 84

François Nguyen

MARCH 22, 2021

TL;DR After setting up and organizing the teams, we are describing 4 topics to make data mesh a reality. TL;DR After setting up and organizing the teams, we are describing 4 topics to make data mesh a reality. How do we build data products ? How can we interoperate between the data domains ?

Architecture 130

Architecture Technology Google Cloud Metadata 130

Data Engineering Weekly

APRIL 7, 2024

Poor data quality and unlcear data ownership remains the top challenges for the data teams. Data Mesh continuously gaining popularity among the enterprises. It is a stark difference from the Gartner report about data mesh. I guess only the time will tell who wins in the data mesh vs data fabric war.

Data Engineering 70

Data Engineering Data Engineer Engineering Unstructured Data 70

Data Engineering Podcast

AUGUST 24, 2020

Summary In order to scale the use of data across an organization there are a number of challenges related to discovery, governance, and integration that need to be solved. The key to those solutions is a robust and flexible metadata management system. If you hand a book to a new data engineer, what wisdom would you add to it?

Metadata 100

Metadata Management Kafka Data Engineering 100

Data Engineering Podcast

JULY 31, 2022

Summary Data lineage is the roadmap for your data platform, providing visibility into all of the dependencies for any report, machine learning model, or data warehouse table that you are working with. Atlan is the metadata hub for your data ecosystem.

IT 100

IT Metadata MongoDB MySQL 100

Jesse Anderson

APRIL 13, 2022

Data mesh is quickly becoming a way for companies to roll out their data strategy. If you haven’t already learned about data mesh , I suggest doing so. I think a crucial part of your data mesh revolves around the choice of publish/subscribe technologies. At the crux of data mesh is a desire for flexibility.

Kafka 124

Kafka Data Cloud Technology 124

Knowledge Hut

APRIL 29, 2024

Utilizing methods and procedures created to safeguard the data is essential for a successful digital revolution. Between June 2018 and March 2022, Indian banks reported 248 successful data breaches by hackers and criminals; the government notified Parliament on Aug 2, 2022. The market value reached $38.72 and a value of $195.5

Banking 98

Banking Government Media Electronics 98

Monte Carlo

DECEMBER 13, 2023

You can’t turn around in the data space without running into hot takes about GenAI. What do data leaders say when the cameras aren’t rolling and the board decks aren’t public? What do data leaders say when the cameras aren’t rolling and the board decks aren’t public? Will it make our jobs obsolete?

BI 59

BI Data Metadata Technology 59

Christophe Blefari

DECEMBER 16, 2022

Prepping me to deliver Christmas' Data News ( credits ) Hey you, the end of the year is coming soon. As a small reminder the Advent of Data 🎄 is still running and this week we got awesome articles again! and How to get started with data and help your local community. I really liked this year with you.

Kafka 130

Kafka Data SQL Cloud 130

Knowledge Hut

JANUARY 29, 2024

Some techniques add to the development of technology in the business sectors, including Data Science and Cloud Computing, essential aspects of the technology industry. With the help of data science, one can gather all the critical analyses from vast chunks of data stored in clouds. In this model, the data is not 100% secure.

Cloud Computing 98

Cloud Computing Data Science Cloud Amazon Web Services 98

Databand.ai

JULY 19, 2023

Data Lineage Tools: Key Capabilities and 5 Notable Solutions Ryan Yackel July 19, 2023 What Are Data Lineage Tools? Data lineage tools provide a visual representation of your data’s journey across multiple systems and transformations. Data lineage tools are not a new concept.

Pipeline-centric 52

Pipeline-centric Data Governance Metadata Government 52

Knowledge Hut

APRIL 25, 2024

Big data can be summed up as a sizable data collection comprising a variety of informational sets. It is a vast and intricate data set. Big data has been a concept for some time, but it has only just begun to change the corporate sector. What is Big Data? What are the Benefits of Big Data?

Big Data 97

Big Data Entertainment Transportation Banking 97

Data Engineering Podcast

MARCH 27, 2022

Summary Data governance is a practice that requires a high degree of flexibility and collaboration at the organizational and technical levels. The growing prominence of cloud and hybrid environments in data management adds additional stress to an already complex endeavor.

Data Governance 130

Data Governance Government Cloud Building 130

Monte Carlo

DECEMBER 13, 2023

You can’t turn around in the data space without running into hot takes about GenAI. What do data leaders say when the cameras aren’t rolling and the board decks aren’t public? What do data leaders say when the cameras aren’t rolling and the board decks aren’t public? Will it make our jobs obsolete?

BI 52

BI Data Metadata Technology 52

Monte Carlo

JUNE 22, 2023

At our latest IMPACT summit, Roche shared their data mesh strategy for creating reliable data products. Here are the session takeaways: Roche , is one of the world’s largest biotech companies, as well as a leading provider of in-vitro diagnostics and a global supplier of transformative innovative solutions across major disease areas.

Building 52

Building Data Ingestion Data Business Analyst 52

Knowledge Hut

AUGUST 1, 2023

Machine learning algorithms can analyze potentially tera bytes of data, identify patterns from these data, and make predictions or decisions. They together empower the industry with quality data analytics, live monitoring capabilities, and customized healthcare solutions.

Machine Learning 52

Machine Learning Medical Healthcare Hospitality 52

Monte Carlo

FEBRUARY 6, 2023

So, you’re planning a cloud data warehouse migration. As you probably already know if you’re reading this, a data warehouse migration is the process of moving data from one warehouse to another. In the old days, data warehouses were bulky, on-prem solutions that were difficult to build and equally difficult to maintain.

Data Warehouse 52

Data Warehouse AWS Data Validation Data 52

Lyft Engineering

OCTOBER 3, 2023

High-quality data is necessary for the success of every data-driven company. It is now the norm for tech companies to have a well-developed data platform. This makes it easy for engineers to generate, transform, store, and analyze data at the petabyte scale. What and Where is Data Quality?

Big Data 84

Big Data Metadata Data Warehouse Data 84

Knowledge Hut

MAY 6, 2024

Systematic discovery of the system's dynamics, limitations, conditions, and illuminating principles (purpose, measure, methods, tools, etc.) It is said that ‘today’s problems are yesterday’s solutions.’ When one component of a system is altered, the effects frequently spread across the entire system.

Systems 98

Systems Transportation Certification Healthcare 98

Knowledge Hut

NOVEMBER 2, 2023

Azure Data engineering projects are complicated and require careful planning and effective team participation for a successful completion. While many technologies are available to help data engineers streamline their workflows and guarantee that each aspect meets its objectives, ensuring that everything works properly takes time.

Data Engineering 52

Data Engineering Data Engineer Coding Project 52

ProjectPro

FEBRUARY 8, 2023

Did you know thousands of businesses, including over 80% of the Fortune 100, use Apache Kafka to modernize their data strategies? Apache Kafka is the most widely used open-source stream-processing solution for gathering, processing, storing, and analyzing large amounts of data. What is Apache Kafka Used For?

Kafka 52

Kafka Banking Medical Healthcare 52

Acceldata

FEBRUARY 10, 2023

Learn the four ways that data discovery is a better solution that a data catalog.

Data 52

Data 52

Knowledge Hut

JANUARY 3, 2024

Soundraw Soundraw , as the name suggests, is a generative AI solution specifically curated for music composition by Daigo Kusunoki. The core of Synthesia’s technology relies on advanced AI models trained on vast amounts of data to understand and generate human-like video content. It was founded in the year 2014 by Alex Zhavoronkov.

Healthcare 52

Healthcare Finance Food Algorithm 52

Data Engineering Weekly

APRIL 30, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack provides data pipelines that make collecting data from every application, website, and SaaS platform easy, then activating it in your warehouse and business tools. DoorDash writes about its discovery of Generative AI and its application to boost its business.

Data Engineering 62

Data Engineering Data Engineer Engineering PostgreSQL 62

Knowledge Hut

JANUARY 30, 2024

Control of Service Performance Data : This practice aims to hold the ITSM vendors responsible for the performance issue. Naturally, the answers to all these questions lie with the Service delivery data. Clients need to ask the vendors for this performance data, otherwise, they can lose visibility of it.

Certification 87

Certification Technology Finance Management 87

Data Engineering Podcast

DECEMBER 21, 2020

Summary Data governance is a term that encompasses a wide range of responsibilities, both technical and process oriented. One of the more complex aspects is that of access control to the data assets that an organization is responsible for managing. If you hand a book to a new data engineer, what wisdom would you add to it?

Data Governance 130

Data Governance Government Data Lake Banking 130

DoorDash Engineering

AUGUST 1, 2023

Just when we most needed observability data, the system would leave us in the lurch. In this article, we explain how we accomplished the migration, first reviewing the challenges we faced with StatsD and then discussing how we selected and implemented Prometheus as our preferred solution.

AWS 82

AWS Transportation Programming Language Government 82