Data Engineering Weekly

NOVEMBER 26, 2023

It’s the end of the year, and there will be a lot of buzz about what the next five years in data engineering might bring. 🕵️‍♂️📈 Before we delve into the patterns, it's important to remember that data infrastructure maturity model. Who knows what insights we might uncover?

Data Engineering 70

Data Engineering Data Engineer Engineering Kafka 70

Confluent

JANUARY 30, 2024

We’re celebrating excellence across the Data Streaming ecosystem. In this blog post, we announce the global and regional categories that Confluent will recognize across its 2023 Partner of the Year award winners.

IT 92

IT Data 92

Knowledge Hut

APRIL 25, 2024

Big data in information technology is used to improve operations, provide better customer service, develop customized marketing campaigns, and take other actions to increase revenue and profits. It is especially true in the world of big data. It is especially true in the world of big data. What Are Big Data T echnologies?

Big Data 98

Big Data Technology NoSQL Hadoop 98

Cloudera

OCTOBER 18, 2021

We are thrilled to announce the finalists of the 2021 Data Impact Awards. This year’s entrants have excelled at demonstrating how innovative data solutions can help solve real-time challenges and positively impact people around the world. . Data for Enterprise AI . Read more about the Data for Enterprise AI category here .

Banking 109

Banking Transportation Government Finance 109

Knowledge Hut

APRIL 26, 2024

Machine Learning is a sub-branch of Artificial Intelligence, used for the analysis of data. It learns from the data that is input and predicts the output from the data rather than being explicitly programmed. If ML algorithms consume more amounts of data while solving a problem, it results in better performance.

Machine Learning 98

Machine Learning Algorithm Deep Learning Programming Language 98

Data Engineering Weekly

MAY 16, 2023

In the first part of this series, we talked about design patterns for data creation and the pros & cons of each system from the data contract perspective. In the second part, we will focus on architectural patterns to implement data quality from a data contract perspective. Why is Data Quality Expensive?

Engineering 110

Engineering Kafka Data Pipeline Data Warehouse 110

Scott Logic

MAY 15, 2024

Introduction In this blog post I’ll show you how to create graph database queries using a large language model (LLM) and very minimal query language knowledge. Graph databases are extremely useful when trying to query your data and find complex insights with a lot simpler form of querying (It won’t seem like this at first though, I am sure).

Database 52

Database Entertainment Relational Database Coding 52

Knowledge Hut

FEBRUARY 28, 2024

It is not feasible to analyze and interpret large amounts of data manually. Data visualization has become vital to understanding the various trends present in the data – both visible, as well as hidden trends.  It also helps you better understand the relationships between your data.

BI 52

BI Business Intelligence Data Data Science 52

Databand.ai

AUGUST 30, 2023

Data testing tools: Key capabilities you should know Helen Soloveichik August 30, 2023 Data testing tools are software applications designed to assist data engineers and other professionals in validating, analyzing and maintaining data quality. There are several types of data testing tools.

Data Cleanse 52

Data Cleanse Data Pipeline Datasets Data Validation 52

Data Engineering Podcast

SEPTEMBER 4, 2022

Summary Data observability is a product category that has seen massive growth and adoption in recent years. Monte Carlo is in the vanguard of companies who have been enabling data teams to observe and understand their complex data systems. Atlan is the metadata hub for your data ecosystem.

IT 100

IT Metadata MongoDB MySQL 100

Cloudera

JUNE 29, 2021

There have been many points in history where society has been forced to reflect on the expectations of diversity and inclusion and as mentioned in our “ Embracing the conversation ” blog post earlier this year, the last 18 months have presented a real opportunity for change. We the people are the data . The People.

Recruitment 84

Recruitment Data Programming Accessible 84

Cloudyard

MARCH 4, 2024

Read Time: 1 Minute, 32 Second In this blog post, we will explore how to leverage Snowpark and Streamlit to build an interactive book exploration application. The combination of Snowpark and Streamlit provides a powerful platform for data exploration and visualization. Get personalized recommendations: Select a specific category (e.g.,

Programming Language 52

Programming Language Python Programming Coding 52

Pinterest Engineering

MAY 9, 2023

Erika Sun ML Engineer | Advertiser Growth Modeling Team; Ogheneovo Dibie Engineering Manager | Advertiser Growth Modeling Team Photo by Jason Blackeye on Unsplash Summary In this blog post, we describe a Machine Learning (ML) powered proactive churn prevention solution that was prototyped with our small & medium business (SMB) advertisers.

Consulting 81

Consulting Data Science Architecture Engineering 81

Cloudera

JANUARY 18, 2022

Each year, the Cloudera Data Impact Awards recognize organizations that have accomplished amazing things with innovative data solutions. . For 2021, the awards will include a new category: People First. As a result of this innovative data solution, the company helped customers while keeping its default rate low. .

Banking 94

Banking Recruitment Data Lake Data 94

Cloudera

JANUARY 26, 2024

In this article, we will walk you through the process of implementing fine grained access control for the data governance framework within the Cloudera platform. In a good data governance strategy, it is important to define roles that allow the business to limit the level of access that users can have to their strategic data assets.

Metadata 73

Metadata Data Governance Government Management 73

Data Engineering Weekly

MARCH 26, 2023

Contribute to the Rudderstack Transformations Library, Win $1000 RudderStack Transformations lets you customize event data in real time with your own JavaScript or Python code. dbt: State of Analytics Engineering dbt publishes the state of analytical [data???🤔] Go to rtasummit.com and register with DEW30 for 30% off.

Data Engineering 52

Data Engineering Data Engineer Engineering Lambda Architecture 52

Cloudera

DECEMBER 11, 2020

The Data Security and Governance category, at the annual Data Impact Awards, has never been so important. The sudden rise in remote working, a huge influx in data as the world turned digital, not to mention the never-ending list of regulations businesses need to remain compliant with (how many acronyms can you name in full?

Data Lake 98

Data Lake Government Data Security Unstructured Data 98

RandomTrees

FEBRUARY 2, 2024

GPT-based data engineering accelerators make the working of data more accessible. These accelerators use GPT models to do data tasks faster, fix any issues, and save a lot of time. GPT models change data in simple language and also provide summaries and explanations. It is the first AI-powered conversational data analyst.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Scott Logic

MARCH 5, 2024

You can read more about it in our previous blog post: Announcing the (proposed) Technology Carbon Standard. Some of the key additions include: An overview of cloud services and their environmental impact in the Operational Emissions category. The embodied carbon emissions from device production are also discussed.

Technology 52

Technology Transportation Manufacturing Cloud 52

DoorDash Engineering

APRIL 26, 2023

The company is exploring the use of Generative AI, a subset of Artificial Intelligence that generates novel content based on existing data, and how it can be implemented effectively with consideration for the privacy and security of personal information. This reduces manual effort and improves the accuracy and speed of data processing.

Food 98

Food Unstructured Data Deep Learning SQL 98

Databand.ai

AUGUST 30, 2023

Data Testing Tools: Key Capabilities and 6 Tools You Should Know Helen Soloveichik August 30, 2023 What Are Data Testing Tools? Data testing tools are software applications designed to assist data engineers and other professionals in validating, analyzing, and maintaining data quality.

Data Cleanse 52

Data Cleanse Data Validation Data Pipeline Datasets 52

Knowledge Hut

JULY 21, 2023

Data visualization charts are an invaluable tool in simplifying data analysis and presenting insights in visually appealing ways. What is Data Visualization? Data visualization charts are graphical representations of data that help organizations understand and interpret complex information.

Raw Data 52

Raw Data BI Data Transparency Data 52

Data Engineering Weekly

MARCH 30, 2023

In this episode of Data Engineering Weekly Radio, we delve into modern data stacks under pressure and the potential consolidation of the data industry. We discussed that the siloed nature of many data products has led to industry consolidation, ultimately benefiting customers.

Machine Learning 52

Machine Learning Government AWS Data Engineering 52

Cloudera

MARCH 4, 2024

In this article, we will walk you through the process of implementing fine grained access control for the data governance framework within the Cloudera platform. In a good data governance strategy, it is important to define roles that allow the business to limit the level of access that users can have to their strategic data assets.

Metadata 52

Metadata Data Governance Government Management 52

Data Engineering Weekly

NOVEMBER 27, 2022

Data Engineering Weekly Is Brought to You by RudderStack RudderStack provides data pipelines that make it easy to collect data from every application, website, and SaaS platform, then activate it in your warehouse and business tools. I have a long list of thoughts on this conversation, which might need a blog post on its own.

Data Engineering 52

Data Engineering Data Engineer Engineering SQL 52

Confluent

OCTOBER 20, 2023

We’re recognizing excellence across the Data Streaming ecosystem. In this blog post, we announce the global and regional categories that Confluent will recognize across its 2023 Partner of the Year award winners.

IT 62

IT Data 62

Cloudera

AUGUST 20, 2020

Businesses had to literally switch operations, and enable better collaboration and access to data in an instant — while streamlining processes to accommodate a whole new way of doing things. This will inspire trust in the organisation, that all their data and information continues to be secure.

Banking 121

Banking Deep Learning Data Science Machine Learning 121

Cloudera

MAY 22, 2023

Data asset standardization is the purposeful and carefully planned consolidation of redundant, contradictory reports, processes, and databases into enterprise standards. Ideally you are realizing these benefits using Cloudera Data Catalog. Ideally you are realizing these benefits using Cloudera Data Catalog.

Professional Services 73

Professional Services Database Data Governance Data 73

Cloudera

APRIL 24, 2023

To this six-part series, where we’ll look at how to get control of the health of your Cloudera Data platform (CDP) environment. With each blog we’ll outline the symptoms and root causes of common environmental health challenges and prescribe solutions. That’s observability. We’ve done it too. We confess.

Professional Services 81

Professional Services Datasets Data Pipeline Data Architecture 81

Cloudera

JULY 21, 2021

In the build-up to this year’s Data Impact Awards, we’re looking back at last year’s winners. Last year’s awards saw OVO crowned as Data Champions. OVO – 2020’s Data Champion award winner . To do this, OVO built UnCover, a contextual offer engine capable of processing the large-scale volumes of data generated by its users.

Insurance 93

Insurance Hospitality Banking Data 93

Cloudera

JANUARY 24, 2022

Many organizations initiate data projects because they want to increase revenue, but a select few tackle projects that truly transform society. . The bank’s data project aimed to give Filipinos better access to credit and financial opportunities. …and congratulations to the winner: Union Bank of the Philippines. .

Hospitality 64

Hospitality Database-centric Banking Medical 64

Cloudera

DECEMBER 14, 2021

But with growing demands, there’s a more nuanced need for enterprise-scale machine learning solutions and better data management systems. The 2021 Data Impact Awards aim to honor organizations who have shown exemplary work in this area. . In 2021, the finalists under this category include the following organizations.

Transportation 109

Transportation Telecommunication Banking Data Lake 109

Cloudera

JUNE 7, 2023

The revised ZTMM is organized by five categories or pillars: identity, devices, networks, applications and workloads, and data, and four levels of maturity: traditional, initial, advanced, and optimal. The data is protected but it is also accessible by the people who need it. How does Cloudera support the evolution to optimal?

Government 81

Government Metadata Data Lake Data 81

Netflix Tech

NOVEMBER 14, 2023

By Abhinaya Shetty , Bharath Mummadisetty At Netflix, our Membership and Finance Data Engineering team harnesses diverse data related to plans, pricing, membership life cycle, and revenue to fuel analytics, power various dashboards, and make data-informed decisions. We expect complete and accurate data at the end of each run.

Data Engineering 105

Data Engineering Data Engineer Engineering Metadata 105

Monte Carlo

DECEMBER 27, 2022

It definitely holds true for our Data Downtime blog with these five articles driving a majority of our traffic in 2022. Table of Contents What is a Data Mesh — and How Not to Mesh it Up What is Data Observability? The 5 Pillars You Need To Know Is the Modern Data Warehouse Broken? What is Data Observability?

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

Cloudera

MAY 12, 2021

2020 saw us hosting our first ever fully digital Data Impact Awards ceremony, and it certainly was one of the highlights of our year. We saw a record number of entries and incredible examples of how customers were using Cloudera’s platform and services to unlock the power of data. DATA FOR ENTERPRISE AI.

Food 68

Food Healthcare Government Medical 68

Cloudera

NOVEMBER 26, 2020

This year’s Data Impact Awards were like none other that we’ve ever hosted. While all our winners are doing phenomenal work, one of the most exciting awards of the night was The Data for Enterprise AI category. In fact, Experian admits to believing that data has the power to change lives.

Machine Learning 104

Machine Learning Big Data Data Database 104