DataKitchen

MAY 10, 2024

The Five Use Cases in Data Observability: Effective Data Anomaly Monitoring (#2) Introduction Ensuring the accuracy and timeliness of data ingestion is a cornerstone for maintaining the integrity of data systems. This process is critical as it ensures data quality from the onset.

Data Ingestion 40

Data Ingestion Transportation High Quality Data Data Schemas 40

Knowledge Hut

FEBRUARY 29, 2024

Being a data scientist means constantly growing, enabling businesses to become more data-propelled, and learning newer trends and tools. There are various excellent resources in data science that can help you to develop your skillset. So, having the right knowledge of tools and technology is important for handling such data.

Data Science 98

Data Science Datasets Database Design Machine Learning 98

Snowflake

APRIL 20, 2023

Generative AI and large language models (LLMs) are revolutionizing many aspects of both developer and non-coder productivity with automation of repetitive tasks and fast generation of insights from large amounts of data. LLMs have the potential to help both developers and less-technically inclined users make sense of the world’s data.

Building 115

Building Unstructured Data Government Coding 115

Cloudera

MARCH 26, 2024

These questions are legitimate, and this is precisely where Cloudera plays a role. Cloudera: Your Trusted Partner in AI With over 25 Exabytes of Data Under Management and hundreds of customers leveraging our platform for Machine Learning, Cloudera has a long and successful history as an industry leader.

Machine Learning 84

Machine Learning Banking Professional Services Architecture 84

Data Engineering Weekly

APRIL 14, 2024

Meta: OpenEQA - From word models to world models Will AI agents soon become a common fixture in our homes and an integral part of our daily lives? Discover how a universal semantic layer is transforming modern business intelligence, making data more accessible and reliable for organizations striving for informed business decisions.

Data Engineering 62

Data Engineering Data Engineer Engineering Business Intelligence 62

Workfall

SEPTEMBER 18, 2023

Reading Time: 9 minutes Imagine your data as pieces of a complex puzzle scattered across different platforms and formats. This is where the power of data integration comes into play. Meet Airbyte, the data magician that turns integration complexities into child’s play.

Data Pipeline 93

Data Pipeline Raw Data Data Schemas Healthcare 93

Pinterest Engineering

NOVEMBER 7, 2023

Jeff Xiang | Software Engineer, Logging Platform Vahid Hashemian | Software Engineer, Logging Platform Jesus Zuniga | Software Engineer, Logging Platform At Pinterest, data is ingested and transported at petabyte scale every day, bringing inspiration for our users to create a life they love.

Kafka 99

Kafka Java Software Engineer Software Engineering 99

DataKitchen

MAY 10, 2024

The Five Use Cases in Data Observability: Mastering Data Production (#3) Introduction Managing the production phase of data analytics is a daunting challenge. Overseeing multi-tool, multi-dataset, and multi-hop data processes ensures high-quality outputs.

Raw Data 40

Raw Data Data Ingestion Datasets Data 40

Data Engineering Weekly

APRIL 21, 2024

Meta: Introducing Meta Llama 3 - The most capable openly available LLM to date Meta is taking an interesting approach in the growing LLM market with the open source approach and distribution across all the leading cloud providers and data platforms. The blog narrates how Chronon fits into Stripe’s online and offline requirements.

Data Engineering 52

Data Engineering Data Engineer Engineering Medical 52

Knowledge Hut

APRIL 12, 2024

In this blog, we will look at three different project organizational structures: functional, matrix, and process. In this blog, we will look at three different project organizational structures: functional, matrix, and process. Project management plays a significant role in the success of every organization.

Project 98

Project Management Certification Designing 98

Data Engineering Weekly

MAY 16, 2023

In the first part of this series, we talked about design patterns for data creation and the pros & cons of each system from the data contract perspective. In the second part, we will focus on architectural patterns to implement data quality from a data contract perspective. Why is Data Quality Expensive?

Engineering 110

Engineering Kafka Data Pipeline Data Warehouse 110

Cloudera

OCTOBER 24, 2023

But to truly drive transformation telcos must ensure AI models are driven by accurate, high-quality, trusted data, and determine how to manage and govern massive volume at scale. AI-powered automation is already driving significant margin growth by reducing costs. This is the essence of “Trusted AI Everywhere.”

Government 81

Government Finance Deep Learning Data Governance 81

Data Engineering Weekly

FEBRUARY 18, 2024

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Our hope is only with the amazing community of data practitioners who constantly support us. Visit rudderstack.com to learn more. We have no sponsors.

Data Engineering 62

Data Engineering Data Engineer Engineering Data 62

Data Engineering Weekly

SEPTEMBER 24, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. Airflow isn’t meant to process the data. See how it works today.

Data Engineering 67

Data Engineering Data Engineer Engineering Kafka 67

Data Engineering Weekly

MARCH 3, 2024

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Editor’s Note: Chennai, India Meetup - March-08 Update We are thankful to Ideas2IT to host our first Data Hero’s meetup.

Data Engineering 67

Data Engineering Data Engineer Pipeline-centric Engineering 67

Data Engineering Weekly

NOVEMBER 26, 2023

It’s the end of the year, and there will be a lot of buzz about what the next five years in data engineering might bring. It’s the end of the year, and there will be a lot of buzz about what the next five years in data engineering might bring. Wow, can you believe it's almost time to wave goodbye 👋 to 2023?

Data Engineering 70

Data Engineering Data Engineer Engineering Kafka 70

Engineering at Meta

OCTOBER 24, 2023

In our last blog post on automatic product deprecation , we talked about the complexities of product deprecations, and a solution Meta has built called the Systematic Code and Asset Removal Framework (SCARF). This naturally leads to the question: How do we automatically unblock SCARF when there is code that references an asset?

Coding 129

Coding Programming Language Python MySQL 129

Workfall

JUNE 12, 2023

In this dynamic realm of data engineering, a monumental challenge takes centre stage: efficiently managing the ever-changing tides of real-time data. Data, the lifeblood of organisations, holds the key to unlocking untapped potential and propelling businesses forward. In this blog, we will cover: What Is CDC and Its Benefits?

Telecommunication 80

Telecommunication Metadata Healthcare Finance 80

Snowflake

NOVEMBER 30, 2023

At TCS , we help companies shift their enterprise data warehouse (EDW) platforms to the cloud as well as offering IT services. We’re extremely familiar with just how tricky a cloud migration can be, especially when it involves moving historical business data.

Data Warehouse 99

Data Warehouse Banking Data Cloud 99

Knowledge Hut

APRIL 25, 2024

Splunk is a powerful data analysis tool that can be used to monitor and troubleshoot a variety of systems. Splunk can also be used to generate reports and dashboards to help visualize data. Splunk is a program that primarily functions as a web-style interface for searching, monitoring, and analyzing machine-generated Big Data.

Architecture 52

Architecture Data Analysis Tools Big Data Certification 52

Cloudera

JUNE 2, 2022

Since 2015, the Cloudera DataFlow team has been helping the largest enterprise organizations in the world adopt Apache NiFi as their enterprise standard data movement tool. This need has generated a market opportunity for a universal data distribution service. Why does every organization need it when using a modern data stack?

Systems 105

Systems Data Lake Google Cloud Data Collection 105

Knowledge Hut

MARCH 7, 2024

BAs are thus expected to have knowledge about key concepts of business analysis and be skilled in using different tools and techniques for eliciting, analyzing, and managing requirements. BABOK ® defines 6 key competencies that a BA is expected to possess. Each competency group constitutes multiple sub-competencies which are as discussed below.

Business Analyst 98

Business Analyst Consulting Data Analysis Finance 98

Knowledge Hut

MARCH 22, 2024

In this blog post, I'll dive into the core essence of what ChatGPT is, discussing its advantages and limitations, exploring a variety of compelling use cases and understanding how to use ChatGPT. Amidst constantly advancing technologies, ChatGPT distinguishes itself with its rapid capability to generate rich and meaningful dialogues.

Education 52

Education Data Analysis Coding Datasets 52

DareData

MARCH 5, 2024

In this blog post, we'll delve into some of our project portfolio in the Generative AI space and understand how we are deploying GenAI at our customers. At DareData Engineering, we believe in a human-centric approach, where AI agents work together with humans to achieve faster and more efficient results. Powered by GPT 3.5

Database-centric 52

Database-centric Telecommunication SQL Unstructured Data 52

Knowledge Hut

MARCH 27, 2024

In this blog, we will learn that UI copywriting is not simply communication with users; rather, UI copywriters research user requirements, predict their questions and even create answers for them. In this age of digital interface, UI copywriters play the primary role of shaping the user experience. What is UI Copywriting?

Designing 52

Designing Technology Government Certification 52

Knowledge Hut

NOVEMBER 16, 2023

One of the biggest challenges faced by corporations today when it comes to cloud adoption is the lack of cloud expertise. There is a clear shortage of professionals certified with Amazon Web Services (AWS). As far as AWS certifications are concerned, there is always a certain debate surrounding them. There is a definite reason for that. What is AWS?

AWS 98

AWS Certification IT Amazon Web Services 98

Knowledge Hut

NOVEMBER 27, 2023

The Data Science learning path is a collective set of curated courses that comprise a learning plan for achieving the required skills for the data scientist role. While the time limit to complete the learning path to become a data scientist can expect 8-9 months to get through all Data Science courses.

Data Science 97

Data Science Healthcare Machine Learning Telecommunication 97

Edureka

DECEMBER 12, 2023

Artificial Intelligence in education is being transformed by automating repetitive chores, personalizing learning experiences, and delivering insightful data on student performance. Welcome to our blog about artificial intelligence (AI) in education and how it is changing the game!

Education 52

Education Algorithm Accessible Accessibility 52

Knowledge Hut

DECEMBER 22, 2023

Every business now incorporates data science into their operations, especially those that recognize the value of data and the potential applications of that knowledge. A data scientist's main responsibility is to draw practical conclusions from complicated data so that you may make informed business decisions.

Portfolio 98

Portfolio Data Science Programming Language Scala 98

Knowledge Hut

MARCH 13, 2024

Google is one of the most attractive employers globally and is well-known to be one of the highest-paying companies. The search engineer giant provides a platform for many exceptional software engineers beyond problem-solving. Reputed companies often seek to hire top-notch software engineers with extensive knowledge across several industries.

Software Engineer 52

Software Engineer Software Engineering Process Engineering 52

Data Engineering Weekly

AUGUST 6, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. The first one that caught my eye is Astronomer published a blog post Introducing Cosmos 1.0:

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Cloudera

JULY 24, 2023

In their effort to reduce their technology spend, some organizations that leverage open source projects for advanced analytics often consider either building and maintaining their own runtime with the required data processing engines or retaining older, now obsolete, versions of legacy Cloudera runtimes (CDH or HDP).

Software Engineer 68

Software Engineer Software Engineering Project Coding 68

Picnic Engineering

FEBRUARY 8, 2023

In our previous blog, Dima Kalashnikov explained how we configure our Internal services pipeline in the Analytics Platform. In this post, we will explain how our team automates the creation of new data pipeline deployments. Question is: how do we ensure the atomicity of the pipeline deployment? What is the Saga pattern?

Data Pipeline 105

Data Pipeline Kafka Data Architecture 105

Cloudera

MARCH 17, 2023

As the use of ChatGPT becomes more prevalent, I frequently encounter customers and data users citing ChatGPT’s responses in their discussions. I love the enthusiasm surrounding ChatGPT and the eagerness to learn about modern data architectures such as data lakehouses, data meshes, and data fabrics.

Education 62

Education Unstructured Data Data Lake Data Warehouse 62

Cloudera

JANUARY 28, 2022

As I meet with our customers, there are always a range of discussions regarding the use of the cloud for financial services data and analytics. Customers vary widely on the topic of public cloud – what data sources, what use cases are right for public cloud deployments – beyond sandbox, experimentation efforts.

Cloud 116

Cloud Banking Data Governance Government 116

Cloudera

FEBRUARY 23, 2022

Their processes are ‘data driven’, their networks are trending towards automation, and AI systems are powering customer engagement in store , online and at home. This is all based on an enterprise IT architecture that is integrated, data rich, and agile. Are we looking at a transformed business?

Telecommunication 95

Telecommunication Cloud Media Government 95

Databand.ai

JULY 19, 2023

Data Lineage Tools: Key Capabilities and 5 Notable Solutions Ryan Yackel July 19, 2023 What Are Data Lineage Tools? Data lineage tools provide a visual representation of your data’s journey across multiple systems and transformations. Data lineage tools are not a new concept.

Pipeline-centric 52

Pipeline-centric Data Governance Metadata Government 52