Snowflake

APRIL 8, 2024

The list of Top 10 semi-finalists is a perfect example: we have use cases for cybersecurity, gen AI, food safety, restaurant chain pricing, quantitative trading analytics, geospatial data, sales pipeline measurement, marketing tech and healthcare. Our sincere thanks go out to everyone who participated in this year’s competition.

Pipeline-centric 127

Pipeline-centric Food Healthcare Unstructured Data 127

Ascend.io

MAY 7, 2024

to bring its cutting-edge automation platform that revolutionizes modern data engineering. This platform, combined with Wizeline’s deep AI expertise, provides a powerful foundation and offering for smarter, faster, and more secure data operations. “Our collaboration with Ascend.io ” The partnership enables Ascend.io

Pipeline-centric 52

Pipeline-centric Data Cleanse Data Security Data Pipeline 52

Data Engineering Podcast

MAY 7, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. What are some of the categories of attributes that need to be managed in a prototypical customer profile?

Pipeline-centric 130

Pipeline-centric Data Lake Machine Learning Data Warehouse 130

Knowledge Hut

JUNE 20, 2023

A novice data scientist prepared to start a rewarding journey may need clarification on the differences between a data scientist and a machine learning engineer. Many people are learning data science for the first time and need help comprehending the two job positions.

Machine Learning 52

Machine Learning Data Engineering Data Engineer Engineering 52

Knowledge Hut

NOVEMBER 2, 2023

For those aspiring to build a career within the Azure ecosystem, navigating the choices between Azure Data Engineers and Azure DevOps Engineers can be quite challenging. Azure Data Engineers and Azure DevOps Engineers are two critical components of the Azure ecosystem for different but interconnected reasons.

Data Engineering 52

Data Engineering Data Engineer Engineering Pipeline-centric 52

Data Engineering Weekly

APRIL 16, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack provides data pipelines that make collecting data from every application, website, and SaaS platform easy, then activating it in your warehouse and business tools. 🍒 We'll send exclusive The Data Stack Show swag just for participating!

Data Engineering 52

Data Engineering Data Engineer Engineering Pipeline-centric 52

Knowledge Hut

NOVEMBER 17, 2023

The demand for data-related professions, including data engineering, has indeed been on the rise due to the increasing importance of data-driven decision-making in various industries. Becoming an Azure Data Engineer in this data-centric landscape is a promising career choice.

Data Engineering 52

Data Engineering Data Engineer Engineering Scala 52

Data Engineering Podcast

AUGUST 13, 2022

Summary Data engineers have typically left the process of data labeling to data scientists or other roles because of its nature as a manual and process heavy undertaking, focusing instead on building automation and repeatable systems. Data stacks are becoming more and more complex.

Machine Learning 100

Machine Learning Pipeline-centric Database-centric MongoDB 100

Knowledge Hut

FEBRUARY 27, 2023

A data engineer is a key member of an enterprise data analytics team and is responsible for handling, leading, optimizing, evaluating, and monitoring the acquisition, storage, and distribution of data across the enterprise. Data Engineers indulge in the whole data process, from data management to analysis.

Data Engineering 52

Data Engineering Data Engineer Database-centric Pipeline-centric 52

Cloudera

DECEMBER 13, 2023

Data was hidden in silos and line-of-business teams were using multiple data management and analytics tools, many of which were not used to their full capability. To realize this cohesive data vision, LGIM adopted Cloudera Data Platform (CDP) Public Cloud.

Pipeline-centric 89

Pipeline-centric Professional Services Datasets BI 89

Ascend.io

OCTOBER 20, 2023

The Nuances of Snowflake Costing Snowflake’s pricing strategy is an exemplification of its user-centric approach: pay for what you use. The Predictability of Pipelines In stark contrast to ad-hoc queries, pipelines are where cost optimization efforts can yield significant dividends.

Pipeline-centric 52

Pipeline-centric IT Data Pipeline Bytes 52

Towards Data Science

MAY 19, 2023

Business users are unable to find and access data assets critical to their workflows. Data engineers spend countless hours troubleshooting broken pipelines. The data team is constantly burning out and has a high employee turnover. Stakeholders fail to see the ROI behind expensive data initiatives.

Pipeline-centric 105

Pipeline-centric Data Software Engineer Software Engineering 105

Databand.ai

JULY 19, 2023

This feature is critical in today’s data-driven business environment, where data may originate from a variety of sources and undergo numerous transformations before reaching its final destination. Metadata Management Metadata, or ‘data about data’, is a crucial component of data management.

Pipeline-centric 52

Pipeline-centric Data Governance Metadata Government 52

ProjectPro

DECEMBER 16, 2021

Are you a newbie in the data science domain ready to embark on a rewarding journey but are confused between the roles of a Machine Learning Engineer vs Data Scientist? Read this article to understand the significant differences and similarities between a machine learning engineer and a data scientist.

Machine Learning 52

Machine Learning Engineering Pipeline-centric Database-centric 52

The Modern Data Company

DECEMBER 29, 2022

This means moving beyond product-centric thinking to a data-driven customer experience model that’s consistent across all channels. Next, the wealth management industry is also shifting away from a product focus to a client-centric model. DataOS is the world’s first operating system.

Banking 52

Banking Pipeline-centric Building Finance 52

Ascend.io

JUNE 22, 2023

The reason is simple yet profound: the very essence of a data mesh is its alignment with business outcomes, and this alignment fundamentally influences the organizational structure of a company. The transition to a decentralized data ownership model presents a unique set of challenges.

Technology 52

Technology Pipeline-centric Consulting Data 52

Monte Carlo

NOVEMBER 14, 2023

This is the world that data orchestration tools aim to create. Data orchestration tools minimize manual intervention by automating the movement of data within data pipelines. According to one Redditor on r/dataengineering, “Seems like 99/100 data engineering jobs mention Airflow.”

Pipeline-centric 40

Pipeline-centric Google Cloud Data Workflow Python 40

Knowledge Hut

SEPTEMBER 26, 2023

The KnowledgeHut Microsoft Data Architect Certification is a valuable credential for people looking to thrive in the field of data design and management, ideally fitting with the industry's changing requirements. This cloud-centric approach ensures scalability, flexibility, and cost-efficiency for your data workloads.

Data Lake 52

Data Lake Database-centric Pipeline-centric Machine Learning 52

Ascend.io

NOVEMBER 14, 2023

Read More: Data Pipeline Automation: The What, How, and Why The Responsibilities of the CAIO Given the complex and multifaceted nature of AI, it is imperative for organizations to delineate clear responsibilities for this crucial role. The CAIO is the vanguard of data privacy and security for the new AI-based capabilities.

Pipeline-centric 52

Pipeline-centric Government Data Pipeline Recruitment 52

phData: Data Engineering

JULY 18, 2023

To truly understand its potential, we need to explore the benefits it brings, particularly when transitioning from traditional data management structures. Why Migrate to a Modern Data Stack? This typically results in long-running ETL pipelines that cause decisions to be made on stale or old data.

Data Warehouse 52

Data Warehouse Pipeline-centric Government Data 52

Knowledge Hut

JANUARY 30, 2024

Whether it's aggregating customer interactions, analyzing historical sales trends, or processing real-time sensor data, data extraction initiates the process. Use Case Essential for data preprocessing and creating usable datasets. Used for knowledge discovery, prediction, and decision support.

ETL Tools 52

ETL Tools Database-centric Data Mining Data Cleanse 52

AltexSoft

JULY 18, 2023

These seemingly unrelated terms unite within the sphere of big data, representing a processing engine that is both enduring and powerfully effective — Apache Spark. Maintained by the Apache Software Foundation, Apache Spark is an open-source, unified engine designed for large-scale data analytics.

Big Data 52

Big Data Data Process Process Hadoop 52

RandomTrees

NOVEMBER 29, 2023

Here if there arises a need to modify the data pipeline , nothing but the data flow from the source to the stage, there is the capability of monitoring the flow processes and other data hold through the governance systems. post which is the ML model trainings.

Data Warehouse 52

Data Warehouse Data Lake Database-centric Data Cleanse 52