Rockset

DECEMBER 9, 2021

Not Just Modern, But Real Time The modern data stack emerged a decade ago, a direct response to the shortcomings of big data. Companies that undertook big data projects ran head-long into the high cost, rigidity and complexity of managing complex on-premises data stacks. The inevitable next stage?

Transportation 52

Transportation BI SQL Data Warehouse 52

Rockset

NOVEMBER 23, 2021

We put our users first and obsess about helping our users achieve speed, scale and simplicity in their modern real-time data stack (some of which I discuss in depth below). Because they help us communicate that performance is one of the core product values at Rockset. Change data capture streams. The problem?

MongoDB 59

MongoDB Data Ingestion PostgreSQL SQL 59

Monte Carlo

AUGUST 15, 2023

Well, that’s because you’re using modern tooling, but with legacy thinking and processes. And while this analogy isn’t a perfect encapsulation of how some data teams operate after moving from on-premises to a modern data stack, it’s close. IMAGE COURTESY OF SHANE MURRAY. Let’s dive in.

Data Storage 88

Data Storage Cloud Metadata Media 88

Towards Data Science

AUGUST 15, 2023

The modern data stacks allow you to do things differently, not just at a larger scale. Well, that’s because you’re using modern tooling, but with legacy thinking and processes. And while this analogy isn’t a perfect encapsulation of how some data teams operate after moving from on-premises to a modern data stack, it’s close.

Cloud 85

Cloud Metadata Media Machine Learning 85

Data Engineering Weekly

APRIL 13, 2023

DBT Reimagined by Pedram Navid Pedram's Data Based dbt Reimagined It was late one night, and I couldn’t sleep, so I started falling into a fantasy where constraints weren’t real and anything was possible. It would make the tool more powerful and easier to use. It would make the tool more powerful and easier to use.

Programming Language 52

Programming Language BI SQL Banking 52

Monte Carlo

JANUARY 20, 2024

The future of business intelligence (BI) is inextricably linked to the future of data. As the amount of data companies create and consume grows exponentially, the speed and ease with which you can access and rely upon that data is going to be more important than ever before.

Business Intelligence 52

Business Intelligence BI Raw Data Datasets 52

Data Engineering Weekly

JULY 9, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork, and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. Andrew Jones: Data Contracts - the book. See how it works today.

Data Engineering 52

Data Engineering Data Engineer Engineering Lambda Architecture 52

Data Engineering Weekly

JULY 23, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. Overall an exciting time for a data contract in 2024. See how it works today.

Data Engineering 52

Data Engineering Data Engineer Engineering Deep Learning 52

Monte Carlo

SEPTEMBER 12, 2023

Now, imagine all the data powering that platform. We had to know how their data team made it happen, especially in such a short span of time, so we recently sat down with Engineering Director Emmanuel Fuentes to learn more. They were among Whatnot’s earliest employees and together built the data foundation for the company.

Machine Learning 52

Machine Learning Data Entertainment SQL 52

Monte Carlo

AUGUST 15, 2023

Well, that’s because you’re using modern tooling, but with legacy thinking and processes. And while this analogy isn’t a perfect encapsulation of how some data teams operate after moving from on-premises to a modern data stack, it’s close. Image courtesy of Shane Murray. Let’s dive in.

Cloud 69

Cloud Metadata Media Machine Learning 69

Data Engineering Weekly

AUGUST 27, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork, and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. 🌐 Tickets are selling fast; use the code DATAHERO for a special discount.

Data Engineering 52

Data Engineering Data Engineer Engineering Machine Learning 52

Netflix Tech

SEPTEMBER 10, 2019

After recreating the dataset, you can plot the raw numbers and perform custom analyses to understand the distribution of the data across test cells. This enables us to optimize their experience at speed. However, the default reports only provide a summary view of the data with some powerful but limited filtering options.

Raw Data 86

Raw Data Python Datasets SQL 86

Rockset

SEPTEMBER 6, 2019

I could be describing either the VR space or Data Engineering, but in fact this post is about the intersection of both. Virtual Reality – The Next Frontier in Media I work as a Data Engineer at a leading company in the VR space, with a mission to capture and transmit reality in perfect fidelity.

Metadata 40

Metadata Kafka Data Cleanse SQL 40

Monte Carlo

JUNE 23, 2022

When it comes to big data quality, bigger data isn’t always better data. At some point in the last two decades, the size of our data became inextricably linked to our ego. We watched enviously as FAANG companies talked about optimizing hundreds of petabyes in their data lakes or data warehouses.

Big Data 40

Big Data Management Machine Learning Data Warehouse 40

Netflix Tech

NOVEMBER 3, 2023

Democratizing Stream Processing @ Netflix By Guil Pires , Mark Cho , Mingliang Liu , Sujay Jain Data powers much of what we do at Netflix. On the Data Platform team, we build the infrastructure used across the company to process data at scale.

SQL 106

SQL Pipeline-centric Kafka Data 106

Rock the JVM

FEBRUARY 5, 2023

Giannis is a proud alumnus of Rock the JVM, working as a Solutions Architect with a focus on Event Streaming and Stream Processing Systems. Enter Giannis: Flink SQL is a powerful high level API for running queries on streaming (and batch) datasets. Streaming (and Batch) SQL 1.1 Streaming (and Batch) SQL 1.1

SQL 63

SQL Kafka Metadata Database 63

Cloudera

APRIL 1, 2024

This recognition underscores Cloudera’s commitment to continuous customer innovation and validates our ability to foresee future data and AI trends, and our strategy in shaping the future of data management. Cloudera, a leader in big data analytics, provides a unified Data Platform for data management, AI, and analytics.

Cloud 107

Cloud Unstructured Data Metadata Datasets 107

Cloudera

NOVEMBER 27, 2023

Flink Forward is a great opportunity to learn about the cutting edge of streaming and stream processing technologies. Flink Forward is a great opportunity to learn about the cutting edge of streaming and stream processing technologies. Flink is, relatively speaking, a newer technology. Takeaway No. billion events/s.

Kafka 83

Kafka SQL ETL Tools Data Lake 83

Towards Data Science

OCTOBER 7, 2023

Here are 5 popular applications data teams at OpenAI, Vimeo, and other companies are putting into practice today. The hype around generative AI is real, and data and ML teams are feeling the heat. What generative AI use cases are realistic, achievable, and actually worth the ROI? Getting started with LLMs?

Unstructured Data 77

Unstructured Data Finance SQL Media 77

Monte Carlo

OCTOBER 4, 2023

The hype around generative AI is real, and data and ML teams are feeling the heat. Across industries, executives are pushing their data leaders to build AI-powered products that will save time, drive revenue, or give them a competitive advantage. What generative AI use cases are realistic, achievable, and actually worth the ROI?

Unstructured Data 72

Unstructured Data Finance SQL Media 72

Towards Data Science

NOVEMBER 4, 2023

Platform Specific Tools and Advanced Techniques Photo by Christopher Burns on Unsplash The modern data ecosystem keeps evolving and new data tools emerge now and then. In this article, I want to talk about crucial things that affect data engineers. Are your data pipelines efficient? Data warehouse exmaple.

Data Engineering 87

Data Engineering Data Engineer Engineering BI 87

AltexSoft

MARCH 30, 2023

The answer is simple: They use the same technology to make the most of data. Along with thousands of other data-driven organizations from different industries, the above-mentioned leaders opted for Databrick to guide strategic business decisions. Let’s see what exactly Databricks has to offer.

Scala 64

Scala Data Lake BI Google Cloud 64

Rockset

APRIL 15, 2022

This is the second post in a series by Rockset's CTO Dhruba Borthakur on Designing the Next Generation of Data Systems for Real-Time Analytics. It’s probably because their analytics database lacks the features necessary to deliver data-driven decisions accurately in real time.

Analytics Application 52

Analytics Application Data Warehouse Raw Data Kafka 52

Knowledge Hut

DECEMBER 13, 2023

The big data industry is growing rapidly. Based on the exploding interest in the competitive edge provided by Big Data analytics, the market for big data is expanding dramatically. The data is the property of various organizations, each of which uses it for various objectives. What Do Big Data Companies Do?

Big Data 52

Big Data Consulting Hadoop Amazon Web Services 52

Monte Carlo

DECEMBER 8, 2023

What’s something you never want your data to be? The best starting place to make sure you really know your data is, well, your data’s starting place – wherever that may be. For data teams, it’s essential to know your data from the source throughout its entire lifecycle. So, how can data teams know this information?

Metadata 52

Metadata Data Data Warehouse Government 52

Monte Carlo

DECEMBER 8, 2023

What’s something you never want your data to be? The best starting place to make sure you really know your data is, well, your data’s starting place – wherever that may be. For data teams, it’s essential to know your data from the source throughout its entire lifecycle. So, how can data teams know this information?

Metadata 52

Metadata Data Data Warehouse Government 52

Monte Carlo

AUGUST 19, 2023

A data platform, often referred to as a ‘modern data stack,’ is the central processing hub for an organization’s data ecosystem. A comprehensive data platform solution powers data acquisition, storage, preparation, delivery, governance, and even the robust security needs of users and applications.

Building 52

Building BI Data Lake Data Governance 52

Cloudera

MAY 18, 2021

Second, since IaaS deployments replicated the on-premises HDFS storage model, they resulted in the same data replication overhead in the cloud (typical 3x), something that could have mostly been avoided by leveraging modern object store. The result is to align cost with business value, irrespective of the technical use case deployed onto CDP.

Hadoop 85

Hadoop Cloud AWS Utilities 85

Monte Carlo

JUNE 28, 2023

Like the omnipresent San Francisco drizzle, Data + AI keynote attendees steadily trickled down the escalators and toward the Moscone Center Hall B + C. There was quite a bit of speculation that attendance would suffer at both events of the major data conferences taking place this week, but that is clearly not the case.

Data Warehouse 52

Data Warehouse Scala Unstructured Data Government 52

Cloudera

AUGUST 17, 2022

What if you could access all your data and execute all your analytics in one workflow, quickly with only a small IT team? CDP One is a new service from Cloudera that is the first data lakehouse SaaS offering with cloud compute, cloud storage, machine learning (ML), streaming analytics, and enterprise grade security built-in.

Cloud Computing 118

Cloud Computing Cloud Storage Data Science Government 118

AltexSoft

MARCH 14, 2023

As the volume and complexity of data continue to grow, organizations seek faster, more efficient, and cost-effective ways to manage and analyze data. In recent years, cloud-based data warehouses have revolutionized data processing with their advanced massively parallel processing (MPP) capabilities and SQL support.

IT 59

IT Data Warehouse Data Governance Data Lake 59

Engineering at Meta

MARCH 9, 2023

Meta is introducing Velox, an open source unified execution engine aimed at accelerating data management systems and streamlining their development. Experimental results from our paper published at the International Conference on Very Large Data Bases (VLDB) 2022 show how Velox improves efficiency and consistency in data management systems.

Engineering 74

Engineering Java Bytes Data Ingestion 74

Data Engineering Weekly

JULY 30, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. We handle Petabytes of data!!! All our data is in S3!!! If you’re a.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Monte Carlo

MAY 11, 2023

When it comes to the data engineer life, to be in the code is to exist. There’s no question that managed SaaS tooling has the power to bring fast, repeatable value to data teams. Let’s take a look at some of what our team has done to deliver the industry’s first truly developer-focused data observability solution.

Data Engineering 52

Data Engineering Data Engineer Engineering IT 52

Rockset

OCTOBER 10, 2019

In this blog, we walk through how to build a real-time dashboard for operational monitoring and analytics on streaming event data from Kafka, which often requires complex SQL, including filtering, aggregations, and joins with other data sets.

Kafka 52

Kafka SQL Building Data 52

phData: Data Engineering

JULY 18, 2023

The Ultimate Modern Data Stack Migration Guide phData Marketing July 18, 2023 This guide was co-written by a team of data experts, including Dakota Kelley, Ahmad Aburia, Sam Hall, and Sunny Yan. Imagine a world where all of your data is organized, easily accessible, and routinely leveraged to drive impactful outcomes.

Data Warehouse 52

Data Warehouse Pipeline-centric Government Data 52

Monte Carlo

MAY 17, 2023

Data observability, an organization’s ability to fully understand the health and quality of the data in their systems, has become one of the hottest technologies in modern data engineering. Because the technology is so extensible, there have been a wide array of suggestions–some more grounded than others–for how it can be used.

Data 52

Data Data Pipeline Data Engineering Data Engineer 52