Snowflake

MAY 4, 2023

The market volatility we’ve seen in recent years has only underscored the value of using data to model and analyze market behavior and avoid to manage risk for the Street. In the end, we want all of DTCC’s data securely accessible to our internal and external stakeholders. This presents what I like to call an “efficiency opportunity.”

Portfolio 52

Portfolio Cloud Analytics Application Data Security 52

Rockset

JANUARY 30, 2024

In 2023, Rockset announced a new cloud architecture for search and analytics that separates compute-storage and compute-compute. With this architecture, users can separate ingestion compute from query compute, all while accessing the same real-time data. minutes to batch load the data. minutes Ingestion compute saving (1-(( 16.7

Data Ingestion 64

Data Ingestion Utilities Architecture SQL 64

Data Science Blog: Data Engineering

JULY 4, 2022

Already familiar with the term big data, right? Despite the fact that we would all discuss Big Data, it takes a very long time before you confront it in your career. Apache Spark is a Big Data tool that aims to handle large datasets in a parallel and distributed manner. Begin with a small sample of the data.

Hadoop 52

Hadoop Big Data Datasets Scala 52

ProjectPro

FEBRUARY 11, 2021

Spoiler Alert: Becoming a machine learning engineer can sound like a hard-to-reach goal but let us tell you the truth – it isn’t as hard as it seems. Image Credit: Makeameme.org So you are considering learning machine learning skills , and you’ve heard that becoming a machine learning engineer is the way to go.

Machine Learning 40

Machine Learning Engineering Programming Language Portfolio 40

Rockset

MAY 17, 2022

This is the fourth post in a series by Rockset's CTO Dhruba Borthakur on Designing the Next Generation of Data Systems for Real-Time Analytics. We'll be publishing more posts in the series in the near future, so subscribe to our blog so you don't miss them! As a result, the use cases remained firmly in batch mode.

SQL 52

SQL NoSQL Hadoop MongoDB 52

Cloudera

FEBRUARY 9, 2021

Today’s customers have a growing need for a faster end to end data ingestion to meet the expected speed of insights and overall business demand. This ‘need for speed’ drives a rethink on building a more modern data warehouse solution, one that balances speed with platform cost management, performance, and reliability.

Data Warehouse 70

Data Warehouse Cloud Kafka Cloud Storage 70

Rockset

JULY 6, 2022

This is the fifth post in a series by Rockset's CTO and Co-founder Dhruba Borthakur on Designing the Next Generation of Data Systems for Real-Time Analytics. We'll be publishing more posts in the series in the near future, so subscribe to our blog so you don't miss them! Traditionally, schemas are strictly enforced.

NoSQL 52

NoSQL SQL Systems PostgreSQL 52

Rockset

MARCH 19, 2020

Intro In recent years, Kafka has become synonymous with “streaming,” and with features like Kafka Streams, KSQL, joins, and integrations into sinks like Elasticsearch and Druid, there are more ways than ever to build a real-time analytics application around streaming data in Kafka.

Kafka 40

Kafka Database Process SQL 40

Knowledge Hut

JULY 26, 2023

Database applications have become vital in current business environments because they enable effective data management, integration, privacy, collaboration, analysis, and reporting. Database applications also help in data-driven decision-making by providing data analysis and reporting tools.

Database 52

Database NoSQL Telecommunication MongoDB 52

ProjectPro

MAY 20, 2015

It is difficult to stay up-to-date with the latest developments in IT industry especially in a fast growing area like big data where new big data companies, products and services pop up daily. With the explosion of Big Data, Big data analytics companies are rising above the rest to dominate the market.

Big Data 40

Big Data Hadoop Business Analyst Unstructured Data 40

Data Science Blog: Data Engineering

NOVEMBER 15, 2023

In addition to Business Intelligence (BI), Process Mining is no longer a new phenomenon, but almost all larger companies are conducting this data-driven process analysis in their organization. The Event Log Data Model for Process Mining Process Mining as an analytical system can very well be imagined as an iceberg.

Architecture 52

Architecture Database-centric Process BI 52

Cloudera

FEBRUARY 8, 2024

Overview This blog post describes support for materialized views for the Iceberg table format. Apache Iceberg is a high-performance open table format for petabyte-scale analytic datasets. Apache Iceberg forms the core foundation for Cloudera’s Open Data Lakehouse with the Cloudera Data Platform (CDP).

Metadata 92

Metadata Data Warehouse BI AWS 92

Cloudera

AUGUST 13, 2021

Every enterprise is trying to collect and analyze data to get better insights into their business. Whether it is consuming log files, sensor metrics, and other unstructured data, most enterprises manage and deliver data to the data lake and leverage various applications like ETL tools, search engines, and databases for analysis.

Data Pipeline 109

Data Pipeline Data Lake ETL Tools Unstructured Data 109

Cloudera

NOVEMBER 7, 2023

Are you struggling to manage the ever-increasing volume and variety of data in today’s constantly evolving landscape of modern data architectures? This blog post is intended to provide guidance to Ozone administrators and application developers on the optimal usage of the bucket layouts for different applications.

Systems 105

Systems Hadoop Unstructured Data Media 105

Rockset

DECEMBER 9, 2019

As a data engineer, my time is spent either moving data from one place to another, or preparing it for exposure to either reporting tools or front end users. As data collection and usage have become more sophisticated, the sources of data have become a lot more varied and disparate, volumes have grown and velocity has increased.

Data Engineering 40

Data Engineering Data Engineer Raw Data Engineering 40

Rockset

FEBRUARY 24, 2023

Introduction Let’s get this out of the way at the beginning: understanding effective streaming data architectures is hard, and understanding how to make use of streaming data for analytics is really hard. What Is Streaming Data? We’re going to start with a basic question: what is streaming data? Kafka or Kinesis ?

Kafka 52

Kafka AWS Amazon Web Services Programming Language 52

Rockset

OCTOBER 26, 2023

JetBlue is the data leader in the airline industry using data to offer industry-leading customer experiences and disruptive low fares to popular destinations around the world. The complexity of the data and situation can be hard to quickly comprehend and take action on without the assistance of machine learning.

Machine Learning 64

Machine Learning Data Science Architecture Data Warehouse 64

Cloudera

NOVEMBER 2, 2020

Users today are asking ever more from their data warehouse. As an example of this, in this post we look at Real Time Data Warehousing (RTDW), which is a category of use cases customers are building on Cloudera and which is becoming more and more common amongst our customers. Ingest 100s of TB of network event data per day .

Data Warehouse 96

Data Warehouse Kafka Lambda Architecture Telecommunication 96

Rockset

MAY 3, 2023

Rockset is a database used for real-time search and analytics on streaming data. In scenarios involving analytics on massive data streams, we’re often asked the maximum throughput and lowest data latency Rockset can achieve and how it stacks up to other databases. Why measure streaming data ingestion?

Data Ingestion 59

Data Ingestion Kafka Database Architecture 59

Rockset

MARCH 15, 2021

This includes making the data available for query as soon as it is ingested, creating proper indexes on the data so that the query latency is very low, and much more. Before it can be ingested, there’s usually a data pipeline for transforming incoming data. latency, indexing, etc.), Let’s look at some examples.

MongoDB 40

MongoDB Data Ingestion Analytics Application Kafka 40

Cloudera

SEPTEMBER 15, 2022

Apache Ozone is a distributed, scalable, and high-performance object store , available with Cloudera Data Platform (CDP), that can scale to billions of objects of varying sizes. Structured data (such as name, date, ID, and so on) will be stored in regular SQL databases like Hive or Impala databases. Diversity of workloads.

Systems 89

Systems Hadoop Metadata Telecommunication 89

phData: Data Engineering

MAY 11, 2023

Practices centered on software engineering principles can create a barrier to entry for teams with skilled data wranglers looking to take their infrastructure to the next level with cloud-native tools like Matillion for the Snowflake Data Cloud. The hallmarks of this method are modular design and effective organization skills.

Coding 52

Coding Software Engineer Software Engineering Government 52

Rockset

JANUARY 23, 2024

Elasticsearch is an open-source search and analytics engine based on Apache Lucene. When building applications on change data capture (CDC) data using Elasticsearch, you’ll want to architect the system to handle frequent updates or modifications to the existing documents in an index. million on average.

Metadata 52

Metadata Coding Analytics Application Python 52

Cloudera

DECEMBER 7, 2017

Cloud promises many advantages as an environment for machine learning and analytics. Cloud makes it fast and easy to spin up resources for new applications. Cloud offers elasticity of those resources to efficiently support transient analytics workloads and data pipelines. There is a better way.

Building 40

Building Management Government BI 40

Cloudera

MARCH 6, 2018

The real power in machine learning and analytics is when multiple analytics disciplines are able to work together in concert, sharing data in service of solving more complex and more valuable questions. Instead, they have separate data stores and inconsistent (if any) frameworks for data governance, management, and security.

Cloud 40

Cloud Metadata Big Data AWS 40

Rockset

FEBRUARY 25, 2021

It’s difficult to create data analytics systems that can easily query across your various data sources while maintaining fast performance and real-time capabilities. Two of these real-time analytics solutions are Elasticsearch and Rockset. Instead, this data is often semi-structured in JSON or arrays.

SQL 40

SQL Data Pipeline Kafka Database 40

Cloudera

APRIL 2, 2019

Cloudera and Intel have a long history of innovation, driving big data analytics and machine learning into the enterprise with unparalleled performance and security. When in memory mode, the data is not saved in the event of a power loss. This is achieved through an architecture that fundamentally separates compute from storage.

NoSQL 48

NoSQL Google Cloud Hadoop Machine Learning 48

ProjectPro

JANUARY 31, 2023

If you're looking to break into the exciting field of big data or advance your big data career, being well-prepared for big data interview questions is essential. Get ready to expand your knowledge and take your big data career to the next level! “Data analytics is the future, and the future is NOW!

Big Data 52

Big Data Hadoop AWS Relational Database 52

AltexSoft

OCTOBER 21, 2022

Similar to Google in web browsing and Photoshop in image processing, it became a gold standard in data streaming, preferred by 70 percent of Fortune 500 companies. Apache Kafka is an open-source, distributed streaming platform for messaging, storing, processing, and integrating large data volumes in real time. What is Kafka?

Kafka 93

Kafka Hadoop ETL Tools Big Data 93

Cloudera

NOVEMBER 13, 2017

As the market moves toward cloud-based big data and analytics, three qualities emerge as vital for success. End-user focused tools accelerate daily tasks like job submission, performance tuning, and workload analytics. The net result is much improved productivity for data engineers, data scientists, and analysts.

Cloud 40

Cloud Big Data Metadata Machine Learning 40

ProjectPro

MAY 23, 2015

It takes in approximately $36 million dollars from across 4300 US stores everyday.This article details into Walmart Big Data Analytical culture to understand how big data analytics is leveraged to improve Customer Emotional Intelligence Quotient and Employee Intelligence Quotient. How Walmart is tracking its customers?

Big Data 52

Big Data Data Analysis Hadoop Retail 52

Cloudera

SEPTEMBER 10, 2018

The other day, it struck me how similar this is to how organizations digitally transform and become data-driven. Your raw material is data instead of wood, and you use multidisciplinary analytics rather than woodworking tools like a hammer, saw, and sander.

Data Warehouse 40

Data Warehouse Metadata Retail Cloud 40

Cloudera

MAY 16, 2018

All three cases require a “big picture” approach that incorporates new and alternative data sources and cross-functional collaboration throughout the organization not only to identify illegal activities, rogue traders, or personal misconduct but also to provide evidential material that demonstrates a deep understanding of the intent.

Data Lake 42

Data Lake Electronics Media Unstructured Data 42

ProjectPro

SEPTEMBER 6, 2021

Are you confused about choosing the best cloud platform for your next data engineering project ? AWS vs. GCP blog compares the two major cloud platforms to help you choose the best one. Typical applications and services under the AWS umbrella are cloud migration, content delivery, backup and restore functions, etc.

AWS 52

AWS Amazon Web Services Google Cloud Cloud Storage 52

Cloudera

JUNE 16, 2022

The ability to perform analytics on data as it is created and collected (a.k.a. real-time data streams) and generate immediate insights for faster decision making provides a competitive edge for organizations. . CSP was recently recognized as a leader in the 2022 GigaOm Radar for Streaming Data Platforms report.

Kafka 88

Kafka Manufacturing Data Lake SQL 88

ProjectPro

MAY 31, 2021

Ace your big data interview by adding some unique and exciting Big Data projects to your portfolio. This blog lists over 20 big data projects you can work on to showcase your big data skills and gain hands-on experience in big data tools and technologies. Table of Contents What is a Big Data Project?

Big Data 52

Big Data Coding Project Hadoop 52

Rockset

APRIL 15, 2022

This is the second post in a series by Rockset's CTO Dhruba Borthakur on Designing the Next Generation of Data Systems for Real-Time Analytics. We'll be publishing more posts in the series in the near future, so subscribe to our blog so you don't miss them!

Analytics Application 52

Analytics Application Data Warehouse Raw Data Kafka 52