LinkedIn Engineering

DECEMBER 19, 2023

Our RU framework ensures that our big data infrastructure, which consists of over 55,000 hosts and 20 clusters holding exabytes of data, is deployed and updated smoothly by minimizing downtime and avoiding performance degradation. The data is accessible through Hive and Trino, allowing queries for different dates and timestamps.

Big Data 92

Big Data Hadoop Metadata Data 92

Rockset

AUGUST 30, 2021

The latest Rockset release, SQL-based rollups, has made real-time analytics on streaming data a lot more affordable and accessible. Anyone who knows SQL, the lingua franca of analytics, can now rollup, transform, enrich and aggregate real-time data at massive scale. You can also optionally use WHERE clauses to filter out data.

SQL 52

SQL Kafka MongoDB MySQL 52

ProjectPro

DECEMBER 7, 2021

Data pipelines are a significant part of the big data domain, and every professional working or willing to work in this field must have extensive knowledge of them. Data pipelines must be scalable due to the volume of big data, which might fluctuate over time.

Data Pipeline 52

Data Pipeline Architecture Kafka AWS 52

ProjectPro

FEBRUARY 8, 2023

Do ETL and data integration activities seem complex to you? Read this blog to understand everything about AWS Glue that makes it one of the most popular data integration solutions in the industry. Did you know the global big data market will likely reach $268.4 Businesses are leveraging big data now more than ever.

AWS 98

AWS Scala Metadata Data Lake 98

AltexSoft

OCTOBER 21, 2022

This scenario involves three main characters — publishers, subscribers, and a message or event broker. A publisher (say, telematics or Internet of Medical Things system) produces data units, also called events or messages , and directs them not to consumers but to a middleware platform — a broker. Kafka cluster and brokers.

Kafka 93

Kafka Hadoop ETL Tools Big Data 93

ProjectPro

AUGUST 24, 2021

Data professionals who work with raw data like data engineers, data analysts, machine learning scientists , and machine learning engineers also play a crucial role in any data science project. And, out of these professions, this blog will discuss the data engineering job role.

Data Engineering 52

Data Engineering Data Engineer Coding Project 52

Rockset

FEBRUARY 25, 2021

Joins are often used in real-time analytics applications to combine streaming data (usually representing events) with static data (like customer information). With Elasticsearch, joins are not a first class citizen and many teams end up denormalizing their data to model relationships.

SQL 40

SQL Data Pipeline Kafka Database 40