Monte Carlo

FEBRUARY 20, 2024

At the heart of every data-driven decision is a deceptively simple question: How do you get the right data to the right place at the right time? The growing field of data ingestion tools offers a range of answers, each with implications to ponder. Fivetran Image courtesy of Fivetran.

Data Ingestion 52

Data Ingestion Google Cloud Kafka AWS 52

Snowflake

MARCH 2, 2023

Now we are able to ingest our data in near real time directly from Kafka topics to a Snowflake table, drastically reducing the cost of ingestion and improving our SLA from 15 minutes to within 60 seconds. Streaming data and historical data should not live in silos or cause infrastructure management complexity.

Kafka 125

Kafka Data Ingestion Data Pipeline Cloud Storage 125

Cloudera

OCTOBER 15, 2020

Data Discovery and Exploration (DDE) was recently released in tech preview in Cloudera Data Platform in public cloud. In this blog we will go through the process of indexing data from S3 into Solr in DDE with the help of NiFi in Data Flow. Spark as the ingest pipeline tool for Search (i.e. nifi-solr-demo.

AWS 118

AWS Data Cloud Cloud Storage 118

Monte Carlo

APRIL 24, 2023

With our new partnership and updated integration, Monte Carlo provides full, end-to-end coverage across data lake and lakehouse environments powered by Databricks. But remember that line from the introduction about the blurring line between data warehouses and data lakes? It works in both directions.

Data Lake 52

Data Lake Google Cloud Data Warehouse AWS 52

Rockset

MARCH 1, 2023

Rockset introduces a new architecture that enables separate virtual instances to isolate streaming ingestion from queries and one application from another. Developers can spin up or down virtual instances based on the performance requirements of their streaming ingest or query workloads.

Architecture 59

Architecture AWS SQL Cloud Storage 59

Confluent

FEBRUARY 6, 2019

It allows real-time data ingestion, processing, model deployment and monitoring in a reliable and scalable way. This blog post focuses on how the Kafka ecosystem can help solve the impedance mismatch between data scientists, data engineers and production engineers.

Machine Learning 107

Machine Learning Python Kafka Java 107

AltexSoft

MARCH 30, 2023

Databricks architecture Databricks provides an ecosystem of tools and services covering the entire analytics process — from data ingestion to training and deploying machine learning models. Besides that, it’s fully compatible with various data ingestion and ETL tools. Let’s see what exactly Databricks has to offer.

Scala 64

Scala Data Lake BI Google Cloud 64