Towards Data Science

DECEMBER 1, 2023

Pet Project for Data/Analytics Engineers: Explore Modern Data Stack Tools — dbt Core, Snowflake, Fivetran, GitHub Actions. Photo by Gaining Visuals on Unsplash Here is a simple and fast pet project for Data/Analytics Engineers, who want to kick the tires on Modern Data Stack tools including dbt Core, Snowflake, Fivetran, and GitHub Actions.

Data Engineering 75

Data Engineering Data Engineer Project Engineering 75

Data Engineering Podcast

SEPTEMBER 3, 2023

The obvious comparison is with systems like Singer/Meltano/Airbyte in the open source space, or Fivetran/Matillion/etc. The obvious comparison is with systems like Singer/Meltano/Airbyte in the open source space, or Fivetran/Matillion/etc. What is the problem you want to solve with dlt? Who is the target audience?

Data Integration 162

Data Integration BI SQL Python 162

Christophe Blefari

FEBRUARY 2, 2024

Fivetran uses DuckDB as the tech to do file merge in the data lake offering Datacamp uses DuckDB to be able in notebooks to query dataframes in SQL and consider it for teaching SQL — I might have something in the making about this on my side. Lastly they pre-compute statistics on datasets with DuckDB.

Datasets 130

Datasets Data Lake SQL Algorithm 130

Data Engineering Podcast

JULY 16, 2023

To help other people find the show please leave a review on Apple Podcasts and tell your friends and co-workers Links Datapreneurs Book SQL Server Snowflake Z80 Processor Navigational Database System R Redshift Microsoft Fabric Databricks Looker Fivetran Podcast Episode Databricks Unity Catalog RelationalAI 6th Normal Form Pinecone Vector DB Podcast (..)

SQL 130

SQL Machine Learning Python Data 130

Start Data Engineering

NOVEMBER 8, 2020

This post aims to answer questions like the ones shown below My company does not have the budget to purchase a tool like fivetran, What should I use to pull data from an API ?

AWS 130

AWS Cloud Storage Data Pipeline Data 130

Snowflake

DECEMBER 11, 2023

Ingest data from the 8 ad platforms with Fivetran and normalize the data sets on Snowflake. A lot of the pre-processing done by Fivetran and use of dbt packages in those first two steps accelerates our time to deployment from weeks to one or two business days. Model data transformations with dbt to unify the data.

Building 84

Building Cloud Media Algorithm 84

Christophe Blefari

JULY 15, 2023

They use Fivetran, dbt and Census. Spanish is the second most spoken language by native speakers and the fourth most spoken by all speakers. Fast News ⚡️ How we cut BigQuery costs 80% by hunting down costly queries — Mixpanel team hugely reduced their BigQuery spending.

Datasets 130

Datasets Python Machine Learning Data 130

Christophe Blefari

JUNE 3, 2023

This is a major difference with the competition (Airbyte, Fivetran, Stitch). With a few configuration and a CLI you can write data pipelines using hundreds of connectors (using Singer spec). The pricing is based on the number of runs and not the volume of data.

Data Pipeline 130

Data Pipeline Data SQL Algorithm 130

Data Engineering Podcast

FEBRUARY 13, 2022

Singer, Stitch, Fivetran) and reverse ETL (e.g. Singer, Stitch, Fivetran) and reverse ETL (e.g. What other tools/systems might you replace with Rudderstack? How do you think about the application of Rudderstack compared to tools for data integration (e.g. Grouparoo, Hightouch, Census)? Grouparoo, Hightouch, Census)?

Building 100

Building Hadoop Data Pipeline Metadata 100

databricks

MAY 3, 2023

The Modern Data Stack (MDS) appeared several years ago as cloud-based modern data platforms put analytics - and the tools that power it.

Building 63

Building Cloud Data IT 63

Christophe Blefari

OCTOBER 20, 2023

They are French and they built a great platform on top of Redpanda and Flink claiming to be 4x cheaper than Fivetran to do data replication. Popsink a real-time ingestion and processing platform released their self-service offering this week. As an echo of last bullet point.

Generalist 130

Generalist Entertainment NoSQL Datasets 130

Data Engineering Podcast

MARCH 22, 2021

Fivetran/Airbyte/Meltano/custom scripts) What are the moving pieces in a CDC workflow that need to be considered as you are designing the system? Fivetran/Airbyte/Meltano/custom scripts) What are the moving pieces in a CDC workflow that need to be considered as you are designing the system?

Data Warehouse 100

Data Warehouse Metadata Data Lake Hadoop 100

Snowflake

MARCH 14, 2024

They applied solutions like SAP BusinessObjects Data Services, Fivetran and Qlik, or used extractors to get SAP data into SAP BW and then attached more tools to get the data from SAP BW into other systems. To extract this data from SAP, customers went to their toolbox.

IT 82

IT Data Ingestion Data AWS 82

Christophe Blefari

JANUARY 20, 2024

In this category I recommend also to have a look at data ingestion (Airbyte, Fivetran, etc.), The main difference between both is the fact that your computation resides in your warehouse with SQL rather than outside with a programming language loading data in memory. workflows (Airflow, Prefect, Dagster, etc.)

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

ThoughtSpot

SEPTEMBER 6, 2023

That’s why ThoughtSpot and Fivetran are joining forces to decrease the amount of time, steps, and effort required to go from raw data to AI-powered insights. Our new SpotApp integration allows users to launch a Fivetran connector and a corresponding ThoughtSpot analytics template in a single workflow.

Data Workflow 91

Data Workflow Raw Data Data Lake Business Intelligence 91

phData: Data Engineering

MARCH 8, 2023

Fivetran, a cloud-based automated data integration platform, has emerged as a leading choice among businesses looking for an easy and cost-effective way to unify their data from various sources. What is Fivetran? What is Fivetran Used For? Why Use Fivetran?

IT 96

IT Data Warehouse Data Ingestion Data Integration 96

Monte Carlo

SEPTEMBER 22, 2021

Today, we’re thrilled to announce that Bob Muglia , entrepreneur, Fivetran board member, and former CEO of Snowflake, and DJ Patil, the first U.S. As part of this announcement, we’re thrilled to share that Fivetran , the industry leader in data integration, will be joining IMPACT as a sponsor!

Data Lake 52

Data Lake Kafka Data Science Cloud 52

Christophe Blefari

JANUARY 27, 2023

Earlier in January Fivetran also announced a free plan. Starting today the connectors that are in alpha and beta will be free to use in Airbyte Cloud. It needs only one side of the sync to be in alpha/beta to have it for free. Once GA you'll have 2 weeks before being charged. SQLAlchemy 2.0

Data 130

Data Algorithm Data Pipeline SQL 130

Monte Carlo

JUNE 19, 2023

Through partnerships with industry leaders like Snowflake, Databricks, FiveTran, and dbt Labs, we’re staying at the forefront of ensuring data reliability across our customers’ entire stack.

MySQL 52

MySQL Database SQL Data 52

Monte Carlo

APRIL 5, 2023

An Airflow job got grounded, a Fivetran sync sunk, a JOIN further upstream went RIGHT instead of LEFT. When there is a data freshness issue, you check Airflow for failed jobs before tabbing open and checking Fivetran and dbt. The query went to execute on data that never arrived. You may even have proactive alerts set up.

Data Warehouse 52

Data Warehouse Data Engineering Data Engineer Coding 52

Christophe Blefari

MARCH 24, 2023

— Georges, Fivetran CEO, ran a performance test to have metrics on DuckDB performance. How LinkedIn reduced processing time with Apache Beam — Beam is a distributed processing framework that proposes a unified execution engine for batch and real-time. How fast is DuckDB really?

Lambda Architecture 130

Lambda Architecture Data Pipeline Data SQL 130

Knowledge Hut

APRIL 25, 2023

Data integration: Data engineers should be able to integrate data from various sources like databases, APIs, or file systems, using tools like Apache NiFi, Fivetran, or Talend. With FiveTran, data engineers can effortlessly extract data from multiple sources and load it into their preferred data warehouse or data lake.

Data Engineering 52

Data Engineering Data Engineer Engineering Google Cloud 52

Christophe Blefari

SEPTEMBER 28, 2023

With the cloud, we got the—false—impression that resources were infinite and cheap, so we switched to ETL by pushing everything into a central data storage.

Cloud Storage 147

Cloud Storage Big Data Hadoop SQL 147

Data Engineering Podcast

APRIL 3, 2022

To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Red Ventures Monte Carlo Opportunity Cost dbt Podcast Episode Apache Ranger Privacera Podcast Episode Segment Fivetran Podcast Episode Databricks Bigquery Redshift Hightouch Podcast Episode Airflow Astronomer Podcast Episode Airbyte Podcast (..)

Designing 100

Designing Data Warehouse Data Engineering Data Engineer 100

Data Engineering Podcast

MAY 15, 2022

To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Robinhood Yelp Kafka Spark Flink Podcast Episode Pulsar Podcast Episode Parquet Change Data Capture Delta Lake Podcast Episode Hudi Podcast Episode Redshift BigQuery Informatica Data Mesh Podcast Episode PrestoDB Trino Airbyte Podcast Episode (..)

Data Lake 100

Data Lake Building BI Architecture 100

Data Engineering Podcast

JULY 3, 2022

__init__ Episode Erez Shinan Fivetran Podcast Episode md5 CRC32 Merkle Tree Locally Optimistic Presto The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA Support Data Engineering Podcast Summary The perennial challenge of data engineers is ensuring that information is integrated reliably.

Data Integration 100

Data Integration MongoDB Scala MySQL 100

Monte Carlo

MAY 2, 2023

For example, we recently announced an integration with Fivetran, which gives customers the ability to accelerate data incident detection and resolution by adding monitoring to data pipelines at the point of creation.

Data Lake 52

Data Lake Data Warehouse Database Data Pipeline 52

Data Engineering Podcast

NOVEMBER 4, 2021

To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Big-Time Data Hightouch Podcast Episode Segment Podcast Episode Customer Data Platform Treasure Data Rudderstack Airflow DBT Cloud Fivetran Podcast Episode Stitch PLG == Product Led Growth ABM == Account Based Marketing Materialize Podcast (..)

Data Warehouse 100

Data Warehouse Business Intelligence ETL System Data Lake 100

Monte Carlo

SEPTEMBER 19, 2023

In addition to our Keynotes, we’ll dive into some of the hottest topics in data, like Generative AI, the modern data stack, data products, and more with a lineup of leaders from across the data space, including: Nga Phan, SVP of Product Management at Salesforce Tristan Handy, Co-Founder and CEO of dbt Labs Krishnan Parasuraman, VP and CTO of Global (..)

Insurance 52

Insurance Software Engineer Software Engineering Data 52

Data Engineering Podcast

JANUARY 1, 2022

Then the problem was data integration and transformation which was solved by data integration vendors and frameworks such as Fivetran / Airbyte, modern orchestration frameworks such as Dagster & dbt and “reverse-ETL” Hightouch. What is the main challenge now? Will SQL be challenged as a primary interface to analytical data?

Data Warehouse 100

Data Warehouse Data Lake Hadoop SQL 100

Monte Carlo

OCTOBER 31, 2023

Pioneering the Modern Data & AI Stack with Tristan Handy , CEO and co-founder of dbt Labs, George Fraser , CEO and co-founder of Fivetran, and Prukalpa Sankar , co-founder of Atlan, about what’s next in MDS tech, particularly as it relates to readying our pipelines for AI applications. Another AI-focused session not to miss?

Data Engineering 72

Data Engineering Data Engineer Data Engineering 72

Data Engineering Podcast

AUGUST 13, 2022

Email hosts@dataengineeringpodcast.com ) with your story. Email hosts@dataengineeringpodcast.com ) with your story.

Metadata 100

Metadata MongoDB Scala MySQL 100

Data Engineering Podcast

FEBRUARY 15, 2021

To help other people find the show please leave a review on iTunes and tell your friends and co-workers Join the community in the new Zulip chat workspace at dataengineeringpodcast.com/chat Links 5xData Looker Podcast Episode Snowflake Podcast Episode Fivetran Podcast Episode DBT Podcast Episode The intro and outro music is from The Hug by The Freak (..)

Building 100

Building Data Warehouse BI Data Pipeline 100

Monte Carlo

MARCH 10, 2022

In addition to Monte Carlo, former and current data companies in the Enterprise Tech 30 include Snowflake, Looker, Databricks, Confluent, and Fivetran. In the data industry, the list frequently includes organizations primed to drive the future of the modern data stack. Talk to us!

Technology 52

Technology Data 52

Data Engineering Podcast

JANUARY 20, 2020

To help other people find the show please leave a review on iTunes and tell your friends and co-workers Join the community in the new Zulip chat workspace at dataengineeringpodcast.com/chat Links Mayvenn Ascend Podcast Episode Google Sawzall Clickstream Apache Kafka Alooma Podcast Episode Amazon Redshift ELT == Extract, Load, Transform DBT Podcast (..)

Kafka 100

Kafka Media Architecture Data Pipeline 100

Data Engineering Podcast

JULY 12, 2021

To help other people find the show please leave a review on iTunes and tell your friends and co-workers Join the community in the new Zulip chat workspace at dataengineeringpodcast.com/chat Links GoDataDriven Deloitte RPA == Robotic Process Automation Analytics Engineer James Webb Space Telescope Fivetran Podcast Episode dbt Podcast Episode Data Governance (..)

Designing 100

Designing Data Warehouse Data Governance Consulting 100

Data Engineering Podcast

DECEMBER 8, 2019

Links SnowflakeDB Free Trial Stack Overflow Data Warehouse Oracle DB MPP == Massively Parallel Processing Shared Nothing Architecture Multi-Cluster Shared Data Architecture Google BigQuery AWS Redshift AWS Redshift Spectrum Presto Podcast Episode SnowflakeDB Semi-Structured Data Types Hive ACID == Atomicity, Consistency, Isolation, Durability 3rd Normal (..)

Data Warehouse 100

Data Warehouse Cloud AWS Relational Database 100