article thumbnail

New Fivetran connector streamlines data workflows for real-time insights

ThoughtSpot

The pathway from ETL to actionable analytics can often feel disconnected and cumbersome, leading to frustration for data teams and long wait times for business users. And even when we manage to streamline the data workflow, those insights aren’t always accessible to users unfamiliar with antiquated business intelligence tools.

article thumbnail

Introducing WorkflowGuard: The Workflow Governance and Observability System That Oversees over 120,000 Data Workflows

Uber Engineering

Our Data Workflow Platform team introduces WorkflowGuard: a new service to govern executions, prioritize resources, and manage life cycle for repetitive data jobs. Check out how it improved workflow reliability and cost efficiency while bringing more observability to users.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Managing Uber’s Data Workflows at Scale

Uber Engineering

At Uber’s scale, thousands of microservices serve millions of rides and deliveries a day, generating more than a hundred petabytes of raw data. Internally, engineering and data teams across the company leverage this data to improve the Uber experience.

article thumbnail

Automate data workflows using AWS Data Pipeline

Medium Data Engineering

If you’re looking to ETL your data using AWS, one option is to use their Data Pipeline service. Data Pipeline is a web service that helps… Continue reading on Medium »

article thumbnail

Revolutionize Data Workflow with Airflow ETL Pipeline in Python

Medium Data Engineering

Explore how to design scalable workflows with Airflow in Python to manage the ETL pipeline. Continue reading on Simform Engineering »

Python 52
article thumbnail

Want to streamline your data workflows and improve collaboration across teams?

Medium Data Engineering

DataOps combines the best practices of Agile, DevOps, and Lean to manage data-driven workflows and deliver high-quality data products and… Continue reading on Medium »

article thumbnail

How to Automate PySpark Pipelines on AWS EMR With Airflow

Medium Data Engineering

Optimising big data workflows orchestration Continue reading on Towards Data Science »

AWS 98