Snowflake

OCTOBER 2, 2023

To help customers overcome these challenges, RudderStack and Snowflake recently launched Profiles , a new product that allows every data team to build a customer 360 directly in their Snowflake Data Cloud environment. Now teams can leverage their existing data engineering tools and workflows to build their customer 360.

Cloud 100

Cloud Building Insurance Data Engineering 100

Precisely

MAY 13, 2024

If you need to build stronger, faster, and flexible data integration, use this checklist of ten key attributes to streamline your search and find the right technology for your organization, faster. #1. It synthesizes all the metadata around your organization’s data assets and arranges the information into a simple, easy-to-understand format.

Data Integration 52

Data Integration Metadata Amazon Web Services Data Governance 52

Data Council

JANUARY 21, 2021

Storing Cold Metadata with Alki (Dropbox) Dropbox shared insights into Alki , the petabyte-scale metadata store it designed for infrequently accessed metadata (“cold data”). Here's our January 2021 roundup of links from across the web that could be relevant to you: 1.

Metadata 52

Metadata AWS Cloud Database 52

Netflix Tech

SEPTEMBER 28, 2021

The Revenue and Growth Tools (RGT) team decided to set off on a journey to build tools in an abstract manner to have solutions readily available within our organization. The traditional approach of bundling dependencies at build time lacks the flexibility we need to empower our stakeholders.

Building 96

Building Metadata Designing Coding 96

Data Engineering Podcast

NOVEMBER 26, 2023

Summary Building a data platform that is enjoyable and accessible for all of its end users is a substantial challenge. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Architecture 147

Architecture Data Lake High Quality Data SQL 147

Data Engineering Podcast

FEBRUARY 13, 2022

To simplify the work of managing the full flow of your customer data and keep you in full control the team at Rudderstack created their eponymous open source platform that allows you to work with first and third party data, as well as build and manage reverse ETL workflows.

Building 100

Building Hadoop Data Pipeline Metadata 100

Data Engineering Podcast

NOVEMBER 9, 2020

Summary A data catalog is a critical piece of infrastructure for any organization who wants to build analytics products, whether internal or external. While there are a number of platforms available for building that catalog, many of them are either difficult to deploy and integrate, or expensive to use at scale.

Building 100

Building PostgreSQL BI Metadata 100

Snowflake

APRIL 11, 2024

At Snowflake, we believe in making the power of data accessible to all. That’s why we prioritize simplicity, governance and quality in everything we build – including our AI-powered tools. Snowflake’s AI research team is at the forefront, exploring the intricacies of building a powerful SQL LLM.

SQL 136

SQL AWS Data Analysis High Quality Data 136

Data Engineering Podcast

JANUARY 23, 2022

He also provides some excellent insights into the factors that play into the build vs. buy decision at different organizational sizes. You can observe your pipelines with built in metadata search and column level lineage. You can observe your pipelines with built in metadata search and column level lineage.

Building 100

Building Management Data Pipeline Metadata 100

Data Engineering Podcast

FEBRUARY 5, 2023

Sign up now for early access to Materialize and get started with the power of streaming data with the same simplicity and low implementation cost as batch cloud data warehouses.

Data Engineering 147

Data Engineering Data Engineer Engineering PostgreSQL 147

The Modern Data Company

JANUARY 22, 2024

This strategic vision has not only broken down data silos but has enabled clients to build a quality and sustainable data initiative that reaches every single department. Data engineering excellence Modern offers robust solutions for building, managing, and operationalizing data pipelines.

Data Integration 98

Data Integration Metadata Government Unstructured Data 98

Databand.ai

AUGUST 30, 2023

Building a Winning Data Quality Strategy: Step by Step Eitan Chazbani August 30, 2023 What Is a Data Quality Strategy? This includes defining roles and responsibilities related to managing datasets and setting guidelines for metadata management. This starts with building a strong business case for your data quality strategy.

Building 52

Building Data Cleanse Data Governance Datasets 52

ThoughtSpot

OCTOBER 9, 2023

ThoughtSpot continually invests in this area, giving users the confidence to build the correct Answers needed for their analysis—and ensuring they can trust the data they are shown. How ThoughtSpot builds trust with data catalog connectors For many, the data catalog is still the primary home for metadata enrichment and governance.

Metadata 83

Metadata Government Data Data Governance 83

Data Engineering Podcast

FEBRUARY 19, 2023

It's supposed to make building smarter, faster, and more flexible data infrastructures a breeze. By bringing all the layers of the data stack together, TimeXtender helps you build data solutions up to 10 times faster and saves you 70-80% on costs. Acryl]([link] The modern data stack needs a reimagined metadata management platform.

IT 147

IT Data Lake Metadata Data Warehouse 147

Data Engineering Podcast

OCTOBER 30, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Business Intelligence 130

Business Intelligence Metadata MongoDB MySQL 130

Data Engineering Podcast

NOVEMBER 20, 2022

Summary The majority of blog posts and presentations about data engineering and analytics assume that the consumers of those efforts are internal business users accessing an environment controlled by the business. Atlan is the metadata hub for your data ecosystem.

Systems 130

Systems Metadata Data Pipeline MongoDB 130

Data Engineering Podcast

DECEMBER 4, 2022

The typical conception of how it is accessed is through a web or desktop application running on a powerful laptop. Zing Data is building a mobile native platform for business intelligence. In this episode Sabin Thomas shares his experiences building the platform and the interesting ways that it is being used.

Business Intelligence 100

Business Intelligence Metadata BI MongoDB 100

François Nguyen

MARCH 22, 2021

How do we build data products ? How to build Data Products or never call me Data Pipeline any more You have this interesting schema in her second article on Data Mesh by Zhamak Dehghani : “Data mesh introduces the concept of data product as its architectural quantum. .” What you have to code is this workflow !

Architecture 130

Architecture Technology Google Cloud Metadata 130

Engineering at Meta

MARCH 18, 2024

Users can query using regular expressions on log lines, arbitrary metadata fields attached to logs, and across log files of hosts and services. Each log line can have zero or more metadata key-value pairs attached to it. The extracted key-value pairs are added to the log line’s metadata. in PyTorch). Multimodal data (e.g.,

Engineering 91

Engineering Metadata Architecture Designing 91

Monte Carlo

MAY 31, 2022

Metadata from the data warehouse/lake and from the BI tool of record can then be used to map the dependencies between the tables and dashboards. onFailure(funcName, qe, exception)) } } Spline provided a head-start in building Spark lineage. A good engineer can build solutions for hard problems. It’s a really elegant solution….here’s

Data Lake 52

Data Lake Building Scala Metadata 52

Knowledge Hut

JANUARY 2, 2024

You will learn how to build up Kube-state-metrics system, pull and collect metrics, deploy a Prometheus server and metrics exporters, configure alerts with Alertmanager, and create Grafana dashboards. Monitoring had to be made more accessible, democratized and expanded to include additional stack tiers.

Architecture 98

Architecture Metadata Utilities Data Collection 98

Snowflake

NOVEMBER 29, 2023

In medicine, lower sequencing costs and improved clinical access to NGS technology has been shown to increase diagnostic yield for a range of diseases, from relatively well-understood Mendelian disorders, including muscular dystrophy and epilepsy , to rare diseases such as Alagille syndrome.

Metadata 83

Metadata Healthcare Medical Data Storage 83

Towards Data Science

FEBRUARY 6, 2024

Accessing Operational Data I used to connect to views in transactional databases or APIs offered by operational systems to request the raw data. Accessibility : I could easily request access to these data products. Additionally, it is really hard to identify the owners, one of them has even recently left the company.

Systems 83

Systems Raw Data Metadata Data Cleanse 83

Snowflake

JANUARY 23, 2024

We have been making it easier and faster to build and manage ML models with Snowpark ML , the Python library and underlying infrastructure for end-to-end ML workflows in Snowflake. Many developers and enterprises looking to use machine learning (ML) to generate insights from data get bogged down by operational complexity.

Machine Learning 118

Machine Learning Metadata Python Telecommunication 118

The Modern Data Company

JANUARY 22, 2024

This strategic vision has not only broken down data silos but has enabled clients to build a quality and sustainable data initiative that reaches every single department. Data products contain data and its metadata, code, and infrastructure so that each product is self-contained and usable independently.

Data Integration 87

Data Integration Metadata Government Unstructured Data 87

Monte Carlo

AUGUST 19, 2023

In today’s data-driven landscape, building a data platform is no longer a nice-to-have, but a necessity for most organizations. I’ve also updated this post to include additional layers frequently found within a modern data platform such as data governance, access management, and machine learning. It depends.

Building 52

Building BI Data Lake Data Governance 52

Data Engineering Podcast

AUGUST 28, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Engineering 100

Data Engineering Data Engineer MongoDB Metadata 100

Snowflake

DECEMBER 12, 2023

To give customers flexibility for how they fit Snowflake into their data architecture, Iceberg Tables can be configured to use either Snowflake or an external service such as AWS Glue as the table’s catalog to track metadata, with an easy, one-line SQL command to convert the table’s catalog to Snowflake in a metadata-only operation.

Metadata 89

Metadata Python AWS Government 89

LinkedIn Engineering

MARCH 21, 2023

That’s why we believe a skills-first approach to hiring talent will help companies gain access to wider talent pools to find the skilled workers their businesses need, especially in sectors that are aggressively looking for talent. What is the skills taxonomy?

Building 85

Building Recruitment Machine Learning Deep Learning 85

Data Engineering Podcast

AUGUST 21, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 100

Metadata MongoDB MySQL Data Lake 100

Data Engineering Podcast

JULY 31, 2022

In this episode Ernie Ostic shares the approach that he and his team at Manta are taking to build a complete view of data lineage across the various data systems in your organization and the useful applications of that information in the work of every data stakeholder. Atlan is the metadata hub for your data ecosystem.

IT 100

IT Metadata MongoDB MySQL 100

LinkedIn Engineering

JUNE 2, 2023

This data must be accessible and available in near real-time in order to accurately reflect current trends and conditions. Discoverability: Consumers must be able to easily discover and access the correct data sources for their needs, whether it is stored in a centralized repository or scattered throughout the wider data landscape.

Building 75

Building Banking Datasets Media 75

Towards Data Science

MAY 7, 2024

meetings with sales reps from all of the SaaS data integration tooling companies and are granted 14 day access to try their wares. This will help create an overall picture of logging, metadata, latency, CDC, and all the other things that come with a pipeline. Gather the team together and determine what these things are.

ETL Tools 84

ETL Tools Metadata Data Integration Data Science 84

Snowflake

APRIL 23, 2024

The connector provides immediate access to up-to-date ServiceNow data without the need to manually integrate against API endpoints. Customers get a fully managed service with no additional access fees set by Snowflake. It’s available with no additional access fees set by Snowflake and uses Snowflake credits for compute and storage.

Data Warehouse 93

Data Warehouse Data Integration Consulting Metadata 93

Data Engineering Podcast

AUGUST 6, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 100

Metadata MongoDB MySQL Business Intelligence 100

Data Engineering Podcast

MARCH 18, 2023

Data Council attendees are amazing founders, data scientists, lead engineers, CTOs, heads of data, investors and community organizers who are all working together to build the future of data. Visit: dataengineeringpodcast.com/data-council today RudderStack makes it easy for data teams to build a customer data platform on their own warehouse.

Data Security 130

Data Security Machine Learning Data Science Metadata 130

Striim

FEBRUARY 5, 2024

It is not a standalone product but comprises key elements, including data integration, ensuring the smooth merging of data; data quality, maintaining high data standards; metadata management, organizing and understanding data context; and security, safeguarding data integrity.

Pharmaceutical 52

Pharmaceutical Data Cleanse Metadata Medical 52