Cloudera

AUGUST 10, 2021

With the release of CDP Private Cloud (PvC) Base 7.1.7, Atlas / Kafka integration provides metadata collection for Kafa producers/consumers so that consumers can manage, govern, and monitor Kafka metadata and metadata lineage in the Atlas UI. Figure 8: Data lineage based on Kafka Atlas Hook metadata.

Cloud 96

Cloud Kafka Metadata SQL 96

Precisely

MAY 13, 2024

Key Takeaways: Data integration is vital for real-time data delivery across diverse cloud models and applications, and for leveraging technologies like generative AI. Whether it’s a cloud data warehouse or a mainframe, look for vendors who have a wide range of capabilities that can adapt to your changing needs.

Data Integration 52

Data Integration Metadata Amazon Web Services Data Governance 52

Acceldata

MARCH 2, 2023

Learn how to use Acceldata's cloud data observability platform to optimize queries for query history metadata.

Metadata 40

Metadata Cloud Data 40

phData: Data Engineering

OCTOBER 12, 2022

So you’ve heard all the talk around dbt , but now you’re working to determine if you should go with dbt Core or dbt Cloud and you’re wanting to know what advantages dbt Cloud has over the free dbt Core offering. However, dbt Cloud offers you much more than that.

Cloud 52

Cloud Metadata SQL Data Warehouse 52

dbt Developer Hub

SEPTEMBER 14, 2021

I'm sure you know that half of the updates in this email are pushed automatically when we upgrade everyone to the latest version of dbt Cloud ? yaml files) Capture changes to macros in state:modified, for best-yet Slim CI dbt Cloud v1.1.32 - v1.1.35: ​ Changelog and additional assets located here. *

Metadata 52

Metadata Data Pipeline Python Cloud 52

Snowflake

JANUARY 19, 2023

Snowflake connected with Sridevi Potu, McKesson’s Director of Data Analytics Platform Services, at the Data Cloud World Tour in Austin, Texas. Learn about the company’s journey to Snowflake and how it’s implementing a cross-cloud data strategy. Having a multi-cloud approach has many advantages beyond just BCDR,” said Potu.

Cloud 52

Cloud Pharmaceutical Healthcare Transportation 52

Cloudera

AUGUST 26, 2021

The object store is readily available alongside HDFS in CDP (Cloudera Data Platform) Private Cloud Base 7.1.3+. Before we jump into the data ingestion step, here is a quick overview of how Ozone manages its metadata namespace through volumes, buckets and keys. . Ozone Namespace Overview. Data ingestion through ‘s3’.

Data Science 109

Data Science Cloud Hadoop Metadata 109

Snowflake

FEBRUARY 2, 2023

During a cloud migration to Snowflake’s Data Cloud, businesses often struggle to know what data they have on premises, what they should migrate, and in what order. And because of this, many organizations fall into a “lift and shift” approach, where everything is simply copied over—as it messily stands—to the cloud.

Cloud 86

Cloud Data Data Governance Government 86

Monte Carlo

OCTOBER 12, 2023

While Monte Carlo sends queries to monitor customer resources, our cloud service doesn’t need to store record level data to deliver the alerts and insights described above. The only information collected by Monte Carlo is table metadata, aggregated metrics, query logs, and anonymized statistics generated across your modern data stack.

Cloud 52

Cloud AWS Metadata Architecture 52

Snowflake

DECEMBER 4, 2023

With this public preview, those external catalog options are either “GLUE”, where Snowflake can retrieve table metadata snapshots from AWS Glue Data Catalog, or “OBJECT_STORE”, where Snowflake retrieves metadata snapshots directly from the specified cloud storage location. With these three options, which one should you use?

Building 95

Building Metadata Cloud Storage AWS 95

Cloudera

SEPTEMBER 28, 2020

Cloudera delivers an enterprise data cloud that enables companies to build end-to-end data pipelines for hybrid cloud, spanning edge devices to public or private cloud, with integrated security and governance underpinning it to protect customers data. Review the Upgrade document topic for the supported upgrade paths.

Cloud 131

Cloud Kafka Professional Services Metadata 131

Cloudera

APRIL 18, 2024

In those discussions, it was clear that everyone understood the need to treat data estates more cohesively as a whole—that means bringing more attention to security, data governance, and metadata management, the latter of which has become increasingly popular.

Metadata 105

Metadata Government Data Governance Data 105

Christophe Blefari

MARCH 1, 2023

dbt Labs also develop dbt Cloud which is a cloud product that hosts and runs dbt Core projects. dbt was born out of the analysis that more and more companies were switching from on-premise Hadoop data infrastructure to cloud data warehouses. With the public clouds—e.g. The company has been founded in May 2016.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

Knowledge Hut

DECEMBER 21, 2023

Cloud computing is a fast-growing area in the technical landscape due to its recent developments. If we look ahead to 2024, there are new research topics in cloud computing that are getting more traction among researchers and practitioners. To master a cloud computing field, we need to check these Cloud Computing online courses.

Cloud Computing 52

Cloud Computing Cloud Algorithm Healthcare 52

Cloudera

NOVEMBER 12, 2020

Cloud has given us hope, with public clouds at our disposal we now have virtually infinite resources, but they come at a different cost – using the cloud means we may be creating yet another series of silos, which also creates unmeasurable new risks in security and traceability of our data. A solution.

Cloud 102

Cloud Data Warehouse Banking Data 102

Cloudera

JULY 15, 2021

The release of Cloudera Data Platform (CDP) Private Cloud Base edition provides customers with a next generation hybrid cloud architecture. Private Cloud Base Overview. The storage layer for CDP Private Cloud, including object storage. Traditional data clusters for workloads not ready for cloud. Edge or Gateway.

Architecture 97

Architecture Cloud Kafka Hadoop 97

Cloudera

JUNE 25, 2021

Many Cloudera customers are making the transition from being completely on-prem to cloud by either backing up their data in the cloud, or running multi-functional analytics on CDP Public cloud in AWS or Azure. Configure the required ports to enable connectivity from CDH to CDP Public Cloud (see docs for details).

Cloud 69

Cloud Data Lake Cloud Storage Metadata 69

Cloudera

JANUARY 19, 2022

Gartner® recognized Cloudera in three recent reports – Magic Quadrant for Cloud Database Management Systems (DBMS), Critical Capabilities for Cloud Database Management Systems for Analytical Use Cases and Critical Capabilities for Cloud Database Management Systems for Operational Use Cases.

Database 87

Database Cloud Data Warehouse Data Lake 87

Cloudera

FEBRUARY 17, 2022

CDP Private Cloud Base is an on-premises version of Cloudera Data Platform (CDP). Provides a consistent experience on Public Cloud, Multi-Cloud, and Private Cloud deployments. One of our previous blogs discussed the four paths to get from legacy platforms to CDP Private Cloud Base. How to upgrade from HDP to CDP.

Cloud 96

Cloud Data Professional Services Metadata 96

Data Engineering Podcast

FEBRUARY 19, 2023

Summary Cloud data warehouses have unlocked a massive amount of innovation and investment in data applications, but they are still inherently limiting. Acryl]([link] The modern data stack needs a reimagined metadata management platform. Acryl]([link] The modern data stack needs a reimagined metadata management platform.

IT 147

IT Data Lake Metadata Data Warehouse 147

dbt Developer Hub

JULY 2, 2023

dbt Cloud: the operating system for Data Vault ​ There are many tools that can be used to implement your Data Vault project but dbt Cloud with its rich set of features provides the functionalities that make the difference by accelerating your project end to end, saving you the trouble of jumping from one tool to another.

Cloud 52

Cloud Data Warehouse Data BI 52

Cloudera

APRIL 2, 2024

I am pleased to announce that Cloudera has achieved FedRAMP “In Process”, a significant milestone that underscores our commitment to providing the public sector with secure and reliable data management solutions across on-prem, hybrid and multi-cloud environments.

Designing 96

Designing Process Government Metadata 96

Booking.com Engineering

DECEMBER 2, 2022

In this article, we want to illustrate our extensive use of the public cloud, specifically Google Cloud Platform (GCP). As an example, in one of our first BigQuery aggregations, we had a large query that joined statistics data with metadata, then aggregated over it. Booking Holdings, as a whole, spent $4.7

Systems 52

Systems Cloud MySQL Relational Database 52

François Nguyen

MARCH 22, 2021

It will be illustrated with our technical choices and the services we are using in the Google Cloud Platform. With this 3rd platform generation, you have more real time data analytics and a cost reduction because it is easier to manage this infrastructure in the cloud thanks to managed services.

Architecture 130

Architecture Technology Google Cloud Metadata 130

Cloudera

OCTOBER 15, 2021

Apache Ozone is a distributed, scalable, and high performance object store, available with Cloudera Data Platform Private Cloud. CDP Private Cloud uses Ozone to separate storage from compute, which enables it to handle billions of objects on-premises, akin to Public Cloud deployments which benefit from the likes of S3.

Cloud 86

Cloud Hadoop Data Analytics Metadata 86

Cloudera

SEPTEMBER 30, 2021

Our recent blog discussed the four paths to get from legacy platforms to CDP Private Cloud Base. In this blog and accompanying video, we will deep dive into the mechanics of running an in-place upgrade from CDH5 or CDH6 to CDP Private Cloud Base. As of CDP Private Cloud Base 7.1.6 CDP Private Cloud Base Upgrade documentation.

Cloud 70

Cloud PostgreSQL Metadata MySQL 70

Data Engineering Podcast

NOVEMBER 13, 2022

With the rise in cloud platforms and self-serve data technologies the barrier of entry is dropping. Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities.

Building 130

Building Metadata MongoDB MySQL 130

Cloudera

JANUARY 15, 2021

Cloud data warehouses allow users to run analytic workloads with greater agility, better isolation and scale, and lower administrative overhead than ever before. DW1 is an anonymized cloud data warehouse running on AWS and DW2 is an anonymized data warehouse running on GCP. Introduction. But don’t just take our word for it.

Data Warehouse 92

Data Warehouse Cloud Consulting SQL 92

ProjectPro

JANUARY 24, 2023

A survey by Data Warehousing Institute TDWI found that AWS Glue and Azure Data Factory are the most popular cloud ETL tools with 69% and 67% of the survey respondents mentioning that they have been using them. Azure Data Factory and AWS Glue are powerful tools for data engineers who want to perform ETL on Big Data in the Cloud.

AWS 52

AWS Cloud Amazon Web Services ETL Tools 52

Data Engineering Podcast

OCTOBER 30, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Atlan is the metadata hub for your data ecosystem. And don’t forget to thank them for their continued support of this show!

Business Intelligence 130

Business Intelligence Metadata MongoDB MySQL 130

Cloudera

FEBRUARY 9, 2021

Today, more and more customers are moving workloads to the public cloud for business agility where cost-saving and management are key considerations. Cloud object storage is used as the main persistent storage layer, which is significantly cheaper than block volumes. Avro Schema without Kafka Metadata Example. {. Cost-Effective.

Data Warehouse 69

Data Warehouse Cloud Kafka Cloud Storage 69

Data Engineering Podcast

JUNE 19, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Atlan is the metadata hub for your data ecosystem. And don’t forget to thank them for their continued support of this show!

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Towards Data Science

MAY 7, 2024

These are primarily collected with cloud based integration platforms such as, but not limited to Fivetran, Airbyte, and Rivery in mind, but could apply to other cases as well! This will help create an overall picture of logging, metadata, latency, CDC, and all the other things that come with a pipeline.

ETL Tools 84

ETL Tools Metadata Data Integration Data Science 84

Precisely

JUNE 6, 2023

Tendü Yogurtçu offered that organizations today are viewing data as a product, and they are working to streamline access to both data and metadata for faster insights that help business users deliver tangible value. Salam noted that organizations are offloading computational horsepower and data from on-premises infrastructure to the cloud.

Data Architecture 52

Data Architecture Architecture Metadata Data Lake 52

Knowledge Hut

APRIL 4, 2024

Chef is compatible with many different cloud platforms. You can think of a cookbook as a collection of all the recipes, files, characteristics, and metadata you'll need to implement a specific scenario. Metadata A small bit of metadata is required for every cookbook. physical, virtual, cloud, network device, etc.

Architecture 52

Architecture Amazon Web Services Metadata AWS 52

Netflix Tech

MARCH 5, 2019

All these micro-services are currently operated in AWS cloud infrastructure. Finally, provisioning our infrastructure itself is also becoming an increasingly complex task, so our data teams contribute to tools for diagnosis and automation of our cloud capacity management.

Cloud 73

Cloud Building Amazon Web Services Metadata 73

ThoughtSpot

MAY 31, 2023

Architecture Let's start with the big picture and tackle how we adjusted our cloud architecture with additional internal and external interfaces to integrate LLM. This multi-tenant service isolates the tenant metadata index, authorizing and filtering the search answer requests from every tenant.

Data Security 86

Data Security Metadata Transportation Data Warehouse 86