Data Engineering Podcast

JUNE 5, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking all of that information into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Go to dataengineeringpodcast.com/atlan today to learn more about how you can take advantage of active metadata and escape the chaos.

Data Security 100

Data Security Metadata MongoDB Scala 100

Cloudera

MARCH 31, 2021

CDP Public Cloud is now available on Google Cloud. The addition of support for Google Cloud enables Cloudera to deliver on its promise to offer its enterprise data platform at a global scale. CDP Public Cloud is already available on Amazon Web Services and Microsoft Azure.

Google Cloud 92

Google Cloud Cloud Amazon Web Services Cloud Storage 92

Precisely

MAY 13, 2024

Key Takeaways: Data integration is vital for real-time data delivery across diverse cloud models and applications, and for leveraging technologies like generative AI. As enterprise technology landscapes grow more complex, the role of data integration is more critical than ever before.

Data Integration 52

Data Integration Metadata Amazon Web Services Data Governance 52

Cloudera

APRIL 2, 2024

I am pleased to announce that Cloudera has achieved FedRAMP “In Process”, a significant milestone that underscores our commitment to providing the public sector with secure and reliable data management solutions across on-prem, hybrid and multi-cloud environments.

Designing 99

Designing Process Government Metadata 99

Cloudera

DECEMBER 16, 2022

We are pleased to announce that Cloudera has been named a Leader in the 2022 Gartner ® Magic Quadrant for Cloud Database Management Systems. Cloudera has been recognized in this cloud DBMS report since its inception in 2020. We do it today when data is even bigger, and hybrid — and clouds — are expensive. This is unique.

Database 90

Database Cloud Systems Management 90

Knowledge Hut

DECEMBER 21, 2023

Cloud computing is a fast-growing area in the technical landscape due to its recent developments. If we look ahead to 2024, there are new research topics in cloud computing that are getting more traction among researchers and practitioners. To master a cloud computing field, we need to check these Cloud Computing online courses.

Cloud Computing 52

Cloud Computing Cloud Algorithm Healthcare 52

Cloudera

NOVEMBER 5, 2020

How do you control data privacy and protect against data breaches when the data is spread across so many different systems? How do you optimize your enterprise-wide infrastructure (mostly cloud) and application expenditures? Main Security Features. IDBroker — identity federation, cloud credentials.

Cloud 72

Cloud Data Lake Data Warehouse Metadata 72

Cloudera

MAY 17, 2021

Making a decision on a cloud data warehouse is a big deal. Modernizing your data warehousing experience with the cloud means moving from dedicated, on-premises hardware focused on traditional relational analytics on structured data to a modern platform.

Data Warehouse 89

Data Warehouse Cloud Government Metadata 89

François Nguyen

MARCH 22, 2021

TL;DR After setting up and organizing the teams, we are describing 4 topics to make data mesh a reality. How can we interoperate between the data domains ? How do we govern all these data products and domains ? It will be illustrated with our technical choices and the services we are using in the Google Cloud Platform.

Technology 130

Technology Architecture Google Cloud Metadata 130

Cloudera

NOVEMBER 12, 2020

Cloud has given us hope, with public clouds at our disposal we now have virtually infinite resources, but they come at a different cost – using the cloud means we may be creating yet another series of silos, which also creates unmeasurable new risks in security and traceability of our data. A solution.

Cloud 103

Cloud Data Warehouse Banking Data 103

Cloudera

MARCH 3, 2022

Data analytics and machine learning can become a business and a compliance risk if data security, governance, lineage, metadata management, and automation are not holistically applied across the entire data lifecycle and all environments. One possible solution is to adopt a hybrid cloud strategy. .

Data Governance 98

Data Governance Government Metadata Medical 98

Cloudera

OCTOBER 7, 2021

In this blog, I will demonstrate the value of Cloudera DataFlow (CDF) , the edge-to-cloud streaming data platform available on the Cloudera Data Platform (CDP) , as a Data integration and Democratization fabric. Data and Metadata: Data inputs and data outputs produced based on the application logic.

Architecture 124

Architecture Metadata Government Kafka 124

Striim

FEBRUARY 5, 2024

Data Fabric is a comprehensive data management approach that goes beyond traditional methods , offering a framework for seamless integration across diverse sources. By upholding data quality, organizations can trust the information they rely on for decision-making, fostering a data-driven culture built on dependable insights.

Pharmaceutical 52

Pharmaceutical Data Cleanse Metadata Medical 52

Christophe Blefari

JUNE 16, 2023

The article shares the recent features and says also something I will never forget: " Malloy aims to be syntactically the same no matter what database contains the data " The Astro Cloud IDE — Astronomer released a bunch of Airflow operators to their Cloud IDE (which was released in Dec. but I missed it).

Programming Language 130

Programming Language SQL PostgreSQL Data 130

Cloudyard

DECEMBER 7, 2023

Auto-Fulfillment: As a provider, you can enable Cross-Cloud Auto-Fulfillment for a listing to automatically replicate your data product to other Snowflake regions without having to manually replicate data. Consumer will create a Database from listing and Query the objects.

Metadata 52

Metadata SQL Data Cloud 52

Netflix Tech

MARCH 5, 2019

All these micro-services are currently operated in AWS cloud infrastructure. As a micro-service owner, a Netflix engineer is responsible for its innovation as well as its operation, which includes making sure the service is reliable, secure, efficient and performant. Can we measure the impact of Inclusion and Diversity initiatives?

Cloud 73

Cloud Building Amazon Web Services Metadata 73

Snowflake

JANUARY 22, 2024

For the past few years, Fidelity Investments has been moving a significant percentage of its applications to a cloud-based infrastructure. In 2020, Fidelity kicked off its digital transformation and established an Enterprise Data Lake (EDL) along with Data Labs.

Engineering 63

Engineering Data Lake Unstructured Data Metadata 63

Cloudera

AUGUST 4, 2021

With hackers now working overtime to expose business data or implant ransomware processes, data security is largely IT managers’ top priority. And if data security tops IT concerns, data governance should be their second priority. Availability: Make the right data available only to the right users.

Data Governance 85

Data Governance Government Metadata Data Security 85

Data Engineering Podcast

FEBRUARY 13, 2022

Today’s episode is Sponsored by Prophecy.io – the low-code data engineering platform for the cloud. Prophecy provides an easy-to-use visual interface to design & deploy data pipelines on Apache Spark & Apache Airflow. You can observe your pipelines with built in metadata search and column level lineage.

Building 100

Building Hadoop Data Pipeline Metadata 100

Cloudera

AUGUST 3, 2021

In recent years the Cloud has been seen as a solution and panacea for many companies digital transformation strategies. However a recent Andereessen Horowitz study has shown that while the Cloud is a viable solution for start-up, expanding and emerging use cases, its true cost on market capitalization is vastly underestimated.

Manufacturing 109

Manufacturing Retail Entertainment Government 109

Data Engineering Podcast

FEBRUARY 6, 2022

Today’s episode is Sponsored by Prophecy.io – the low-code data engineering platform for the cloud. Prophecy provides an easy-to-use visual interface to design & deploy data pipelines on Apache Spark & Apache Airflow. You can observe your pipelines with built in metadata search and column level lineage.

Data Pipeline 100

Data Pipeline Data Metadata Software Engineer 100

Cloudera

NOVEMBER 22, 2021

Explosion of data availability from a variety of sources, including on-premises data stores used by enterprise data warehousing / data lake platforms, data on cloud object stores typically produced by heterogenous, cloud-only processing technologies, or data produced by SaaS applications that have now evolved into distinct platform ecosystems (e.g.,

Government 92

Government Hadoop Data Security Data Warehouse 92

Monte Carlo

APRIL 24, 2023

Traditionally, after being stored in a data lake, raw data was then often moved to various destinations like a data warehouse for further processing, analysis, and consumption. Databricks Data Catalog and AWS Lake Formation are examples in this vein. It works in both directions.

Data Lake 52

Data Lake Google Cloud Data Warehouse AWS 52

AltexSoft

FEBRUARY 11, 2023

Data architecture is the organization and design of how data is collected, transformed, integrated, stored, and used by a company. If KPI goals are not met, a data architect recommends solutions (including new technologies) to improve the existing framework.

Data Architect 87

Data Architect Certification Generalist Big Data 87

Cloudera

APRIL 21, 2022

But thanks to cloud services and a next-generation electronic health-care record (EHR) system that has taken more than two decades to develop and deploy, human error has been cut drastically and a system has finally evolved from paper-based to near real-time. Shared catalog of data, metadata aids compliance requirements.

Electronics 86

Electronics Medical Hospitality Insurance 86

Cloudera

NOVEMBER 17, 2022

With Cloudera Data Platform, we aim to unlock value faster and offer consistent data security and governance to meet this goal. The Solution: CDP Private Cloud brings a next-generation hybrid architecture with cloud-native benefits to HBL’s data platform.

Banking 84

Banking Management Data Lake Professional Services 84

AltexSoft

NOVEMBER 23, 2021

Nowadays, all organizations need real-time data to make instant business decisions and bring value to their customers faster. But this data is all over the place: It lives in the cloud, on social media platforms, in operational systems, and on websites, to name a few. Enhanced data security and governance.

Process 69

Process Data Lake Metadata Data Warehouse 69

François Nguyen

MARCH 7, 2021

In this article, Juan Sequada gives maybe one of the best definition of Data Mesh ” It is paradigm shift towards a distributed architecture that attempts to find an ideal balance between centralization and decentralization of metadata and data management.”

Government 130

Government Data Governance Data Metadata 130

Precisely

DECEMBER 8, 2023

Together, these forces have pushed companies to accelerate the shift to technologies like Cloud, AI, and workflow automation. In the context of this change, business leaders recognize the pressing need for data-driven decision-making. Consistency in unified data governance is also essential, according to Seiner.

Data Governance 67

Data Governance Government Datasets Programming 67

Monte Carlo

MARCH 3, 2021

When it comes to managing your company’s data, security is high on your list of priorities. Today, I’m thrilled to share that Monte Carlo has achieved SOC 2 Type I certification , an industry-leading standard for the security, availability, and confidentiality that our organization adopted. What does this mean for you?

Metadata 52

Metadata Data Lake Business Intelligence Certification 52

Monte Carlo

MARCH 18, 2021

With more and more data being ingested and stored by your company than ever before and increased scrutiny around how this data is accessed and used, layering in any type of new third-party solution to your existing data infrastructure implicitly comes with its fair share of security concerns. Image courtesy of authors.

Architecture 52

Architecture Metadata Software Engineer Software Engineering 52

AltexSoft

NOVEMBER 10, 2021

In a nutshell, the lakehouse system leverages low-cost storage to keep large volumes of data in its raw formats just like data lakes. At the same time, it brings structure to data and empowers data management features similar to those in data warehouses by implementing the metadata layer on top of the store.

Architecture 52

Architecture Data Lake Data Warehouse Metadata 52

AltexSoft

AUGUST 29, 2023

Instead of relying on traditional hierarchical structures and predefined schemas, as in the case of data warehouses, a data lake utilizes a flat architecture. This structure is made efficient by data engineering practices that include object storage. Watch our video explaining how data engineering works.

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

Cloudera

MARCH 9, 2021

In this blog, we’ll highlight the key CDP aspects that provide data governance and lineage and show how they can be extended to incorporate metadata for non-CDP systems from across the enterprise. The SDX layer of CDP leverages the full spectrum of Atlas to automatically track and control all data assets.

Data Governance 95

Data Governance Government Metadata Datasets 95

Knowledge Hut

SEPTEMBER 29, 2023

Moreover, Azure's growing popularity in the market makes it a strategic choice for those looking to enter the world of data engineering. Azure currently stands as the second-largest vendor of cloud services, solidifying its position as a platform worth investing in for aspiring Azure data engineers.

Certification 52

Certification Data Engineering Data Engineer Engineering 52

Knowledge Hut

APRIL 20, 2023

The following are some of the key reasons why data governance is important: Ensuring data accuracy and consistency: Data governance helps to ensure that data is accurate, consistent, and trustworthy. This helps organisations make informed decisions based on reliable data.

Data Governance 59

Data Governance Government Data Cleanse Data Security 59

AltexSoft

MARCH 2, 2023

Data integrity is about maintaining the quality of data as it is stored, converted, transmitted, and displayed. Learn more about data integrity in our dedicated article. However, decentralized models may result in inconsistent and duplicate master data. Its goal is to define and control all data governance initiatives.

Data Governance 52

Data Governance Government Healthcare Programming 52