Snowflake

NOVEMBER 2, 2023

Over the years, the technology landscape for data management has given rise to various architecture patterns, each thoughtfully designed to cater to specific use cases and requirements. These patterns include both centralized storage patterns like data warehouse , data lake and data lakehouse , and distributed patterns such as data mesh.

Data Lake 89

Data Lake Data Warehouse Cloud Unstructured Data 89

Christophe Blefari

AUGUST 4, 2023

Summer Edition ( credits ) This is the first article of the Data News Summer Edition: how to build a data platform. The modern data stack has been criticised a lot, a few are saying it's dead other are saying we are in the post-modern era. In this article we will try to design the fancy data stack for a batch usage.

Google Cloud 225

Google Cloud MongoDB NoSQL Data 225

Knowledge Hut

NOVEMBER 16, 2023

From nebulous beginnings, the cloud has grown into a platform that has gained universal acceptance and is transforming businesses across industries. Companies that have adopted cloud technology have seen significant payoffs, with cloud-based tools redefining their data storage, data sharing, marketing and project management capabilities.

Cloud Computing 98

Cloud Computing Cloud Cloud Storage AWS 98

DoorDash Engineering

NOVEMBER 29, 2022

As DoorDash’s business grows, engineers strive for a better network infrastructure to ensure more third-party services could be integrated into our system while keeping data securely transmitted. The data transfer between DoorDash and payment processors needs to be encrypted to protect customers’ privacy and sensitive data.

Cloud 97

Cloud AWS Amazon Web Services Data 97

Knowledge Hut

JANUARY 12, 2024

What comes to your mind when you hear the term 'Cloud'? Well, in a technologically advanced world, Cloud refers to a place where you can store and manage data on a device. Personally, I find it fascinating how saying, "I can handle the Cloud," has become a ticket to professional opportunities. What is Cloud Computing?

Cloud Computing 98

Cloud Computing Cloud Computer Science Programming Language 98

Confluent

DECEMBER 6, 2021

At Confluent, we’re serious about security, and we’re focused on simplifying security visibility across our cloud and on-premises solution. This blog demonstrates how to monitor Confluent Cloud authorization events using […].

Cloud 113

Cloud Data 113

Christophe Blefari

MAY 17, 2024

The sun is out, the days are getting longer and Data News is still here. Next week marks 3 years of this newsletter/blog (yay 🎉 ). Let me introduce an application of this on the Data Council 2024 80 videos. After all, companies need to make decisions, and these decisions need to be informed by data.

Food 130

Food Data BI Engineering 130

Cloudera

OCTOBER 30, 2023

Cloud transformation is ranked as the cornerstone of innovation and digitalization. The legacy IT infrastructure to run the business operations — mainly data centers — has a deadline to shift to cloud-based services. However, the reality is that the “move to cloud” is a turbulent flight for many of them. Why FinOps?

Cloud 75

Cloud Finance Cloud Computing Government 75

Striim

FEBRUARY 2, 2024

Introducing Striim Cloud for Application Integration: A fully managed, simple, and scalable SaaS service for application connectors. With this new application integration service, users can stream real-time CRM, ERP, Billing, and Payment data from their cloud applications to data warehouses in minutes with zero coding.

Cloud 52

Cloud Google Cloud Data Integration Data Pipeline 52

Scott Logic

NOVEMBER 23, 2023

This instalment of the Conscientious Computing series dives into a far more practical example of a consideration that many aren’t making consciously: The cloud region being used. This in turn means that the cleanliness of the electricity differs with the data centre you choose.

Cloud 52

Cloud Transportation Finance AWS 52

Scott Logic

MAY 7, 2024

Businesses today want to keep an eye on their carbon emissions and do their bit to help the climate crisis and so they need to understand and reduce all their emissions including those from cloud computing. This blog explains how we got around it. This blog is one in a series from an internal project undertaken here at Scott Logic.

Cloud 52

Cloud Java AWS Coding 52

Cloudera

OCTOBER 1, 2020

Since 2013 the UK Government’s flagship ‘Cloud First’ policy has been at the forefront of enabling departments to shed their legacy IT architecture in order to meaningfully embrace digital transformation. Combined they have the potential to create something of a data storm. . The first is cloud concentration risk.

Government 125

Government Cloud Data Storage Architecture 125

Snowflake

OCTOBER 17, 2023

See how ctrl+s provides in-depth insights into supply chain sustainability, while protecting sensitive customer information—all through Snowflake’s powerful, scalable Data Cloud. One is a comprehensive and advanced data set of emissions factors, made up of several billion rows of data, called matter+s.

Cloud 82

Cloud Consulting Transportation Manufacturing 82

Scott Logic

OCTOBER 18, 2023

Introduction In my previous blog post I discussed how migrating to the Cloud could help your organisation reach its Net Zero goals. I discussed how shifting your workloads away from on-premises data centres can reduce emissions by allowing you to leverage the expertise of cloud providers and their greater efficiency of scale.

Cloud 87

Cloud AWS BI Accessible 87

Christophe Blefari

APRIL 5, 2024

Lost between ideas ( credits ) Hey, new Data News edition. MDS Fest is a free virtual 5 days conference about Modern Data Stack topics, a lot of awesome speakers, there are a few talks I can't wait to watch. It shows a few RAGs, Agents and parsers on documents to retrieve the data you need. But here we are.

SQL 130

SQL Data Python Datasets 130

Towards Data Science

JANUARY 26, 2023

In this post we consider the case in which our data application requires access to one or more large files that reside in cloud object storage. This continues a series of posts on the topic of efficient ingestion of data from the cloud (e.g., here , here , and here ).

Cloud Storage 75

Cloud Storage Big Data Cloud AWS 75

Data Science Blog: Data Engineering

SEPTEMBER 19, 2023

In the contemporary age of Big Data, Data Warehouse Systems and Data Science Analytics Infrastructures have become an essential component for organizations to store, analyze, and make data-driven decisions. So why using IaC for Cloud Data Infrastructures?

Data Warehouse 52

Data Warehouse Coding Systems Cloud 52

Christophe Blefari

MARCH 8, 2024

Today I want to introduce a new feature in the Data News galaxy. I don't talk much about my freelance life in Data News because sometimes I think that's not the contract we have together. Data News promise is to give you, every week, the links I've hand-picked with my spicy opinion about them.

Data 130

Data Algorithm Database Education 130

Cloudera

JANUARY 28, 2022

As I meet with our customers, there are always a range of discussions regarding the use of the cloud for financial services data and analytics. Customers vary widely on the topic of public cloud – what data sources, what use cases are right for public cloud deployments – beyond sandbox, experimentation efforts.

Cloud 111

Cloud Banking Data Governance Government 111

The Modern Data Company

FEBRUARY 28, 2023

Is Your Head Too High up in the Cloud? There is no doubt that the cloud is here to stay and that it will be a part of every company’s future data and analytics strategy. While there are many cloud success stories, there are also a lot of stories of frustration, missed deadlines, cost shocks, and lack of anticipated results.

Cloud 98

Cloud Hadoop Technology Coding 98

Data Engineering Podcast

MARCH 10, 2024

Summary Data lakehouse architectures are gaining popularity due to the flexibility and cost effectiveness that they offer. The link that bridges the gap between data lake and warehouse capabilities is the catalog. Data lakes are notoriously complex. Join us at the top event for the global data community, Data Council Austin.

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

DoorDash Engineering

JANUARY 16, 2024

As the number of microservices and back-ends has grown, however, DoorDash has observed an uptick in cross-availability zone (AZ) data transfer costs. These data transfer costs — incurred on both send and receive — allow DoorDash to provide its end users a highly available service that can withstand degradations of one or more AZs.

Bytes 84

Bytes Cloud Management PostgreSQL 84

Cloudera

AUGUST 26, 2021

Ozone natively provides Amazon S3 and Hadoop Filesystem compatible endpoints in addition to its own native object store API endpoint and is designed to work seamlessly with enterprise scale data warehousing, machine learning and streaming workloads. Learn more about the impacts of global data sharing in this blog, The Ethics of Data Exchange.

Data Science 109

Data Science Cloud Hadoop Metadata 109

Rockset

MARCH 18, 2024

That’s why we’re thrilled that Confluent Cloud is making it easier to use Flink, providing efficient and performant stream processing while saving engineers from complex infrastructure management. LLMs like ChatGPT are trained on vast amounts of text data available up to a cutoff date. What is RAG?

Cloud 64

Cloud Building Metadata Kafka 64

Cloudera

FEBRUARY 17, 2022

CDP Private Cloud Base is an on-premises version of Cloudera Data Platform (CDP). This new product combines the best of Cloudera Enterprise Data Hub and Hortonworks Data Platform Enterprise along with new features and enhancements across the stack. Zookeeper data. HDFS Master Node data directories.

Cloud 96

Cloud Data Professional Services Metadata 96

Cloudera

MARCH 29, 2022

State and local governments generate and store enormous amounts of data essential to their ability to deliver citizen services. But how can they capitalize on all of their data to become engines of growth and innovation, empowering and enhancing their ability to provide services and better serve their communities?

Government 93

Government Cloud Cloud Computing Data Science 93

Cloudera

FEBRUARY 23, 2022

Their processes are ‘data driven’, their networks are trending towards automation, and AI systems are powering customer engagement in store , online and at home. This is all based on an enterprise IT architecture that is integrated, data rich, and agile. How do we combine the challenges of network and IT clouds?

Telecommunication 90

Telecommunication Cloud Media Government 90

Christophe Blefari

JUNE 24, 2023

( credits ) Hey, this is the Data News. The VCs’ actual use case for AI is treating workers badly" Too perfect to be a real picture ( credits ) Fast News ⚡️ MotherDuck announcing DuckDB in the cloud — First, context. So this week MotherDuck finally announced their cloud offering.

PostgreSQL 130

PostgreSQL Data Data Engineering Data Engineer 130

Data Engineering Weekly

MAY 12, 2024

This year, we added some additional features to bring the data community together. Gloss Genius: How We Migrated From dbt Cloud and Scaled Our Data Development Gloss Genius describes its migration journey from dbt cloud to Airflow + custom Github actions. At scale, it becomes impossible to enrich the data assets manually.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Lake 52

Cloudyard

MARCH 4, 2023

Read Time: 5 Minute, 16 Second As we know Snowflake has introduced latest badge “Data Cloud Deployment Framework” which helps to understand knowledge in designing, deploying, and managing the Snowflake landscape. Respective Cloud would consume/Store the data in bucket or containers.

Architecture 52

Architecture Cloud Metadata Data Ingestion 52

Cloudera

MAY 9, 2024

The Gartner Data and Analytics Summit in London is quickly approaching on May 13 th to 15 th , and the Cloudera team is ready to hit the show floor! With the rapid pace of AI innovations, today’s leaders must connect, share, and learn from each other on how to best leverage AI technology to effectively analyze data to our advantage.

Banking 94

Banking Data Storage Data Analytics Cloud 94

Hevo

FEBRUARY 2, 2024

Cloud Migration is one of the most crucial areas of the data science ecosystem. As data is growing more rapidly than ever, analyzing the data is becoming even more important for business profitability. Local storage cannot store huge volumes of data, so a transition from on-premises to cloud storage is needed.

Cloud 40

Cloud Process Cloud Storage Data Science 40

Cloudera

JANUARY 15, 2021

Cloud data warehouses allow users to run analytic workloads with greater agility, better isolation and scale, and lower administrative overhead than ever before. The results demonstrate superior price performance of Cloudera Data Warehouse on the full set of 99 queries from the TPC-DS benchmark. Introduction.

Data Warehouse 92

Data Warehouse Cloud Consulting SQL 92

phData: Data Engineering

DECEMBER 20, 2022

This blog was co-written with our partners at Anomalo. Congratulations—at last, you’ve finished your cloud data migration and it feels as if you just bought a brand new car! Hopefully, your cutting-edge cloud data warehouse comes with the latest and greatest safety features, too.

Cloud 52

Cloud Database-centric Data Data Warehouse 52

Cloudera

JULY 15, 2021

The release of Cloudera Data Platform (CDP) Private Cloud Base edition provides customers with a next generation hybrid cloud architecture. Private Cloud Base Overview. The storage layer for CDP Private Cloud, including object storage. Traditional data clusters for workloads not ready for cloud.

Architecture 97

Architecture Cloud Kafka Hadoop 97

Snowflake

SEPTEMBER 11, 2023

Data privacy regulations are sweeping across the globe, with some 71% of countries worldwide adopting data protection and privacy legislation. The European Union’s General Data Protection Regulation (GDPR) , one of the more well-known and far-reaching of these privacy regulations, went into effect on May 25, 2018.

Cloud 76

Cloud Electronics Government Data Governance 76

Knowledge Hut

DECEMBER 4, 2023

This is where cloud computing comes to the rescue. Cloud computing makes the services of a physical machine available to you as per your convenience, demand and budget, that too at the click of a button. A meticulous cloud computing certification will help you learn and use this technology effectively. What a tedious task!

Cloud Computing 52

Cloud Computing Cloud Google Cloud AWS 52