Data Engineering Podcast

SEPTEMBER 3, 2023

Summary Cloud data warehouses and the introduction of the ELT paradigm has led to the creation of multiple options for flexible data integration, with a roughly equal distribution of commercial and open source options. What are the most interesting, unexpected, or challenging lessons that you have learned while working on dlt?

Data Integration 162

Data Integration BI SQL Python 162

Data Engineering Podcast

DECEMBER 3, 2023

Andrei Tserakhau has dedicated his careeer to this problem, and in this episode he shares the lessons that he has learned and the work he is doing on his most recent data transfer system at DoubleCloud. Datafold has recently launched a 3-in-1 product experience to support accelerated data migrations. With Materialize, you can!

Systems 130

Systems Designing Data Lake SQL 130

Data Engineering Podcast

OCTOBER 15, 2023

If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold You shouldn't have to throw away the database to build with fast-changing data. With Materialize, you can!

Process 182

Process Building SQL BI 182

Data Engineering Podcast

OCTOBER 22, 2023

You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. The Machine Learning Podcast helps you go from idea to production with machine learning.

BI 162

BI SQL Machine Learning Programming Language 162

Snowflake

APRIL 22, 2024

As business needs demanded more frequent data sharing across these units, the costs associated with transferring large data sets across these cloud regions also began to rise. Recognizing the inefficiencies and escalating costs of operating in separate cloud regions, Magnite decided to consolidate its operations in AWS US East.

AWS 69

AWS Cloud Storage Cloud Technology 69

Data Engineering Podcast

AUGUST 27, 2023

In the era of the cloud most developers rely on hosted services to manage their databases, but what if you are a cloud service? If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. With Materialize, you can!

Database 130

Database Building PostgreSQL BI 130

Data Engineering Podcast

OCTOBER 8, 2023

If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold As more people start using AI for projects, two things are clear: It’s a rapidly advancing field, but it’s tough to navigate.

Insurance 162

Insurance BI SQL Machine Learning 162

Data Engineering Podcast

OCTOBER 1, 2023

If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold You shouldn't have to throw away the database to build with fast-changing data. With Materialize, you can!

Building 162

Building BI SQL Machine Learning 162

The Pragmatic Engineer

MAY 11, 2023

Prometheus is part of the Cloud Native Foundation, membership of which indicates that it’s safe to build on top of Prometheus, as it’s actively maintained and will continue to be. We cover approaches like double-writing in the issue Migrations done well. What can we learn from Coinbase’s cost reduction exercise?

AWS 304

AWS Software Engineer Software Engineering Google Cloud 304

Data Engineering Podcast

NOVEMBER 5, 2023

You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. The Machine Learning Podcast helps you go from idea to production with machine learning.

PostgreSQL 147

PostgreSQL Data Lake High Quality Data SQL 147

Data Engineering Podcast

SEPTEMBER 24, 2023

Summary The rapid growth of machine learning, especially large language models, have led to a commensurate growth in the need to store and compare vectors. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. With Materialize, you can!

BI 147

BI SQL Machine Learning Python 147

Data Engineering Podcast

NOVEMBER 12, 2023

If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold Data lakes are notoriously complex. You shouldn't have to throw away the database to build with fast-changing data.

Software Engineer 147

Software Engineer Software Engineering Engineering Data Lake 147

The Pragmatic Engineer

JUNE 13, 2023

Many subscribers expense this newsletter to their learning and development budget. In a previous two-part series , we dived into Uber’s multi-year project to move onto the cloud , away from operating its own data centers. The hardware inside Agoda’s private cloud. Agoda’s cloud strategy and usage.

Cloud 192

Cloud Utilities Database BI 192

Snowflake

NOVEMBER 30, 2023

At TCS , we help companies shift their enterprise data warehouse (EDW) platforms to the cloud as well as offering IT services. We’re extremely familiar with just how tricky a cloud migration can be, especially when it involves moving historical business data. How many tables and views will be migrated, and how much raw data?

Data Warehouse 98

Data Warehouse Banking Data Cloud 98

Data Engineering Podcast

AUGUST 20, 2023

If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold You shouldn't have to throw away the database to build with fast-changing data. With Materialize, you can!

Education 130

Education BI SQL Database 130

Knowledge Hut

DECEMBER 26, 2023

Cloud computing enables an organization to use on-demand IT resources and scale up or down as per their requirements. This makes the judicious and informed use of Cloud Computing Tools really important. What Are the Types of Cloud Computing Tools Available? The main types of Cloud services which are available are: 1.

Cloud Computing 98

Cloud Computing Cloud Amazon Web Services AWS 98

The Pragmatic Engineer

AUGUST 10, 2023

Before we start: plenty of people who subscribe to the newsletter utilize a learning and development budget their company provides. A week and a half ago, machine learning engineer Ayhan Fuat Çelik ​​wrote an interesting analysis titled The fall of Stack Overflow. To get full issues twice a week, subscribe here.

Retail 171

Retail Utilities Software Engineer Software Engineering 171

The Pragmatic Engineer

JUNE 22, 2023

” So, Google Domains and Google Cloud Domains are shutting down, possibly in about 9 months’ time. Squarespace has bought the 10M domain registrations and customer accounts, will migrate them over with the help of the current Google Domains team, then Google Domains will be disbanded. The impact on Google Cloud.

Google Cloud 234

Google Cloud Media Cloud Engineering 234

Knowledge Hut

MARCH 5, 2024

Examining the latest advancements in cloud computing, it is evident that significant transformations have occurred. An extensive suite of services is provided by Amazon Web Services (AWS), a platform for cloud computing that helps companies of all sizes manage their operations effectively. What is AWS?

AWS 52

AWS Amazon Web Services Cloud Computing PostgreSQL 52

Data Engineering Weekly

MAY 12, 2024

Gloss Genius: How We Migrated From dbt Cloud and Scaled Our Data Development Gloss Genius describes its migration journey from dbt cloud to Airflow + custom Github actions. The post highlights how Gloss Genuis established feature parity with SlimCI and enhanced post-migration with open-source tooling.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Lake 52

The Modern Data Company

FEBRUARY 28, 2023

Is Your Head Too High up in the Cloud? There is no doubt that the cloud is here to stay and that it will be a part of every company’s future data and analytics strategy. While there are many cloud success stories, there are also a lot of stories of frustration, missed deadlines, cost shocks, and lack of anticipated results.

Cloud 98

Cloud Hadoop Technology Coding 98

Snowflake

DECEMBER 20, 2023

Migrating to Snowflake helped ServiceNow democratize data, accelerate innovation and increase efficiency. Careful planning and execution were essential to the team’s migration plan, which included: Clear goal setting: Speed of implementation wasn’t the only indicator of success. We decided to go with Snowflake,” says Siva Prasad, Sr.

IT 67

IT Data Government Cloud 67

Knowledge Hut

MARCH 28, 2024

As a result, organizations are looking to capitalize on cloud-based data solutions. This is the perfect time to search for Azure data engineer jobs for freshers and to start learning Azure for Dat a Engineers out there. Role Level: Intermediate Responsibilities Develop machine learning pipelines using Azure Machine Learning service.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

Knowledge Hut

MARCH 18, 2024

Businesses are turning to cloud computing in the digital age to gain scalability, safe storage in one location, and simplified processes. That is to say, rather than erecting expensive data centre infrastructure, firms are selecting cloud service providers. Learning Curve 3. Hybrid Cloud Capabilities 4. What is Azure?

Portfolio 52

Portfolio Cloud Computing Machine Learning Cloud 52

Snowflake

JANUARY 24, 2024

To learn more about the impact AI and other developments are expected to have on the public sector this year, we sat down with our Snowflake industry and technology experts. This modernization strategy will center around investing in cloud-based data platforms to integrate and scale agencies’ data/technology stacks.

Government 96

Government Data Management Cloud Data 96

Snowflake

FEBRUARY 2, 2023

During a cloud migration to Snowflake’s Data Cloud, businesses often struggle to know what data they have on premises, what they should migrate, and in what order. And because of this, many organizations fall into a “lift and shift” approach, where everything is simply copied over—as it messily stands—to the cloud.

Cloud 86

Cloud Data Data Governance Government 86

Knowledge Hut

NOVEMBER 7, 2023

The applications of cloud computing in businesses of all sizes, types, and industries for a wide range of applications, including data backup, email, disaster recovery, virtual desktops big data analytics, software development and testing, and customer-facing web apps. What Is Cloud Computing?

Cloud Computing 98

Cloud Computing Cloud Amazon Web Services Entertainment 98

Snowflake

SEPTEMBER 14, 2023

It should empower people with varying skill sets to effectively use data and harness the power of AI and machine learning (ML). The Snowflake Data Cloud gives you the flexibility to build a modern architecture of choice to unlock value from your data. Snowflake was built from the ground up in the cloud.

Data Architecture 95

Data Architecture Architecture Hadoop Data Warehouse 95

Ascend.io

MAY 7, 2024

is excited to partner with Wizeline, bringing together our advanced data automation platform with their AI and machine learning expertise,” said Sean Knapp, CEO of Ascend.io. Data Transformation and Legacy Migration Services The combined expertise of Wizeline and Ascend.io Learn more at Ascend.io “Ascend.io

Pipeline-centric 52

Pipeline-centric Data Cleanse Data Security Data Pipeline 52

Cloudera

SEPTEMBER 26, 2023

These solutions include on-premises and public cloud offerings with the data management, security and governance needed to build an enterprise data and analytic big data solution. You can now elevate your data potential and activate AI’s capabilities through the synergic integration between IBM watsonx and Cloudera.

Kafka 92

Kafka Technology IT Government 92

Knowledge Hut

MARCH 19, 2024

With its extensive range of cloud services, Amazon Web Services (AWS) has completely changed the way businesses run. So, let's discuss the AWS cloud migration case study and its importance in getting a better understanding of the topic in detail. What are AWS Case Studies, and Why are They Important?

AWS 52

AWS Amazon Web Services Hospitality Cloud Computing 52

Knowledge Hut

NOVEMBER 16, 2023

Azure cloud is the future of digital transformation. The importance of cloud security has surged, particularly in the aftermath of the COVID-19 pandemic. The Azure cloud engineer roles and responsibilities administer the different aspects of Azure to ensure optimal data security. However, my tasks do not end there.

Cloud 52

Cloud Engineering Education Cloud Computing 52

Striim

APRIL 30, 2024

Challenges Kramp Was Facing Kramp began a significant transformation of its data management systems with a goal to shift their existing data warehouse to a cloud-based infrastructure on the Google Cloud Platform. It strengthened business operations, empowering sophisticated machine learning projects and immediate data analysis.

Google Cloud 52

Google Cloud High Quality Data Business Intelligence Data Warehouse 52

Snowflake

AUGUST 30, 2023

In this blog post, we will delve into six such capabilities – comprehensive cross-cloud replication, zero copy database and schema clone, collation support, stored procedures, multi-table transactions, and transparent online upgrade – that every enterprise must consider while choosing their data platforms.

Scala 88

Scala Government Database Cloud 88

Knowledge Hut

JANUARY 3, 2024

Cloud Computing is a revolutionary technology transforming enterprises and accelerating digital transformation. It is said that more businesses around the world will increase their investments in Cloud technology solutions, and there has been a sudden spike in Cloud deployments ever since the COVID-19 pandemic started.

Cloud Computing 52

Cloud Computing Cloud Amazon Web Services Government 52

Monte Carlo

OCTOBER 2, 2023

Fast forward to today: the Abacus data platform is 100% in the cloud, leverages Databricks for business critical machine learning applications, and supports a growing business that’s doubled its headcount and tripled its revenue since 2018. And up until the last few years, that all happened within an on-premises architecture.

Pharmaceutical 72

Pharmaceutical Healthcare Machine Learning Architecture 72

RandomTrees

FEBRUARY 22, 2024

How to Migrate Your Business to the Cloud Moving to the google cloud workload is one of the most impressive things your business can utilize to build your adaptability, flexibility, and productivity. Migrating your business to the cloud implies a ton of planning and attention.

Google Cloud 52

Google Cloud Cloud Utilities Machine Learning 52