Data Engineering Podcast

APRIL 21, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Can you describe how you have architected the Shortwave platform? Go to dataengineeringpodcast.com/dagster today to get started.

Data Lake 173

Data Lake High Quality Data Data Pipeline Machine Learning 173

Data Engineering Podcast

APRIL 7, 2024

In this episode Artyom Keydunov, creator of Cube, discusses the evolution and applications of the semantic layer as a component of your data platform, and how Cube provides speed and cost optimization for your data consumers. Dagster offers a new approach to building and running data platforms and data pipelines.

Data Lake 162

Data Lake High Quality Data BI Data Workflow 162

Data Engineering Podcast

APRIL 14, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is brought to you by Datafold – a testing automation platform for data engineers that prevents data quality issues from entering every part of your data workflow, from migration to dbt deployment.

Non-relational Database 147

Non-relational Database Relational Database Database Designing 147

Data Engineering Podcast

FEBRUARY 18, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. What are the notable advancements in recent months/years that make them a more viable platform choice? Your first 30 days are free!

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Data Engineering Podcast

MARCH 24, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. What are the notable enhancements beyond the Dagster Core project that this updated platform provides? Your first 30 days are free!

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

Data Engineering Podcast

MARCH 31, 2024

Trusted by teams of all sizes, including Comcast and Doordash, Starburst is a data lake analytics platform that delivers the adaptability and flexibility a lakehouse ecosystem promises. Dagster offers a new approach to building and running data platforms and data pipelines. Go to dataengineeringpodcast.com/dagster today to get started.

Project 130

Project Data Lake High Quality Data Data Workflow 130

Data Engineering Podcast

MARCH 3, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. What are some of the useful clarifying/scoping questions to address when deciding the path to deployment for different definitions of "AI"?

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Data Engineering Podcast

FEBRUARY 11, 2024

There are also numerous technical considerations to be made, particularly if the producer and consumer of the data aren't using the same platforms. Trusted by teams of all sizes, including Comcast and Doordash, Starburst is a data lake analytics platform that delivers the adaptability and flexibility a lakehouse ecosystem promises.

Data Lake 147

Data Lake High Quality Data Government Data Pipeline 147

Data Engineering Podcast

FEBRUARY 4, 2024

Trusted by teams of all sizes, including Comcast and Doordash, Starburst is a data lake analytics platform that delivers the adaptability and flexibility a lakehouse ecosystem promises. Dagster offers a new approach to building and running data platforms and data pipelines. Go to dataengineeringpodcast.com/dagster today to get started.

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

Data Engineering Podcast

MARCH 10, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Go to dataengineeringpodcast.com/dagster today to get started. Your first 30 days are free! Data lakes are notoriously complex.

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

KDnuggets

APRIL 25, 2024

List of top MLOPs platforms that will help you with integration, training, tracking, deployment, monitoring, CI/CD, and optimizing the infrastructure.

91

91

Data Engineering Podcast

FEBRUARY 25, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. What is the engineering work that is involved in building a cohesive platform from those components? Your first 30 days are free!

Database 162

Database Technology Data Lake High Quality Data 162

Snowflake

JANUARY 26, 2024

Snowflake’s planned expansion of EMEA deployments seeks to meet increasing regional customer demand for Snowflake’s Data Cloud, and enable local enterprises to maintain proximity to their data while enabling compliance with regional data regulations. Next fiscal year (Feb.

Google Cloud 84

Google Cloud AWS Government Cloud 84

Christophe Blefari

MARCH 15, 2024

Databricks invests in Mistral AI — Mistral successfully positioned as the main OpenAI rival by being integrated in all major data platforms (Azure, Snowflake previously). Starting today it will be supported by 3 majors platforms: Kaggle, HuggingFace and OpenML. This is Croissant. Give a lot of insights on the market.

Metadata 272

Metadata Datasets Data Data Warehouse 272

Data Engineering Podcast

OCTOBER 15, 2023

Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Can you describe what Decodable is and the story behind it?

Process 182

Process Building SQL BI 182

Data Engineering Podcast

MARCH 17, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Go to dataengineeringpodcast.com/dagster today to get started. Your first 30 days are free! Data lakes are notoriously complex.

Database 147

Database Data Lake High Quality Data Data Workflow 147

The Pragmatic Engineer

JUNE 1, 2023

At companies with dedicated platform teams, those teams take exactly this kind of load off other teams building greenfield projects. We covered how Stack Overflow learned this the hard way, a few months back. Serving a web page. This is the point where the app reached the “Hello world!” ” stage Improve deploys.

Education 363

Education Project PostgreSQL Software Engineer 363

Data Engineering Podcast

APRIL 23, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. How have the focus and goals of the platform and company evolved over the past 2 years? Can you describe what Meroxa is and the story behind it?

Data Lake 277

Data Lake Kafka Machine Learning Data Warehouse 277

Data Engineering Podcast

OCTOBER 22, 2023

This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Attend the dev and ML talks at NODES 2023, a free online conference on October 26 featuring some of the brightest minds in tech. Datafold : ![Datafold]([link]

BI 162

BI SQL Machine Learning Programming Language 162

Confessions of a Data Guy

SEPTEMBER 9, 2023

In the vast world of data, it’s not just about gathering and analyzing information anymore; it’s also about ensuring that data pipelines, processes, and platforms run seamlessly and efficiently.

Data Engineering 162

Data Engineering Data Engineer Engineering Data Pipeline 162

Knowledge Hut

APRIL 25, 2024

Using Terraform in DevOps, you can easily automate and manage your infrastructure and the services running on your organizational platform using declarative programming. Leverage Terraform in DevOps as a single method for all clouds, and streamline deployments effortlessly. Why is this crucial?

Amazon Web Services 52

Amazon Web Services Cloud AWS Programming Language 52

Snowflake

APRIL 8, 2024

Capable of a headless deployment into Snowpark Container Services, BigGeo can be used to speed up queries of data stored in Snowflake, gather those insights into a dashboard, visualize them on a map, and more. Stellar Stellar is designed to make generative AI easy for Snowflake customers.

Pipeline-centric 119

Pipeline-centric Food Healthcare Unstructured Data 119

Knowledge Hut

APRIL 26, 2024

It is one of the safest platforms for cloud service. Google Cloud Platform Next on the list is the Google Cloud Platform (GCP). Google Cloud Platform is a global leader in AI, machine learning, and data analytics. Alibaba Cloud Alibaba Cloud is known as the fourth largest cloud provider platform in the world.

Cloud Computing 52

Cloud Computing Amazon Web Services Cloud Google Cloud 52

Cloudera

FEBRUARY 1, 2024

Implementing the right platform is half the battle won — so congratulations on your choosing Cloudera’s industry-leading hybrid data platform for building your data solutions on a foundation of trusted data. Status indicators at the component level that illustrate the state of the platform: healthy, warning, and critical.

Cloud 82

Cloud Architecture Utilities Designing 82

Knowledge Hut

MARCH 7, 2024

It is one of the safest platforms for cloud service. Google Cloud Platform Next on the list is the Google Cloud Platform (GCP). Google Cloud Platform is a global leader in AI , machine learning, and data analytics. Alibaba Cloud Alibaba Cloud is known as the fourth largest cloud provider platform in the world.

Cloud Computing 93

Cloud Computing Amazon Web Services Cloud Google Cloud 93

Tweag

APRIL 17, 2024

Internal Developer Platforms (IDPs) IDPs were a hot topic at Kubecon. An IDP is a platform - usually a web interface in practice - which glues several developer tools and services together and acts as the central entry point for most developer workflows. We already emphasized this emerging new abstraction in last year’s post.

Cloud 76

Cloud Consulting Portfolio Programming 76

Edureka

APRIL 17, 2024

WebServer using Docker Docker, one of the extensively used containerization platforms, allows you t o envelop dependencies with an application and create a container which is portable. This practice assignment is for Docker deployment of the web server. In this project, you’ll develop a Java app with Gradle as a build.

Project 52

Project Java AWS Coding 52

Snowflake

NOVEMBER 1, 2023

Since then, we’ve significantly opened up the ways Snowflake’s platform, including its elastic compute engine can be used to accelerate the path from AI/ML development to production. Snowpark ML unifies data pre-processing, feature engineering, model training and integrated deployment into a single, easy-to-use Python library.

Building 98

Building Python SQL Programming Language 98

LinkedIn Engineering

OCTOBER 19, 2023

This robust framework empowers near real-time data processing for critical services and platforms, ranging from machine learning and notifications to anti-abuse AI modeling. With over 950 million members, ensuring that our platform is running smoothly is critical to connecting members to opportunities worldwide.

Process 119

Process Lambda Architecture Kafka Machine Learning 119

Analytics Vidhya

FEBRUARY 22, 2023

Introduction Suppose you are appearing in an interview for the manager or senior role.

Management 228

Management Google Cloud Cloud Cloud Computing 228

Snowflake

APRIL 4, 2024

Today, we are excited to announce that Snowflake Ventures is investing in our partner, Coalesce, which offers an intuitive, low-code transformation platform for developing and managing data pipelines. The Coalesce platform is uniquely built for Snowflake.

Cloud 101

Cloud Management Data Pipeline SQL 101

KDnuggets

OCTOBER 1, 2023

Explore the essentials of Google Cloud Platform for data science and ML, from account setup to model deployment, with hands-on project examples.

Google Cloud 136

Google Cloud Cloud Data Science Project 136

Pinterest Engineering

OCTOBER 17, 2023

Charles Wu | Software Engineer; Isabel Tallam | Software Engineer; Kapil Bajaj | Engineering Manager Overview In this blog, we present a pragmatic way of integrating analytics, written in Python, with our distributed anomaly detection platform, written in Java. Background Warden is the distributed anomaly detection platform at Pinterest.

Algorithm 86

Algorithm Java Python Software Engineer 86

Pinterest Engineering

OCTOBER 31, 2023

Harry Zhang, Jiajun Wang, Yi Li, Shunyao Li, Ming Zong, Haniel Martino, Cathy Lu, Quentin Miao, Hao Jiang, James Wen, David Westbrook | Cloud Runtime Team Image Source: [link] Overview Modern compute platforms are foundational to accelerating innovation and running applications more efficiently.

Architecture 95

Architecture Pipeline-centric Accessible Accessibility 95

Monte Carlo

MARCH 28, 2024

DevOps focuses on service uptime, continuous integration, continuous deployment without breaking, container orchestration, security, and more. Nowadays, DevOps Engineer is occasionally called a Platform Engineer. Upon successful development of code, the same is prepared for deployment into various environments.

Data Pipeline 52

Data Pipeline Coding Machine Learning Data Warehouse 52

Snowflake

DECEMBER 20, 2023

Within the scope of gen AI, this new Snowpark runtime empowers developers to efficiently and securely deploy containers to do things like the following and more: LLM fine-tuning Open-source vector database deployment Distributed embedding processing Voice to text transcription Why did Snowflake build a container service?

Scala 117

Scala Government Java AWS 117