Knowledge Hut

MARCH 22, 2024

Manage data storage and build dashboards for reporting. Cloud DevOps Engineer As Cloud DevOps Engineers, you will design and implement AWS cloud solutions to enhance business operations through automation, continuous integration, and continuous deployment practices. Automate deployment processes and streamline workflows.

AWS 52

AWS Amazon Web Services Cloud Computing Google Cloud 52

Christophe Blefari

MARCH 1, 2023

dbt Core is an open-source framework that helps you organise data warehouse SQL transformation. dbt was born out of the analysis that more and more companies were switching from on-premise Hadoop data infrastructure to cloud data warehouses. This switch has been lead by modern data stack vision. Enter the ELT.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

Knowledge Hut

SEPTEMBER 21, 2023

Questions such as which is better and easier to learn—AWS, Azure, or GCP— are often asked by organization leaders before starting out on their cloud journey. But how do you really know which is better - AWS, GCP, or Azure? In this article, we will explore two leading platforms, Azure and GCP, and how they compare. What Is Azure?

Cloud 52

Cloud Google Cloud Cloud Computing Cloud Storage 52

Knowledge Hut

AUGUST 18, 2023

This program offers Google cloud certifications in various areas, such as cloud infrastructure , data analytics , machine learning, & application development. GCP Cloud Certification is made for professionals who want to enhance their careers in cloud computing & stay competitive in the job market.

Google Cloud 52

Google Cloud Certification Cloud Cloud Computing 52

Data Engineering Podcast

NOVEMBER 13, 2022

Summary CreditKarma builds data products that help consumers take advantage of their credit and financial capabilities. To make that possible they need a reliable data platform that empowers all of the organization’s stakeholders. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

MongoDB 100

MongoDB Scala MySQL Google Cloud 100

The Pragmatic Engineer

MAY 11, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. I wanted to find the truth, so I tracked down software engineers at the company. I can confirm this, having talked with both current and former software engineers at Coinbase who shared details of what happened. And I got my answer: Yes.

AWS 303

AWS Software Engineer Software Engineering Google Cloud 303

Christophe Blefari

JANUARY 20, 2024

We are deep in the winter, it's time for comfy Data News to read near the fire 🔥 This week, on Monday, I started my annual university lecture. In exchange we can wear the new Ray Ban Meta glasses with AI inside to give Meta more training data. So it's normal that they don't have any questions at first.

Data 130

Data Google Cloud Python Algorithm 130

Christophe Blefari

OCTOBER 20, 2023

Read my dbt multi-project guide 📺 On the content side I'll also present next week the Fancy Data Stack project at the Data Engineering And Machine Learning Summit 2023 organised by Seattle Data Guy. New Vertex AI Feature Store — GCP Vertex AI is the place to do "serverless" AI.

Generalist 130

Generalist Entertainment NoSQL Datasets 130

Christophe Blefari

FEBRUARY 3, 2023

Delivering the data news ( credits ) Hey you, it's already February. The article shows our Netflix art creators are using past data to create new artworks. ebay, Variable Hub a data access layer for risk decisioning — Looks like a feature store but for risk topics. Analytics Engineer- A Glorified BI Engineer?

BI 130

BI Google Cloud SQL Machine Learning 130

Knowledge Hut

JUNE 30, 2023

Unless you genuinely want to become a GCP Career Hero, you are not required to pursue all the GCP qualifications. For instance, obtaining a Google Cloud Professional DevOps Engineer certification is the best approach to learning about GCP for DevOps and demonstrating your expertise.

Google Cloud 52

Google Cloud Cloud Cloud Computing Certification 52

Data Engineering Podcast

NOVEMBER 6, 2022

Summary Despite the best efforts of data engineers, data is as messy as the real world. Entity resolution and fuzzy matching are powerful utilities for cleaning up data from disconnected sources, but it has typically required custom development and training machine learning models.

MongoDB 130

MongoDB Scala MySQL Data Lake 130

Snowflake

APRIL 8, 2024

The list of Top 10 semi-finalists is a perfect example: we have use cases for cybersecurity, gen AI, food safety, restaurant chain pricing, quantitative trading analytics, geospatial data, sales pipeline measurement, marketing tech and healthcare. Our sincere thanks go out to everyone who participated in this year’s competition.

Pipeline-centric 117

Pipeline-centric Food Healthcare Unstructured Data 117

Data Engineering Podcast

JUNE 12, 2022

Summary Unstructured data takes many forms in an organization. From a data engineering perspective that often means things like JSON files, audio or video recordings, images, etc. This episode is brought to you by Acryl Data, the company behind DataHub, the leading developer-friendly data catalog for the modern data stack.

Unstructured Data 130

Unstructured Data MongoDB Scala MySQL 130

Knowledge Hut

APRIL 25, 2023

The tremendous growth in data generation, then the rise in data engineer jobs - there’s no arguing the fact that the big data industry is at its best pace and you, as an aspiring data engineer, have a lot to learn and make out of it - including some tools! What are Data Engineering Tools?

Data Engineering 52

Data Engineering Data Engineer Engineering Google Cloud 52

ProjectPro

FEBRUARY 6, 2023

Google Cloud Pub/Sub is a global, cloud-based messaging framework that has become increasingly popular among data engineers over recent years. Data engineers often use Google Cloud Pub/Sub to design asynchronous workflows, publish event notifications, and stream data from several processes or devices.

Google Cloud 52

Google Cloud Cloud Cloud Storage Data Ingestion 52

Christophe Blefari

DECEMBER 31, 2023

Before moving on to the Data News, a bit of personal news, in December, I took part in the MotherDuck meetup in Berlin. End of January, on the 31st I'll speak at a Modern Data Stack conf in Paris, still about DuckDB, but this time in French. Enjoy this last 2023 Data News. We're going to get to know each other.

Data 100

Data Cloud Storage Datasets Python 100

Data Engineering Podcast

AUGUST 21, 2022

Summary Data has permeated every aspect of our lives and the products that we interact with. In this episode Shruti Bhat gives her view on the state of the ecosystem for real-time data and the work that she and her team at Rockset is doing to make it easier for engineers to build those experiences.

Lambda Architecture 100

Lambda Architecture MongoDB Scala MySQL 100

Data Engineering Podcast

JULY 17, 2022

Summary Data engineering is a large and growing subject, with new technologies, specializations, and "best practices" emerging at an accelerating pace. RudderStack helps you build a customer data platform on your warehouse or data lake. Data teams are increasingly under pressure to deliver.

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Team Data Science

NOVEMBER 4, 2020

Week 1: 10/9/20 - 10/16/20 In my quest to further improve my overall data science skills, I pulled the trigger on October 9th, 2020, and enrolled in a Data Engineering boot camp lead by Andreas Kretz. I have a background in Aerospace Engineering and have been in the industry for close to 15 years now.

Data Science 130

Data Science Machine Learning Programming Language Business Intelligence 130

Christophe Blefari

MAY 19, 2023

In 2021, I was doing Twitch lives twice a week, every Wednesday I was doing a data news round-up. This is what made me decide to send emails containing my round-up. 😱 If you only want to read Data News you can read my selection of talks from the Data Council. At the time, only 3 people received it. We achieved that.

Data 130

Data Data Engineering Data Engineer Hadoop 130

Data Engineering Podcast

SEPTEMBER 11, 2022

Summary Any business that wants to understand their operations and customers through data requires some form of pipeline. Building reliable data pipelines is a complex and costly undertaking with many layered requirements. Data stacks are becoming more and more complex. Sifflet also offers a 2-week free trial.

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Booking.com Engineering

DECEMBER 2, 2022

In this article, we want to illustrate our extensive use of the public cloud, specifically Google Cloud Platform (GCP). From data ingestion, data science, to our ad bidding[2], GCP is an accelerant in our development cycle, sometimes reducing time-to-market from months to weeks. What are PPC’s Challenges?

Systems 52

Systems Cloud MySQL Relational Database 52

Data Engineering Podcast

OCTOBER 23, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

Database 100

Database MySQL Cloud MongoDB 100

Data Engineering Podcast

OCTOBER 30, 2022

Summary One of the most impactful technologies for data analytics in recent years has been dbt. It’s hard to have a conversation about data engineering or analysis without mentioning it. Despite its widespread adoption there are still rough edges in its workflow that cause friction for data analysts.

Engineering 100

Engineering MongoDB Scala MySQL 100

Scott Logic

NOVEMBER 15, 2022

The system in question is a serverless data pipeline. These services are deployed on Google Cloud Platform (GCP). Individual parts of the system are deployed as Google Cloud Functions (GCFs), which are serverless functions triggered by some event, or as Kubernetes services through Google Kubernetes Engine (GKE).

Google Cloud 52

Google Cloud Cloud Systems Coding 52

Snowflake

MARCH 14, 2024

As a cohesive ERP solution, SAP is often one of the largest data resources in an organization, containing everything from financial and transactional data to master information about customers, vendors, materials, facilities, planning and even HR. What’s the challenge with unlocking SAP data?

IT 82

IT Data Ingestion Data AWS 82

Knowledge Hut

JULY 31, 2023

Who is a Google Cloud Engineer? Google Cloud Engineer is a capable individual who creates, develops, and maintains cloud-based solutions leveraging Google Cloud Platform technology. Google Cloud Engineers must thoroughly understand cloud computing concepts and technologies and have programming experience with Python, Java, and Go.

Google Cloud 52

Google Cloud Cloud Engineering Cloud Computing 52

Data Engineering Podcast

JULY 10, 2022

Summary Building and maintaining reliable data assets is the prime directive for data engineers. While it is easy to say, it is endlessly complex to implement, requiring data professionals to be experts in a wide range of disparate topics while designing and implementing complex topologies of information workflows.

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

ProjectPro

OCTOBER 6, 2021

With 67 zones, 140 edge locations, over 90 services, and 940163 organizations using GCP across 200 countries - GCP is slowly garnering the attention of cloud users in the market. Companies like Verizon, Yahoo, Facebook, SAP, LinkedIn, Newscorp, Marriott, Paypal are the big spenders on GCP products.

Google Cloud 52

Google Cloud Project Data Lake Healthcare 52

ProjectPro

JANUARY 19, 2022

Data Engineering is gradually becoming a popular career option for young enthusiasts. Explore this page further and learn everything about data engineers to find the answer. We will cover it all, from its definition, skills, responsibilities to the significance of data engineer in an institution.

Data Engineering 52

Data Engineering Data Engineer Engineering Amazon Web Services 52

Data Engineering Podcast

OCTOBER 2, 2022

Summary Data lineage is something that has grown from a convenient feature to a critical need as data systems have grown in scale, complexity, and centrality to business. Alvin is a platform that aims to provide a low effort solution for data lineage capabilities focused on simplifying the work of data engineers.

IT 100

IT Food PostgreSQL MongoDB 100

Data Engineering Podcast

AUGUST 13, 2022

Summary Data is useless if it isn’t being used, and you can’t use it if you don’t know where it is. Data catalogs were the first solution to this problem, but they are only helpful if you know what you are looking for. Data stacks are becoming more and more complex. Sifflet also offers a 2-week free trial.

Metadata 100

Metadata MongoDB Scala MySQL 100

Data Engineering Podcast

OCTOBER 16, 2022

Summary The "data lakehouse" architecture balances the scalability and flexibility of data lakes with the ease of use and transaction support of data warehouses. Enter Metaplane, the industry’s only self-serve data observability tool. Data teams are increasingly under pressure to deliver.

Data Lake 100

Data Lake Food MongoDB Scala 100

Data Engineering Podcast

AUGUST 28, 2022

Summary AirBnB pioneered a number of the organizational practices that have become the goal of modern data teams. Data stacks are becoming more and more complex. This brings infinite possibilities for data pipelines to break and a host of other issues, severely deteriorating the quality of the data and causing teams to lose trust.

Building 100

Building MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 25, 2022

Summary Data integration from source systems to their downstream destinations is the foundational step for any data product. With the increasing expecation for information to be instantly accessible, it drives the need for reliable change data capture. Data teams are increasingly under pressure to deliver.

Food 100

Food MongoDB Scala MySQL 100

ProjectPro

FEBRUARY 21, 2023

With so many data engineering certifications available , choosing the right one can be a daunting task. There are over 133K data engineer job openings in the US, but how will you stand out in such a crowded job market? The answer is- by earning professional data engineering certifications! AWS or Azure?

Certification 52

Certification Data Engineering Data Engineer Engineering 52

Data Engineering Podcast

SEPTEMBER 18, 2022

He discusses the challenges of making data collection purposeful and efficient without distracting employees from their primary duties and how business owners can use the provided analytics to support their staff in their duties. RudderStack helps you build a customer data platform on your warehouse or data lake.

Hospitality 100

Hospitality Food MongoDB Scala 100