Towards Data Science

NOVEMBER 4, 2023

Indeed, datalakes can store all types of data including unstructured ones and we still need to be able to analyse these datasets. These days many companies choose this approach to simplify data interactions with their external data sources. Among other benefits, I like that it works well with semi-complex data schemas.

Data Engineering 82

Data Engineering Data Engineer Engineering BI 82

Towards Data Science

MARCH 9, 2023

Code implementations for ML pipelines: from raw data to predictions Photo by Rodion Kutsaiev on Unsplash Real-life machine learning involves a series of tasks to prepare the data before the magic predictions take place. Those are the features and their respective data types: Image 1 —Features and data types.

Machine Learning 75

Machine Learning Building Datasets Scala 75

ProjectPro

FEBRUARY 8, 2023

Application programming interfaces (APIs) are used to modify the retrieved data set for integration and to support users in keeping track of all the jobs. When Glue receives a trigger, it collects the data, transforms it using code that Glue generates automatically, and then loads it into Amazon S3 or Amazon Redshift.

AWS 98

AWS Scala Metadata Data Lake 98

Ascend.io

MAY 24, 2023

With more than eight years of experience in diverse industries, Sarwat has spent the last four building over 20 data pipelines in both Python and PySpark with hundreds of lines of code. Dive right into Sarwat’s full presentation at the Data Pipeline Automation Summit 2023. Reading not your thing? billion financial records and 8.3

Healthcare 52

Healthcare Data Pipeline Hospitality Datasets 52

Monte Carlo

FEBRUARY 26, 2024

Data catalogs are important because they allow users of varying types to access useful data quickly and effectively and can help team members collaborate and maintain consistent organization-wide data definitions. There’s no shortage of choices when it comes to choosing a data catalog.

Metadata 52

Metadata Government Data Data Governance 52

Striim

NOVEMBER 8, 2023

Organizations can have data product managers who control the data in their domain. They’re responsible for ensuring data quality and making data available to those in the business who might need it. Data as a product This principle can be summarized as applying product thinking to data.

Architecture 52

Architecture Generalist Government Datasets 52

dbt Developer Hub

FEBRUARY 7, 2022

A customer 360 is a fancy way of saying that you have a holistic dataset that lets understand your customers’ behavior. Make sure you check your dataset to see if this is a valid assumption. This is very important for making sure that the domain knowledge is used in the CRM definitions. What's a customer 360? ​

Data Warehouse 52

Data Warehouse Datasets Data SQL 52