Monte Carlo

AUGUST 31, 2023

This book focuses primarily on data quality as a function of powering data analytical data pipelines for building decision making dashboards, data products, ML models, and other data science outputs. For example, a new field added to a data table by one team may cause another team’s pipeline to fail, resulting in missing or partial data.

Data Lake 52

Data Lake Data Pipeline Unstructured Data Data Warehouse 52

DataKitchen

JULY 27, 2023

Azure Databricks Workflows : An Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. Workflows is a DAG runner embedded in Databricks. Databricks Notebooks are often used in conjunction with Workflows. SQL Server Integration Services (SSIS): You know it; your father used it.

Data Pipeline 52

Data Pipeline BI Machine Learning Data Preparation 52

Rockset

OCTOBER 26, 2023

In this blog, we’ll discuss how JetBlue built an in-house machine learning platform, BlueML, that enables teams to quickly productionize new machine learning applications using a common library and configuration. As a result, teams can productionize new features and models with minimal engineering lift.

Machine Learning 64

Machine Learning Data Science Architecture Data Warehouse 64

dbt Developer Hub

OCTOBER 17, 2022

For years working in data and analytics engineering roles, I treasured the daily camaraderie sharing a small office space with talented folks using a range of tools - from analysts using SQL and Excel to data scientists working in Python. When combined in an intentional manner, SQL, dataframes, and Python are also stronger together.

SQL 52

SQL Python Programming Language Data Analysis 52

ProjectPro

AUGUST 24, 2021

For beginners or peeps who are utterly new to the data industry, Data Scientist is likely to be the first job title they come across, and the perks of being one usually make them go crazy. And, out of these professions, this blog will discuss the data engineering job role. Machine Learning web service to host forecasting code.

Data Engineering 52

Data Engineering Data Engineer Coding Project 52

DataKitchen

APRIL 13, 2021

Read the complete blog below for a more detailed description of the vendors and their capabilities. Because it is such a new category, both overly narrow and overly broad definitions of DataOps abound. Composable Analytics — A DataOps Enterprise Platform with built-in services for data orchestration, automation, and analytics.

Consulting 123

Consulting Machine Learning Data Science Data Pipeline 123

ProjectPro

JANUARY 31, 2022

This blog walks you through what does Snowflake do , the various features it offers, the Snowflake architecture, and so much more. Snowflake is a Data Warehouse solution that supports ANSI SQL and is available as a SaaS (Software-as-a-Service). Launched in 2014, Snowflake is one of the most popular cloud data solutions on the market.

Architecture 52

Architecture IT Data Warehouse Amazon Web Services 52

ProjectPro

DECEMBER 10, 2021

This blog covers the top 50 most frequently asked Azure interview questions and answers. According to Microsoft , around 1000 firms are turning to or employing Microsoft Azure for cloud computing, implying that nearly 365,000 new businesses will employ Azure specialists each year. So, let's dive right into it! of the time. of the time.

BI 52

BI Cloud Computing SQL Database 52

ProjectPro

NOVEMBER 15, 2021

Contributing to an open-source big data project has numerous potential benefits for developers and data scientists, including acquiring new skills, interacting with the community, developing a solid network, and sharpening skillset. This blog will walk through the most popular and fascinating open source big data projects.

Big Data 52

Big Data Project Metadata Programming Language 52

ProjectPro

NOVEMBER 22, 2021

Although Spark was originally created in Scala, the Spark Community has published a new tool called PySpark, which allows Python to be used with Spark. It's more commonly used to alter data with functional programming structures than with domain-specific expressions. They are, however, able to do this only through the use of Py4j.

Hadoop 52

Hadoop Python Datasets Metadata 52

ProjectPro

JULY 27, 2021

This blog is your one-stop solution for the top 100+ Data Engineer Interview Questions and Answers. In this blog, we have collated the frequently asked data engineer interview questions based on tools and technologies that are highly useful for a data engineer in the Big Data industry. SQL works on data arranged in a predefined schema.

Data Engineering 40

Data Engineering Data Engineer Engineering Hadoop 40

phData: Data Engineering

JANUARY 3, 2022

The tools do not fully support software engineering principles and the following questions begin to take root: How do I introduce new functionality without impacting existing users? How do I ensure customers aren’t impacted by changes or new functionality? This informs you as to what changes are new and what is existing.

IT 52

IT AWS Software Engineer Software Engineering 52

DareData

FEBRUARY 1, 2021

It especially applies if you have legacy systems such as Oracle, SAS, SAP, Microsoft SQL Server, in-house infrastructure, etc. When application developers need to test a piece of functionality you can usually spin up an entire environment on your own machine and test it as a whole. Let's not use something just because Google does.

Programming Language 52

Programming Language Project Engineering Consulting 52

Monte Carlo

JANUARY 23, 2024

In 2024, data engineers can run big jobs very quickly thanks to the compute power of BigQuery, Snowflake, Firebolt, Databricks, and other cloud warehousing technologies. For example, while Maxime was at Airbnb, Dataportal was built to democratize data access and empower all Airbnb employees to explore, understand, and trust data.

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

ProjectPro

MAY 31, 2021

This blog lists over 20 big data projects you can work on to showcase your big data skills and gain hands-on experience in big data tools and technologies. A definite purpose of what you want to do with data must be identified, such as a specific question to be answered, a data product to be built, etc.,

Big Data 52

Big Data Coding Project Hadoop 52

Rockset

NOVEMBER 23, 2021

We built Rockset with the mission to make real-time analytics easy and affordable in the cloud. Benchmarking Responsibly We are in complete agreement with Snowflake and Databricks on one thing: that anyone who publishes benchmarks should do them in a fair, transparent, and replicable manner. Deeply-nested JSON and dynamic schemas.

MongoDB 59

MongoDB Data Ingestion PostgreSQL SQL 59

Cloudera

APRIL 1, 2024

The integrated data services from stream analytics, data flow, and data engineering to data warehousing and machine learning all operate on a single copy of data residing in Iceberg tables for a true multi-function analytics experience. We have embedded LLMs in our Hue interface so you can write SQL queries in English or any other language.

Cloud 100

Cloud Unstructured Data Metadata Datasets 100