dbt Developer Hub

MAY 9, 2022

We only suggest diving into this once you have a strong knowledge of dbt + dbt Cloud. check out the collection documentation to learn how to use it. The dbt Cloud API has well-documented endpoints for creating, triggering and managing dbt Cloud jobs. Please keep in mind that the collection is not an evergreen doc.

Cloud 52

Cloud Management Government Data Warehouse 52

Data Engineering Podcast

JANUARY 13, 2019

New products have been launched, more businesses have moved to streaming analytics, and the team at Timescale has been keeping busy. release, how the use cases for timeseries data have proliferated, and how they are continuing to simplify the task of processing your time oriented events.

Database 100

Database PostgreSQL SQL MongoDB 100

Scott Logic

OCTOBER 18, 2023

Introduction In my previous blog post I discussed how migrating to the Cloud could help your organisation reach its Net Zero goals. I discussed how shifting your workloads away from on-premises data centres can reduce emissions by allowing you to leverage the expertise of cloud providers and their greater efficiency of scale.

Cloud 87

Cloud AWS BI Accessible 87

Snowflake

MAY 8, 2023

Along with our Snowflake technology partners (with similar examples of innovation early last year ), we will demonstrate how they use the new get_query_operator_stats function within their respective products and further empower all of our customers for ongoing success with Snowflake.

Technology 109

Technology SQL BI Utilities 109

Cloudera

JULY 15, 2021

This blog post provides an overview of best practice for the design and deployment of clusters incorporating hardware and operating system configuration, along with guidance for networking and security as well as integration with existing enterprise infrastructure. Further information and documentation [link] .

Architecture 96

Architecture Cloud Kafka Hadoop 96

Cloudera

SEPTEMBER 29, 2022

In this article, we attempt to show how these concepts may be related to each other, and even suggest thinking about all of them all at once (gasp!). In this article, we attempt to show how these concepts may be related to each other, and even suggest thinking about all of them all at once (gasp!). Data mesh defined.

Architecture 73

Architecture Data Architecture Metadata Data Warehouse 73

Data Engineering Weekly

APRIL 23, 2023

Editor’s Note: 🚀🌐🎉 Calling all Data Engineering Weekly Readers! Editor’s Note: 🚀🌐🎉 Calling all Data Engineering Weekly Readers! I believe we have reached a point in the industry where we no longer need to explain what is data contract is and why it is beneficial.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Knowledge Hut

SEPTEMBER 28, 2023

AWS (Amazon Web Service) is a cloud computing platform that provides a range of services virtually, such as storage, computing, deployment services, databases, platform as a service(PaaS), etc. Therefore if we are looking for a high-paying, in-demand job with a lot of growth potential, then look out for AWS jobs.

AWS 52

AWS Cloud Computing Amazon Web Services Portfolio 52

Cloudera

OCTOBER 4, 2022

This blog post will provide guidance to administrators currently using or interested in using Kafka nodes to maintain cluster changes as they scale up or down to balance performance and cloud costs in production deployments. This creates flexibility to handle real-time data feed volumes as they fluctuate. or higher.

Kafka 76

Kafka Data Cloud Big Data 76

Snowflake

AUGUST 8, 2023

In this blog post we will discuss the possibilities this opens for our customers, a high-level view of how it works, and some of the initial use cases we are exploring with early adopters. And they generally know little to nothing about your specific company, your policies, your products, or your customers.

Generalist 73

Generalist Data Ingestion SQL Technology 73

Rockset

JANUARY 26, 2023

The goal of this blog post is to provide best practices on how to use terraform to configure Rockset to ingest the data into two collections, and how to setup a view and query lambdas that are used in an application, plus to show the workflow of later updating the query lambdas. Create a file called _provider.tf

AWS 52

AWS SQL Datasets Kafka 52

Cloudera

SEPTEMBER 1, 2020

Imagine how many times IT has lost valuable time spending hours trying to understand Apache Solr application requirements and map them into how to best size and deploy the Solr service. DDE is a new template flavor within CDP Data Hub in Cloudera’s public cloud deployment option (CDP PC). Stores source documents.

Cloud Storage 66

Cloud Storage Unstructured Data AWS Analytics Application 66

Rockset

AUGUST 28, 2023

Why and what to automate As application developers and architects, whenever we see repeating tasks, we immediately think about how to automate them. Typical use cases for automation Let’s now take a look into typical use cases for automation and show how you would implement them in Rockset.

AWS 52

AWS SQL Datasets Metadata 52

Monte Carlo

JULY 7, 2022

Editor’s Note : We ran into Andrew at our London IMPACT event in early 2022. For posterity, we have preserved Barr’s forward that examines what was then a very nascent trend, but we have also added an updated data contract FAQ as an addendum. Forward: A Data Engineering Movement Afoot?

Data Warehouse 52

Data Warehouse Software Engineer Software Engineering Data 52

Confluent

FEBRUARY 6, 2019

This is important to note since machine learning is clearly gainin g steam, though many who use the term do so by misusing the term. It allows real-time data ingestion, processing, model deployment and monitoring in a reliable and scalable way. Impedance mismatch between data scientists, data engineers and production engineers.

Machine Learning 106

Machine Learning Python Kafka Java 106

Knowledge Hut

NOVEMBER 15, 2023

While there are many valid career options, we have shortlisted the best ones for you. In this article, we will discuss the best career objectives that are suitable for recent graduates or those who are still studying. In today’s competitive job market, it is crucial to have a well-crafted resume that stands out from the rest.

Finance 98

Finance Business Intelligence Database-centric Certification 98

U-Next

AUGUST 23, 2022

Still, it has not been apparent how to utilize this revolutionary technology effectively. We’ll examine what is blockchain technology and how it works in this blog. Recently, blockchain technologies are gaining attention, and it’s easy to understand why. Understanding Blockchain Technology.

Food 40

Food Transportation Medical Utilities 40

Zalando Engineering

JULY 27, 2016

I recently gave a talk at Droidcon Berlin about the best practices for Android developer productivity that we use in our Zalando Tech team. To check your manifest before the APK build, we can use a new feature presented in Android Studio 2.2: What does your AndroidManifest really look like? Merged Manifest Viewer.

Metadata 40

Metadata Coding Java Project 40

Confluent

AUGUST 29, 2019

We know that Apache Kafka ® is great when you’re dealing with streams, allowing you to conveniently look at streams as tables. For example, in a social network, understanding the network means we need to look at the friend relationships between people. But what about when the relationships between items dominate your application?

Kafka 55

Kafka Process Algorithm Cloud 55

DoorDash Engineering

MARCH 7, 2023

While most engineering tooling at DoorDash is focused on making safe incremental improvements to existing systems, in part by testing in production (learn more about our end-to-end testing strategy ), this is not always the best approach when launching an entirely new business line. Often we’re going from zero to one.

AWS 119

AWS PostgreSQL Database SQL 119

Netflix Tech

JUNE 13, 2023

Our previous blog post presented replay traffic testing — a crucial instrument in our toolkit that allows us to implement these transformations with precision and reliability. This blog post will delve into the techniques leveraged at Netflix to introduce these changes to production.

Systems 111

Systems Entertainment Utilities Coding 111

Cloudera

OCTOBER 22, 2021

With the latest release of Cloudera DataFlow for the Public Cloud (CDF-PC) we added new CLI capabilities that allow you to automate data flow deployments, making it easier than ever before to incorporate Apache NiFi flow deployments into your CI/CD pipelines. Understanding the data flow development lifecycle.

Cloud 89

Cloud Data Accessible Accessibility 89

Christophe Blefari

DECEMBER 30, 2022

ANALYTICS ENGINEERING We have to be honest in 2022 Analytics Engineering shaped up the data field and concentrated a lot of data discussions. Analytics Engineering can be seen as a renaming of the BI Engineering, if we look at it more precisely it mainly comes out of the data roles specialisation. Now let's jump to my selection.

Data Warehouse 130

Data Warehouse BI Data Machine Learning 130

Tweag

MAY 16, 2023

A bit more than one year ago, we released the very first public version Nickel (0.1). The manifest of a web app, the configuration of an Apache virtual host, an Infrastructure-as-Code (IaC) cloud deployment (Terraform, Kubernetes, etc.). Please see the release notes for more details. Today, I am very excited to announce the 1.0

MySQL 131

MySQL Metadata Data Validation Coding 131

Knowledge Hut

JANUARY 23, 2024

In this blog, I’ve covered the basics of Markdown. This blog is for beginners as well as for people who have used markdown but may not know the entire syntax and what you can do with it. Markdown is a lightweight markup language that describes how text should look on a page. What is Markdown?

Portfolio 96

Portfolio Media Python Programming Language 96

Confluent

MAY 21, 2019

An example can be the backend architecture for an insurance product. If the user changed their residential address, the profile service will also call the quote service so that the insurance coverage can be recalculated (after all, some neighborhoods are safer than others). Chattanooga, TN” }. What will the profile service publish?

Kafka 110

Kafka Insurance Architecture Metadata 110

Lyft Engineering

NOVEMBER 17, 2022

This blog post will focus on the systems we’ve built to address OS and OS-package level vulnerabilities in a timely manner across hundreds of services run on Kubernetes. Along the way, we’ll highlight the technical challenges we encountered and how we eliminated most of the work required from other engineers.

Management 59

Management AWS Programming Project 59

Knowledge Hut

NOVEMBER 20, 2023

Also, the course outlines how software developers and IT services professionals communicate, work, and integrate as a single unit. In this article, we will discuss everything that you need to know about the DevOps Foundation certification course. Are you looking to become a certified DevOps professional?

Certification 52

Certification Education Electronics Recruitment 52

Cloudera

APRIL 21, 2021

For more than a decade, Cloudera has built internal tools and data analysis primarily on a single production CDH cluster. In this blog, we discuss our journey to CDP for this critical cluster. In this blog, we discuss our journey to CDP for this critical cluster. Our Internal Environment Before Upgrade.

Cloud 116

Cloud Professional Services Java Data Warehouse 116

Engineering at Meta

MAY 3, 2023

We know that malicious groups behind malware campaigns are extremely persistent, and we fully expect them to keep trying to come up with new tactics and tooling in an effort to survive disruptions by any one platform where they spread. For more security tips, visit our Newsroom. For more security tips, visit our Newsroom.

Media 110

Media Metadata Coding Database 110

Knowledge Hut

JANUARY 12, 2024

Docker has gained immense popularity for the dramatic change it has brought to the IT world. Also, we will learn how to build a Docker environment and use Docker commands. Containerization enables tremendous economies of scale and has made development scalable while remaining user-friendly. What Is Docker? What Are Containers?

Systems 97

Systems MySQL Architecture Certification 97

Confluent

JUNE 18, 2019

Luckily, this task looks a lot like the way we tackle problems that arise when connecting data. A common first step is using the application persistence layer to save the documents directly to the database as well as to the search engine. Luckily, this task looks a lot like the way we tackle problems that arise when connecting data.

Architecture 95

Architecture Building Kafka Database-centric 95

phData: Data Engineering

OCTOBER 7, 2022

So you’ve built your first set of transformations in dbt , but now you need to figure out how to automate your deployment and code changes to your various environments. Thankfully, you’ve found this blog! The “how” will depend on the way that you decide to architect your data warehouse.

Database 52

Database Cloud Coding Python 52

Workfall

MARCH 21, 2023

We can use database management systems to perform all database actions through a graphical user interface. With these GUIs, we can get a bird’s-eye view of all the data in our database for easy analysis of the schema or data types, as well as general ease of administration. In this blog, we’ll explore: What is SurrealDB?

PostgreSQL 68

PostgreSQL NoSQL Database Unstructured Data 68

dbt Developer Hub

AUGUST 17, 2022

When running a job that has over 1,700 models, how do you know what a “good” runtime is? Here at dbt Labs, we used the Model Timing tab in our internal analytics dbt project to help us identify inefficiencies in our incremental dbt Cloud job that eventually led to major financial savings, and a path forward for periodic improvement checks.

Data Warehouse 59

Data Warehouse Datasets Cloud Coding 59

Knowledge Hut

JANUARY 31, 2024

In this blog, I’ll take you through a step-by-step guide on creating, building, and using the Angular library. In this article, we are going to talk about Libraries in Angular 13 and how to build a library for angular apps. You can actually create your custom brick to fill that gap and make your structure even more unique.

Building 52

Building Coding Project Programming Language 52

phData: Data Engineering

FEBRUARY 16, 2023

This is due to a number of reasons, which we covered in a previous blog post , but in short, it’s primarily due to a need to accelerate platform adoption outside of central IT teams. This blog will walk you through the steps to successfully execute a dbt Core to dbt Cloud migration.

Cloud 52

Cloud Database Data Ingestion Coding 52