Cloudera

JUNE 2, 2022

Over the last two years, the Cloudera DataFlow team has been hard at work building Cloudera DataFlow for the Public Cloud (CDF-PC). allowing developers to connect to any data source anywhere with any structure, process it, and deliver to any destination. How to onboard data into their system? I don’t care about their system.

Systems 99

Systems Data Lake Google Cloud Data Collection 99

Knowledge Hut

MARCH 18, 2024

Integrated Change Control (ICC) provides a comprehensive process for reviewing, evaluating, and approving changes to project scope, schedule, and resources. In this blog, I will walk you through the definition, purpose, process, and importance of integrated change control, as well as explore the benefits and best practices associated with it.

Process 52

Process Certification Project Utilities 52

DataKitchen

DECEMBER 9, 2022

ChatGPT> DataOps, or data operations, is a set of practices and technologies that organizations use to improve the speed, quality, and reliability of their data analytics processes. It involves bringing together people, processes, and technology to enable data-driven decision making and improve the efficiency of data-related workflows.

Machine Learning 52

Machine Learning Data Preparation Government Data Analytics 52

Knowledge Hut

JANUARY 31, 2024

As business systems get increasingly complex, information technology plays a critical role in ensuring the smooth day-to-day functioning of present-day systems. To know more about ITIL standards, CMBD, and ITIL learning paths, read the blog. It calls this value system the ITIL Service Value System (SVS) , involving 34 practices.

Process 52

Process Certification Telecommunication Database 52

Pinterest Engineering

SEPTEMBER 20, 2023

Our commitment is evidenced by our history of building products that champion inclusivity. We know from experience that building for marginalized communities helps make the product work better for everyone. Signal Development and Indexing The process of developing our visual body type signal essentially begins with data collection.

Building 85

Building Pipeline-centric Machine Learning Datasets 85

Knowledge Hut

JANUARY 31, 2024

Managing competing priorities, costs, risks, and responsibilities necessitates a clear process and cross-functional collaboration. This blog reflects on the SDP in ITIL. Exploring the principles, processes, and importance of ITIL service design. This article will teach us about ITIL Service Design.

Designing 52

Designing Process IT Portfolio 52

Cloudera

JUNE 28, 2022

In a previous blog of this series, Turning Streams Into Data Products , we talked about the increased need for reducing the latency between data generation/ingestion and producing analytical results and insights from this data. Building real-time streaming analytics data pipelines requires the ability to process data in the stream.

Process 81

Process Kafka SQL Machine Learning 81

Knowledge Hut

JANUARY 30, 2024

This blog is an account of the conversation which will serve ITIL aspirants well. So, this framework provides uniform and consistent guidelines to all IT industries to define their IT Service Management processes. Ensuring organizations can improve Customer satisfaction by building and maintaining positive business relationships.

Process 52

Process Certification Telecommunication Government 52

FreshBI

MAY 25, 2021

Business Processing Dashboards Explained A Business Workflow is visual representation of steps through a business activity used to support and validate your Business Intelligence Objectives. Overall Business Efficiency: Business Processing Dashboards are the epitome of heightened business efficiency.

Process 52

Process Business Intelligence BI Professional Services 52

Cloudera

JULY 18, 2022

In part 1 of this blog we discussed how Cloudera DataFlow for the Public Cloud (CDF-PC), the universal data distribution service powered by Apache NiFi, can make it easy to acquire data from wherever it originates and move it efficiently to make it available to other applications in a streaming fashion. Data decays! Use case recap.

Process 86

Process Kafka Scala SQL 86

AltexSoft

AUGUST 25, 2021

And this technology of Natural Language Processing is available to all businesses. Available methods for text processing and which one to choose. Tools you can use to build NLP models. What is Natural Language Processing? Here are some big text processing types and how they can be applied in real life.

Process 139

Process Deep Learning Datasets Machine Learning 139

Snowflake

DECEMBER 11, 2023

While digital advertisers and paid media professionals are on the hook to build ample sales pipeline and maximize return on ad spend (ROAS), they’re also expected to deliver personalized advertising content while navigating evolving privacy requirements and adhering to consumer expectations—all while extracting insights from siloed ad platforms.

Building 88

Building Cloud Media Algorithm 88

Data Engineering Podcast

NOVEMBER 26, 2019

In this episode James Cunningham and Ted Kaemming describe the process of rearchitecting a production system, what they learned in the process, and some useful tips for anyone else evaluating Clickhouse. What did the previous system look like? What was your design criteria for building a new platform?

Data Warehouse 100

Data Warehouse Building PostgreSQL Kafka 100

Databand.ai

AUGUST 30, 2023

Building a Winning Data Quality Strategy: Step by Step Eitan Chazbani August 30, 2023 What Is a Data Quality Strategy? A data quality strategy details the processes, tools, and techniques employed to ensure your company’s data is accurate, consistent, complete, and up-to-date.

Building 52

Building Data Cleanse Data Governance Datasets 52

Data Engineering Podcast

APRIL 19, 2020

Summary Modern applications frequently require access to real-time data, but building and maintaining the systems that make that possible is a complex and time consuming endeavor. This was an interesting inside look at building a business on top of open source stream processing frameworks and how to reduce the burden on end users.

Building 100

Building PostgreSQL MongoDB SQL 100

Netflix Tech

OCTOBER 19, 2020

which is difficult when troubleshooting distributed systems. In our previous blog post we introduced Edgar, our troubleshooting tool for streaming sessions. The process started with manual pull of member account information that was part of the session. The following sections describe our journey in building these components.

Building 93

Building Transportation Metadata Java 93

Zalando Engineering

JUNE 30, 2020

Our Engineering Blog was launched in June 2020 after a long break of the previous tech blog. What customizations we applied to design the blog and the publishing process. Static Site Generator Our previous tech blog used a CMS which only a limited number of people had access to. It's actively developed.

Engineering 40

Engineering Bytes AWS Python 40

Pinterest Engineering

MAY 13, 2024

The subsequent blog post will delve into how we looked into our specific needs, evaluated multiple candidates and decided on the adoption of a new database technology. Yet the HBase version upgrade is a slow and painful process due to a legacy build/deploy/provisioning pipeline and compatibility issues (the last upgrade from 0.94

NoSQL 69

NoSQL MySQL Database Systems 69

Data Engineering Weekly

MAY 3, 2024

There is a lot of context missing in that tweet, so I decided to write a blog about it. However, modern Notebooks like Databricks seamlessly integrate with Git to build pull requests and code review processes. There is no underlying semantics for unit testing the code and data testing build-in. Where Notebook Shines?

Programming Language 117

Programming Language ETL Tools Data Pipeline Coding 117

DareData

JULY 5, 2023

Learn how we build data lake infrastructures and help organizations all around the world achieving their data goals. In today's data-driven world, organizations are faced with the challenge of managing and processing large volumes of data efficiently.

Data Lake 52

Data Lake Building Raw Data ETL Tools 52

Knowledge Hut

JULY 28, 2023

Data Labeling is the process of assigning meaningful tags or annotations to raw data, typically in the form of text, images, audio, or video. If some terminologies in the blog around Machine Learning seems unfamiliar to you, don’t worry we have the Best Data Science courses to help you out. How Does Data Labeling Work?

Machine Learning 52

Machine Learning Process Datasets Raw Data 52

Striim

NOVEMBER 17, 2023

Striim serves as a real-time data integration platform that seamlessly and continuously moves data from diverse data sources to destinations such as cloud databases, messaging systems, and data warehouses, making it a vital component in modern data architectures. In this blog, we will focus on a PostgreSQL database.

Machine Learning 52

Machine Learning Data Process PostgreSQL Process 52

Data Engineering Weekly

APRIL 21, 2023

Ananth and Aswin discussed a blog from BuzzFeed that shares lessons learned from building products powered by generative AI. The blog highlights how generative AI can be integrated into a company's work culture and workflow to enhance creativity rather than replace jobs.

Building 52

Building Data Architect Education Data Engineering 52

DataKitchen

MAY 10, 2024

This blog post introduces five critical use cases for data observability, each pivotal in maintaining the integrity and usability of data throughout its journey in any enterprise. Data ingestion observability helps organizations avoid costly mistakes when poor-quality data is used in critical business processes.

Data Ingestion 56

Data Ingestion Datasets Data Coding 56

Data Engineering Weekly

MAY 12, 2024

[link] Gradient Flow: Learning from the Past - Comparing the Hype Cycles of Big Data and GenAI The blogs compare the hype cycle of Big Data with Gen AI. The blog narrates the initial hype of Big Data, followed by talent shortages and lead time to build production applications.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Lake 52

Knowledge Hut

JUNE 20, 2023

However, to succeed in e-commerce, businesses require certain skills and tools to build and run an effective online storefront. Hence, it is recommended that you build your career with professional courses that help you acquire the necessary skills to thrive in the world of e-commerce.

Building 52

Building Media Data Analysis Certification 52

Tweag

JUNE 7, 2023

The availability of deep learning frameworks like PyTorch or JAX has revolutionized array processing, regardless of whether one is working on machine learning tasks or other numerical algorithms. However, writing high-performance array processing code in Haskell is still a non-trivial endeavor.

Process 89

Process Coding Python Deep Learning 89

Netflix Tech

MAY 4, 2023

Behind the scenes, a myriad of systems and services are involved in orchestrating the product experience. These backend systems are consistently being evolved and optimized to meet and exceed customer and product expectations. This blog series will examine the tools, techniques, and strategies we have utilized to achieve this goal.

Utilities 135

Utilities Systems Architecture Coding 135

Zalando Engineering

MARCH 1, 2021

In order to provide an excellent customer experience, we must ensure our systems can technically handle high traffic events such as Cyber Week or other sales campaigns. Checkout and payments related systems are particularly important during sales events. We have published a detailed article on how Zalando prepares for the Cyberweek.

Systems 40

Systems Building Java Algorithm 40

Monte Carlo

JUNE 22, 2023

Building the platform as composable architecture Their next step was to build the product…that would be used to build data products. The platform team started thinking through which services domains would need to build, maintain, and deliver high-quality, FAIR data products to their end-users.

Building 52

Building Data Ingestion Data Business Analyst 52

Netflix Tech

JUNE 1, 2023

To do this, we devised a novel way to simulate the projected traffic weeks ahead of launch by building upon the traffic migration framework described here. Replay traffic enabled us to test our new systems and algorithms at scale before launch, while also making the traffic as realistic as possible.

Algorithm 136

Algorithm Metadata Kafka Systems 136

Cloudera

SEPTEMBER 11, 2018

The open data processing pipeline. To take advantage of all the disparate types of data, Red Hat, Eurotech and Cloudera have built an open and scalable data processing pipeline as part of an end-to-end IoT architecture. Telemetry data routed to the Cloudera Enterprise Data Hub flows into Apache Kafka.

Data Process 40

Data Process Process Building Machine Learning 40

Edureka

MAY 6, 2024

However, building and deploying these models is just the beginning. MLOps is an emerging discipline that aims to unify and streamline the machine learning system development (Dev) and operations (Ops) lifecycle. But before we begin, Let’s have a look at what we will be covering in this blog: What is MLOPS?

Machine Learning 52

Machine Learning Metadata Programming Language Healthcare 52

Cloudera

JANUARY 20, 2021

For more context, this demo is based on concepts discussed in this blog post How to deploy ML models to production. With this example as inspiration, I decided to build off of sensor data and serve results from a model in real-time. As a result, I decided to use an open-source Occupancy Detection Data Set to build this application.

Machine Learning 112

Machine Learning Database Data Science Building 112

Data Engineering Weekly

NOVEMBER 11, 2022

Not too long ago, I worked for a famous messaging platform; You may be typing on it as you read this blog. I took a break from data engineering and focused on system monitoring and observability for some time. Little did I know I would be building data pipelines to measure system efficiency instead of measuring business efficiency.

Systems 40

Systems High Quality Data Consulting Metadata 40

Data Engineering Weekly

FEBRUARY 4, 2024

The user journey, sales process, marketing campaign, everything falls under a state machine. Data modeling is a collaborative process across business units to capture state changes in business activity. The challenge with the OLAP engine is that it is an additional system to maintain, and indexing everything is too costly.

Data Engineering 62

Data Engineering Data Engineer Engineering PostgreSQL 62

Snowflake

APRIL 29, 2024

DORA: Building a More Secure Financial System DORA, enacted in January 2023, moves beyond reactive measures, requiring FEs and their service providers to proactively identify vulnerabilities, prevent disruptions and plan for swift recovery from incidents. Five Core Pillars of DORA 1.

Transportation 83

Transportation Data Governance Government Consulting 83