DareData

MARCH 5, 2024

At DareData Engineering, we believe in a human-centric approach, where AI agents work together with humans to achieve faster and more efficient results. At its core, RAG harnesses the power of large language models and vector databases to augment pre-trained models (such as GPT 3.5 ).

Database-centric 52

Database-centric Telecommunication SQL Unstructured Data 52

Ripple Engineering

JANUARY 29, 2024

Why do we need a Data Platform? At Ripple, different business units require a platform to quickly and easily turn their data into insights and compelling customer experiences.   A lack of a centralized system makes building a single source of high-quality data difficult.

Database-centric 52

Database-centric Pipeline-centric NoSQL High Quality Data 52

Data Engineering Podcast

AUGUST 13, 2022

Summary Data engineers have typically left the process of data labeling to data scientists or other roles because of its nature as a manual and process heavy undertaking, focusing instead on building automation and repeatable systems. Data stacks are becoming more and more complex.

Machine Learning 100

Machine Learning Pipeline-centric Database-centric MongoDB 100

Grouparoo

MAY 13, 2021

In this post, we’ll look at the historical reasons for the 191 character limit as a default in most relational databases. The first question you might ask is why limit the length of the strings you can store in a database at all? Indexes can be made to perform better when assumptions can be made about the type of data they store.

Database 52

Database Bytes MySQL Database-centric 52

Christophe Blefari

APRIL 8, 2023

Data News entering in town ( credits ) Hey you, if I wasn't late in my newsletter writing it wouldn't be me. But here is your usual Data News. Data modeling Dear readers, I have to confess something. I did not care about data modeling for years. Data modeling should not be a required data engineer skill.

Pipeline-centric 130

Pipeline-centric Database-centric Algorithm Data 130

Christophe Blefari

APRIL 8, 2023

Data News entering in town ( credits ) Hey you, if I wasn't late in my newsletter writing it wouldn't be me. But here is your usual Data News. Data modeling Dear readers, I have to confess something. I did not care about data modeling for years. Data modeling should not be a required data engineer skill.

Pipeline-centric 130

Pipeline-centric Database-centric Algorithm Data 130

Data Engineering Weekly

JULY 2, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork, and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. So, let's shape the future of Data Engineering together. See how it works today.

Data Engineering 52

Data Engineering Data Engineer Engineering Pipeline-centric 52

Knowledge Hut

MARCH 15, 2024

Making decisions in the database space requires deciding between RDBMS (Relational Database Management System) and NoSQL, each of which has unique features. RDBMS uses SQL to organize data into structured tables, whereas NoSQL is more flexible and can handle a wider range of data types because of its dynamic schemas.

NoSQL 52

NoSQL Database-centric Relational Database PostgreSQL 52

Knowledge Hut

MARCH 29, 2024

Showcases a data-driven approach to continuous improvement and process optimization. Data Scientist Template Link: Data Scientist (Download here) Why This Cover Letter Works: Highlights technical proficiency in relevant data analysis tools and programming languages. Showcases Python, Scala, and data modeling skills.

Database-centric 52

Database-centric Programming Language BI Finance 52

U-Next

AUGUST 17, 2022

Introduction to 2022 Data Engineer Roles and Responsibilities. Companies and enterprises, large and small, are built on data. Data Engineer roles and responsibilities include aiding in the collection of issues and the delivery of remedies addressing customer demand and product accessibility.

Data Engineering 52

Data Engineering Data Engineer Pipeline-centric Database-centric 52

Knowledge Hut

JUNE 20, 2023

A novice data scientist prepared to start a rewarding journey may need clarification on the differences between a data scientist and a machine learning engineer. Many people are learning data science for the first time and need help comprehending the two job positions. Apache Spark, Microsoft Azure, Amazon Web services, etc.

Machine Learning 52

Machine Learning Data Engineering Data Engineer Engineering 52

Knowledge Hut

FEBRUARY 27, 2023

A data engineer is a key member of an enterprise data analytics team and is responsible for handling, leading, optimizing, evaluating, and monitoring the acquisition, storage, and distribution of data across the enterprise. Data Engineers indulge in the whole data process, from data management to analysis.

Data Engineering 52

Data Engineering Data Engineer Database-centric Pipeline-centric 52

Ascend.io

SEPTEMBER 14, 2023

The rise of data-intensive operations has positioned data engineering at the core of today’s organizations. As the demand to efficiently collect, process, and store data increases, data engineers have started to rely on Python to meet this escalating demand. Why Python for Data Engineering?

Data Engineering 52

Data Engineering Data Engineer Python Engineering 52

Knowledge Hut

JANUARY 30, 2024

In today's world, where data rules the roost, data extraction is the key to unlocking its hidden treasures. As someone deeply immersed in the world of data science, I know that raw data is the lifeblood of innovation, decision-making, and business progress. What is data extraction?

ETL Tools 52

ETL Tools Database-centric Data Mining Data Cleanse 52

RandomTrees

NOVEMBER 29, 2023

The choice of tools and adopting the right methodologies becomes the drive for any business and in this the selection on the database technologies is very crucial. In this choice, Big Data will play an important role and its choice is also inevitably crucial in the Business Intelligence and related systems.

Data Warehouse 52

Data Warehouse Data Lake Database-centric Data Cleanse 52

Knowledge Hut

APRIL 25, 2024

Kubernetes (sometimes shortened to K8s with the 8 standing for the number of letters between the “K” and the “s”) is an open-source system to deploy, scale, and manage containerized applications anywhere. Kubernetes is a container-centric management software that allows the creation and deployment of containerized applications with ease.

Database-centric 81

Database-centric Metadata MongoDB Pipeline-centric 81

Cloudera

OCTOBER 17, 2023

The adoption of Flink mirrors growth in streaming data volumes and maturity of the streaming market. Cloudera’s perspective: Cloudera saw the increasing volumes of data our customers were moving via streams early on. Also, why solve for connectivity and data distribution again if it’s already solved for? What about data lock-in?

Database-centric 77

Database-centric Kafka Pipeline-centric Database 77

Knowledge Hut

NOVEMBER 2, 2023

For those aspiring to build a career within the Azure ecosystem, navigating the choices between Azure Data Engineers and Azure DevOps Engineers can be quite challenging. Azure Data Engineers and Azure DevOps Engineers are two critical components of the Azure ecosystem for different but interconnected reasons.

Data Engineering 52

Data Engineering Data Engineer Engineering Pipeline-centric 52

Knowledge Hut

DECEMBER 26, 2023

Today's trends include data analytics, artificial intelligence, big data, and data science. Business organizations are adopting data-driven models to simplify their processes and make decisions based on the insights derived from data analytics. What i s Data Science ?

Data Science 52

Data Science Database-centric Pipeline-centric Data Mining 52

Data Engineering Podcast

APRIL 7, 2019

Summary How much time do you spend maintaining your data pipeline? This was a fascinating conversation with someone who has spent his entire career working on simplifying complex data problems. Managing and auditing access to your servers and databases is a problem that grows in difficulty alongside the growth of your teams.

Data Pipeline 100

Data Pipeline Pipeline-centric Database-centric AWS 100

Towards Data Science

JULY 13, 2023

The example we’ll walk you through will mirror a typical LLM application workflow you’d run to populate a vector database with some text knowledge. Specifically, we’ll cover pulling data from the web, creating text embeddings (vectors) and pushing them to a vector store. The application will receive a small data input (e.g.,

Building 72

Building Database-centric Database Coding 72

Knowledge Hut

APRIL 24, 2024

It may run on various operating systems, including Windows, Linux, and Mac OS. Any form of data in js applications was never buffered. Data is output in chunks by Node.js host never awaits data from such an API. As a result, it is used to create network-centric applications. It is now crystal clear to us.

Programming Language 52

Programming Language Database-centric Programming Python 52

Rockset

AUGUST 2, 2023

With 90% of trade being transported via sea , this data is crucial to keeping the global supply chain on track but can be difficult to disentangle and take action on. In 2022, Windward embarked on several changes to its application prompting a reconsideration of its underlying data stack.

Database-centric 52

Database-centric PostgreSQL Transportation Insurance 52

phData: Data Engineering

DECEMBER 20, 2022

Congratulations—at last, you’ve finished your cloud data migration and it feels as if you just bought a brand new car! Hopefully, your cutting-edge cloud data warehouse comes with the latest and greatest safety features, too. A complete migration needs a state-of-the-art data quality monitoring tool as a seatbelt to keep the data safe.

Cloud 52

Cloud Database-centric Data Data Warehouse 52

The Pragmatic Engineer

JANUARY 5, 2023

We also recommend customers review internal logs for their systems for any unauthorized access starting from December 21, 2022, through today, January 4, 2023, or upon completion of your secrets rotation. (.)We We take the security of our systems and our customers’ systems extremely seriously.

Pipeline-centric 130

Pipeline-centric Database-centric Coding Accessible 130

Meltano

JANUARY 25, 2023

A curated list of the top 9 must read blogs on data. The data world is in turmoil and lots of exciting things happen every day, week and year. At Meltano we’re ourselves avid users of data, data engineers, data PMs, and data enthusiasts through and through. Happy reading!

Pipeline-centric 52

Pipeline-centric Database-centric Data Warehouse SQL 52

Ascend.io

JUNE 8, 2023

Because of its transformative potential, data has graduated from being merely a by-product of business operations to a critical asset in its own right. Is it possible to treat data not just as a necessary operational output, but as a product that holds immense strategic value? Without them, data products can’t exist.

Pipeline-centric 52

Pipeline-centric Database-centric Data Ingestion Data Pipeline 52

Knowledge Hut

NOVEMBER 17, 2023

The demand for data-related professions, including data engineering, has indeed been on the rise due to the increasing importance of data-driven decision-making in various industries. Becoming an Azure Data Engineer in this data-centric landscape is a promising career choice. Who is an Azure Data Engineer?

Data Engineering 52

Data Engineering Data Engineer Engineering Scala 52

Knowledge Hut

DECEMBER 27, 2023

In the early 1800s, as the field of statistics expanded, it included collecting and analyzing data. But it saw the first problem with the overwhelming amount of data. Throughout the 20th century, volumes of data kept growing at an unexpected speed and machines started storing information magnetically and in other ways.

Big Data Tools 59

Big Data Tools Big Data Hadoop Database-centric 59

AltexSoft

JUNE 7, 2021

Hadoop and Spark are the two most popular platforms for Big Data processing. They both enable you to deal with huge collections of data no matter its format — from Excel tables to user feedback on websites to images and video files. Which Big Data tasks does Spark solve most effectively? How does it work? cost-effectiveness.

Big Data Tools 98

Big Data Tools Hadoop Big Data Database-centric 98

Precisely

JANUARY 2, 2024

IBM Power Systems running the IBM i operating systems are noted for their robust security. Yet without proactive user management and attention to security settings (zero trust), even these highly reliable systems are vulnerable to attack. NIST describes the concept as a “system of systems.” The world has changed.

IT 57

IT Database-centric Government Cloud Computing 57

The Modern Data Company

MARCH 1, 2024

For today’s Chief Data Officers (CDOs) and data teams, the struggle is real. We’re drowning in data yet thirsting for actionable insights. We need a new approach, a paradigm shift that delivers data with the agility and efficiency of a speedboat – enter Data Products.

Database-centric 40

Database-centric Data Lake Government Retail 40

Maxime Beauchemin

JANUARY 20, 2017

By the time I left in 2013, I was a data engineer. We were data engineers! Data Engineering? Data science as a discipline was going through its adolescence of self-affirming and defining itself. At the same time, data engineering was the slightly younger sibling, but it was going through something similar.

Data Engineering 130

Data Engineering Data Engineer Engineering ETL Tools 130

Dataquest

JANUARY 25, 2017

From helping cars drive themselves to helping Facebook tag you in photos , data science has attracted a lot of buzz recently. Data scientists have become extremely sought after , and for good reason — a skilled data scientist can add incredible value to a business. But what about data engineers?

Data Engineering 52

Data Engineering Data Engineer Pipeline-centric Database-centric 52

Monte Carlo

JANUARY 23, 2024

Predicting the future of data and AI in 2024 is not for the faint of heart. Read on to hear their predictions for the 5 pillars the industry will be built on in 2024, and their recommendations for how data teams can maximize AI’s current capabilities in the short-term — while laying the groundwork for long-term transformation.

Pipeline-centric 52

Pipeline-centric Database-centric Metadata Unstructured Data 52

Knowledge Hut

MARCH 26, 2024

Solution architects work with developers, stakeholders, and other IT specialists to build unified and effective systems. Extensive experience in software development, architecture design, and system integration. Responsibilities: Define the architecture for software systems, including components, interfaces, and integration points.

Amazon Web Services 52

Amazon Web Services Google Cloud Computer Science AWS 52

Monte Carlo

JANUARY 9, 2024

One dark and stormy night, an innocent data team completed a crucial user-facing system conversion. Neither the old or the new data was wrong, but because it was inconsistent with their existing mental model, it was a phantom that distorted reality for all the teams involved. But unlike most nightmares, they’re real.

Pipeline-centric 52

Pipeline-centric Database-centric Data Manufacturing 52