Data Engineering Weekly

JUNE 18, 2023

Sanjeev Mohan: How to Use Large Language Models (LLMs) on Private Data: A Data Strategy Guide Despite all the attention AI garners, some high-profile missteps have reminded the world once again of “garbage in, garbage out.” Sanjeev highlights three options for an organization to adopt with their private model.

Data Engineering 52

Data Engineering Data Engineer Engineering MySQL 52

Data Engineering Weekly

MARCH 12, 2023

We take three articles and do an in-depth analysis, and we hope to bring the author of the blogs to discuss more. The blog narrates the architecture to implement an outbox pattern with Debezium, the usage of the outbox router in Debezium, and lessons learned. We call it Data Engineering Weekly Radio.

Data Engineering 52

Data Engineering Data Engineer Engineering SQL 52

Data Engineering Weekly

JUNE 25, 2023

Financial Times writes about evaluating the vectorization model for the following use cases. The blog narrates the essence of the asset model below. Debezium writes a blog about its exactly-once semantics support with Kafka and narrates the case of what will happen if a database connection breaks.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Lake 52

Data Engineering Weekly

JULY 2, 2023

link] Apache Doris: Building a Data Warehouse for Traditional Industry The blog narates the incremental update and the overall update strategy of a typical legacy datawarehouses. Similarly, the blog narrates how to use vector searching in Postgres using PGVector. Can't we use the vector feature in the existing databases?

Data Engineering 52

Data Engineering Data Engineer Engineering Pipeline-centric 52

Data Engineering Weekly

JUNE 8, 2023

On DEW #131, we selected the following article Ramon Marrero: DBT Model Contracts - Importance and Pitfalls dbt introduces model contract with 1.5 There were a few critics of the dbt model implementation, such as The False Promise of dbt Contracts. The author did an amazing job writing a balanced view of dbt model contract.

Architecture 57

Architecture SQL Data Pipeline Data Engineering 57

Data Engineering Weekly

JUNE 12, 2023

The recent frenzy around large language models (LLM) increases the dilemma around the open source vs. non-commercial use licensing model. The author highlights the recent trend of increasing non-commercial & restrictive licenses. Is there a bottleneck? Is the pipeline performing optimally?

Data Engineering 57

Data Engineering Data Engineer Engineering AWS 57

Data Engineering Weekly

SEPTEMBER 24, 2023

The blog talks about the limitations of rule engines and how LLM can enrich additional context to make the rule engine more effective. The blog specifically focuses on the following areas of LLM implementations. Your app and its documentation should be very clear about: The models or third-party AI providers you're using.

Data Engineering 67

Data Engineering Data Engineer Engineering Kafka 67

Data Engineering Weekly

OCTOBER 30, 2022

link] Iris Sun: Anything and Everything about Generative Models The generative AI is on everyone’s lips. I found the blog helpful in understanding the generative model’s historical development and the path forward. The current state of these systems is inherently passive systems.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Ingestion 52

Data Engineering Weekly

OCTOBER 1, 2023

link] Whatnot: Enhancing Search Using Large Language Models Search functionality undeniably plays a pivotal role in the user experience of e-commerce apps. link] Criteo: Recommender systems need a user model Criteo makes a strong case for recommender system problems as user preference rather than pattern recognition.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Data Engineering Weekly

FEBRUARY 18, 2024

link] Sponsored: Data modeling and exploration in Playground 2.0 See how recent product updates make it easier for developers and data engineers to debug data models, work in a BI-like UI, and fine-tune deployments for better query performance. We are so over the Big Data Era to Modern Data Stack. What are we stepping into?

Data Engineering 62

Data Engineering Data Engineer Engineering Data 62

Data Engineering Weekly

NOVEMBER 27, 2022

I have a long list of thoughts on this conversation, which might need a blog post on its own. link] Airbnb: How AI Text Generation Models Are Reshaping Customer Support at Airbnb Airbnb takes customer service from a simple customer service response template to the AI text generation model.

Data Engineering 52

Data Engineering Data Engineer Engineering SQL 52

Knowledge Hut

JANUARY 24, 2024

This blog offers an exclusive glimpse into the daily rituals, challenges, and moments of triumph that punctuate the professional journey of a data scientist. So, let's embark on this adventure, where each day's narrative unfolds as a unique chapter in the grand story of decoding the data maze. Who is a Data Scientist?

Database-centric 52

Database-centric Data Science Machine Learning Datasets 52

FreshBI

OCTOBER 30, 2023

Imagine the Flow: Imagine a 3-D model of a river, portrayed in a Discovery Channel special. Automated data models such as Power BI offer continuous insights without the manual hassle. Building the data connections and mapping all this information into one centralized model is essential. Ready to power up your business ?

Consulting 52

Consulting BI Business Intelligence Designing 52

Netflix Tech

NOVEMBER 13, 2023

In this blog post, we will introduce speech and music detection as an enabling technology for a variety of audio applications in Film & TV, as well as introduce our speech and music activity detection (SMAD) system which we recently published as a journal article in EURASIP Journal on Audio, Speech, and Music Processing.

Datasets 89

Datasets Metadata Algorithm Architecture 89

Edureka

DECEMBER 27, 2023

The blog “LLM vs Generative AI” focuses on the differences and uses of two modern technologies. The blog explores how emerging technologies improve user experiences and operational efficiency in industries such as healthcare, education, and entertainment. What are large language models (LLMs)?

Entertainment 40

Entertainment Education Healthcare Medical 40

Knowledge Hut

NOVEMBER 17, 2023

This blog intends to give expert perspectives on the essential AI skills required for success in the field. Mathematics and Statistics: A solid understanding of mathematics and statistics is crucial to develop and refine AI models. Model Deployment: Knowing how to deploy AI models is crucial.

Deep Learning 52

Deep Learning Data Science Machine Learning Certification 52

Data Engineering Weekly

APRIL 23, 2023

. 💪🏽 Click the link to connect 🖇️ [link] 🔗 [link] ✍🏼Replit: How to train your own Large Language Models Replit writes about building your own Large Language Model using Databricks, Hugging Face, and MosaicML.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Data Engineering Weekly

MARCH 5, 2023

The blog narrates how they overcome the challenge with the combination of Cassandra & ElasticSearch [link] Sponsored: [New Guide] The Ultimate Guide to Data Mesh Architecture If implementing data mesh is high on your list of priorities, you’re not alone. link] The blog highlights six key principles of the value creation of data.

Data Engineering 52

Data Engineering Data Engineer Engineering Datasets 52

RandomTrees

MARCH 27, 2024

The effect of generative AI stretches out across assorted areas, upsetting conventional work processes and reshaping business models. By combining assorted information inputs, generative AI can independently create unique results, going from artwork and plans to virtual environments and narratives.

IT 52

IT Medical Entertainment Healthcare 52

Data Engineering Weekly

OCTOBER 23, 2022

link] Kumu: ML Engineering at Kumu - Turning Models into Products Kumu writes about its ML platform journey with the end-to-end Machine Learning Platform lifecycle. Shopify writes an exciting blog narrating how to structure the data team to maximize the influence in an org. Is all-in-one data stacks the future?

Data Engineering 52

Data Engineering Data Engineer Engineering Deep Learning 52

Cloudera

NOVEMBER 15, 2021

The word “data” is ubiquitous in narratives of the modern world. This blog discusses quantifications, types, and implications of data. In other words, structured data has a pre-defined data model , whereas unstructured data doesn’t. . Everyone wants to do the model work, not the data work” starts the title of this paper.

Unstructured Data 85

Unstructured Data Pipeline-centric Database-centric Entertainment 85

Data Engineering Weekly

NOVEMBER 6, 2022

I plan to write a series of blogs on Schemata and Data Contract in the coming weeks. Martin kindly stepped in for me to give the update for my promised blog posts. But How well do we know the Machine Learning models? The blog narrates how Azure InterpretML service can help to understand the ML models' predictions better.

Data Engineering 40

Data Engineering Data Engineer Engineering Machine Learning 40

Knowledge Hut

JANUARY 22, 2024

Students learn to build, train and optimize models using Python, R, TensorFlow and other libraries. The curriculum focuses on real-world applications of AI and ML for tasks like predictive modeling, image recognition, recommender systems etc.

Data Science 52

Data Science Certification Education Data Lake 52

Cloudera

SEPTEMBER 20, 2022

This blog is intended to serve as an ethics sheet for the task of AI-assisted comic book art generation, inspired by “ Ethics Sheets for AI Tasks.” AI-assisted comic book art generation is a task I proposed in a blog post I authored on behalf of my employer, Cloudera. Introduction. Access — who can use it?

Deep Learning 84

Deep Learning High Quality Data Telecommunication Systems 84

Knowledge Hut

DECEMBER 6, 2023

Voice technologies have played a very important role in making websites and blogs more accessible to people with disabilities and will continue to do so in the coming years. Let’s explore how we can add voice to a blog or a webpage. How can you make a web page talk or add voice to your blog? utterance.volume = 0.5

Telecommunication 52

Telecommunication Education Coding Technology 52

AltexSoft

JANUARY 18, 2023

Dubbed GPT-3 and developed by OpenAI in San Francisco, it was the latest and strongest of its kind — a “large language model” capable of producing fluent text after having ingested billions of words from books, articles, and websites. With these advances, the concept of language modeling entered a whole new era. Pretty cool, right?

Datasets 64

Datasets Architecture Deep Learning SQL 64

U-Next

AUGUST 18, 2022

This blog encompasses their fundamental duties, Data Science specializations, or how Data Scientists work in various sectors. Their work frequently involves more sophisticated Data Science approaches, such as pattern recognition, statistical modeling, and predictive modeling. Can a Data Scientist work from home ?

Unstructured Data 40

Unstructured Data Medical Data Science Business Intelligence 40

U-Next

OCTOBER 20, 2022

The topic of client acquisition is covered in this blog. This response hierarchy model represents the route that leads someone from learning about your product or service to using it. . It investigates a business’s offer to its target customers to persuade them to purchase and continue purchasing.

Management 40

Management Retail Utilities Media 40

Netflix Tech

JUNE 20, 2023

Avneesh Saluja , Andy Yao , Hossein Taghavi Introduction When watching a movie or an episode of a TV show, we experience a cohesive narrative that unfolds before us, often without giving much thought to the underlying structure that makes it all possible. sluglines). sluglines). Figure 1: a scene consists of a sequence of shots.

Machine Learning 101

Machine Learning Datasets Algorithm Architecture 101

Workfall

DECEMBER 26, 2023

Reading Time: 14 minutes In the evolving landscape of AI-driven innovation, crafting compelling narratives has reached new heights with the power of Generative AI, and Amazon SageMaker stands as a pivotal platform for realizing this potential. Explore different hubs, and for this hands-on, choose the GPT-2 model from the HuggingFace hub.

AWS 76

AWS Coding Accessible Accessibility 76

Data Engineering Weekly

SEPTEMBER 11, 2023

Niels Claeys: Head-to-head comparison of 3 dbt SQL engines The blog post compares benchmarks for the popular dbt SQL engines Trino, DuckDB, and Spark. link] Faith Lierheimer: An appropriately unhinged deep dive into Kimball's dimensional modeling primer An interesting article about Kimball’s dimensional modeling.

Data Engineering 62

Data Engineering Data Engineer Engineering Kafka 62

Data Engineering Weekly

MARCH 31, 2024

The blog further emphasizes its increased investment in Data Mesh and clean data. The blog is an excellent overview of all the improvements made to PySpark in 2023. The blog narrates how Uber utilizes the existing infrastructure to support various AI/ML use cases. link] Jet Brains: Polars vs. Pandas: What’s the Difference?

Data Engineering 62

Data Engineering Data Engineer Engineering Scala 62

Data Engineering Weekly

NOVEMBER 20, 2022

Data Cards include the following: upstream sources Data collection and annotation methods Training and evaluation methods Intended use of the dataset Decisions affecting model performance The data cards approach is fascinating, especially As machine learning (ML) research moves toward large-scale models capable of numerous downstream tasks.

Data Engineering 52

Data Engineering Data Engineer Engineering Datasets 52

Data Engineering Weekly

MAY 21, 2023

Ramon Marrero: DBT Model Contracts - Importance and Pitfalls dbt introduces model contract with 1.5 There were a few critics of the dbt model implementation, such as The False Promise of dbt Contracts. As a model owner, if I change the columns or types in the SQL, it's usually intentional. -

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Data Engineering Weekly

DECEMBER 18, 2022

The author writes an exciting blog, Modern data stack in a Box!! link] Percona: JSON and Relational Databases – Part One Whether we like it or not, most data engineering and modeling challenges will be handling semi-structured data in the coming years. The Percona blog walkthrough JSON support in the relational databases.

Data Engineering 52

Data Engineering Data Engineer Engineering Relational Database 52

Netflix Tech

AUGUST 17, 2023

to pull moments from video content that authentically represent the title’s narrative themes, main characters, and visual characteristics. For Visual Search We found that the model was influenced by the text present in the image. For Prominent Characters We found that our current algorithm model did not handle animated faces well.

Media 67

Media Algorithm Architecture Building 67

Edureka

JANUARY 18, 2024

They manage intricate datasets, create forecasting models, and examine consumer behavior to deliver tailored experiences. Basic Foundations Mathematics In data science, mathematics is essential because it allows understanding of algorithms, model optimization, and data insights.

Data Science 40

Data Science Deep Learning NoSQL Machine Learning 40