Start Data Engineering
MAY 11, 2022
1. Introduction 2. Objective 3. Design 4. Setup 4.1 Prerequisite 4.2 AWS Infrastructure costs 4.3 Data lake structure 5. Code walkthrough 5.1 Loading user purchase data into the data warehouse 5.2 Loading classified movie review data into the data warehouse 5.3 Generating user behavior metric 5.4. Checking results 6. Tear down infra 7. Design considerations 8.
KDnuggets
MAY 13, 2022
This article is the first in a series of articles looking at the different aspects of k-means clustering, beginning with a discussion on centroid initialization.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Confluent
MAY 12, 2022
Internet connectivity is something we sometimes take for granted. For many, most places we visit, work, or reside have some form of connectivity whether it be cellular, Wi-Fi, fiber, etc. […].
Data Engineering Podcast
MAY 8, 2022
Summary Many of the events, ideas, and objects that we try to represent through data have a high degree of connectivity in the real world. These connections are best represented and analyzed as graphs to provide efficient and accurate analysis of their relationships. TigerGraph is a leading database that offers a highly scalable and performant native graph engine for powering graph analytics and machine learning.
Advertisement
Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.
AltexSoft
MAY 12, 2022
We live in the world of sounds: Pleasant and annoying, low and high, quiet and loud, they impact our mood and our decisions. Our brains are constantly processing sounds to give us important information about our environment. But acoustic signals can tell us even more if analyze them using modern technologies. Today, we have AI and machine learning to extract insights, inaudible to human beings, from speech, voices, snoring, music, industrial and traffic noise, and other types of acoustic signals
KDnuggets
MAY 12, 2022
By implementing the different NLP techniques into the production processes, compliance departments can maintain detailed checks and keep up with regulator demands.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Cloudera
MAY 9, 2022
Tuning Hive on Tez queries can never be done in a one-size-fits-all approach. The performance on queries depends on the size of the data, file types, query design, and query patterns. During performance testing, evaluate and validate configuration parameters and any SQL modifications. It is advisable to make one change at a time during performance testing of the workload, and would be best to assess the impact of tuning changes in your development and QA environments before using them in product
Teradata
MAY 12, 2022
The next generation of customers expects newer technologies & advanced self-service capabilities as the airline business becomes more competitive. How can airlines meet these expectations?
KDnuggets
MAY 10, 2022
This is a list of FREE data science resources and notes that are available online, some of which are provided by universities.
Rockset
MAY 12, 2022
This is the third post in a series by Rockset's CTO Dhruba Borthakur on Designing the Next Generation of Data Systems for Real-Time Analytics. We'll be publishing more posts in the series in the near future, so subscribe to our blog so you don't miss them! Posts published so far in the series: Why Mutability Is Essential for Real-Time Data Analytics Handling Out-of-Order Data in Real-Time Analytics Applications Handling Bursty Traffic in Real-Time Analytics Applications SQL and Complex Queries A
Speaker: Scott Sehlhorst
We know we want to create products which our customers find to be valuable. Whether we label it as customer-centric or product-led depends on how long we've been doing product management. There are three challenges we face when doing this. The obvious challenge is figuring out what our users need; the non-obvious challenges are in creating a shared understanding of those needs and in sensing if what we're doing is meeting those needs.
Monte Carlo
MAY 11, 2022
If you’ve been a data analyst, BI analyst, or general business user of dashboards and reports, you’ve probably asked these questions (and more) before: What’s the most reliable field to use? When was the last time this table was updated? Should there be this many null entries in this column? Who can I reach out to figure out if this data is expected?
Palantir
MAY 11, 2022
(Scroll down for English translation below) Avec l’adoption des accords de Paris en 2016, les institutions du secteur public et privé ont considérablement renforcé leurs ambitions en matière de décarbonation. Plus particulièrement, la capacité des organisations à s’adapter et améliorer leur prise de décision va devenir un élément clé de différenciation et de compétitivité.
KDnuggets
MAY 9, 2022
Read this overview of 12 important machine learning concepts, presented in a no frills, straightforward definition style.
dbt Developer Hub
MAY 9, 2022
❓ Who is this for: This is for advanced users of dbt Cloud that are interested in expanding their knowledge of the dbt API via an interactive Postman Collection. We only suggest diving into this once you have a strong knowledge of dbt + dbt Cloud. You have a couple of options to review the collection: get a live version of the collection via. check out the collection documentation to learn how to use it.
Speaker: Timothy Chan, PhD., Head of Data Science
Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.
Data Engineering Podcast
MAY 8, 2022
Summary Dan Delorey helped to build the core technologies of Google’s cloud data services for many years before embarking on his latest adventure as the VP of Data at SoFi. From being an early engineer on the Dremel project, to helping launch and manage BigQuery, on to helping enterprises adopt Google’s data products he learned all of the critical details of how to run services used by data platform teams.
Cloudera
MAY 13, 2022
Cloudera Data Platform (CDP) unifies the technologies from Cloudera Enterprise Data Hub (CDH) and Hortonworks Data Platform (HDP). A few functionalities that existed in the legacy platforms (HDP and CDH) are substituted by other alternatives based on a detailed and careful analysis. CDH users would have used Fair Scheduler (FS), and HDP users would have used Capacity Scheduler (CS).
KDnuggets
MAY 10, 2022
While it is true that Machine Learning today isn’t ready for prime time in many business cases that revolve around Document Analysis, there are indeed scenarios where a pure ML approach can be considered.
dbt Developer Hub
MAY 10, 2022
We’ve all been there: In a user signup form, user A typed in their name as Kira Furuichi , user B typed it in as john blust , and user C wrote DAvid KrevitT (what’s up with that, David??) Your backend application engineers are adamant customer emails are in all caps All of your event tracking names are lowercase In the real world of human imperfection, opinions, and error, string values are likely to take inconsistent capitalization across different data sources (or even within the same data sou
Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage
Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.
Rockset
MAY 10, 2022
DynamoDB is a popular NoSQL database available in AWS. It is a managed service with minimal setup and pay-as-you-go costing. Developers can quickly create databases that store complex objects with flexible schemas that can mutate over time. DynamoDB is resilient and scalable due to the use of sharding techniques. This seamless, horizontal scaling is a huge advantage that allows developers to move from a proof of concept into a productionized service very quickly.
KDnuggets
MAY 11, 2022
The objective of data mesh is to establish coherence between data coming from different domains across an enterprise. The domains are handled autonomously to eliminate the challenges of data availability and accessibility for cross-functional teams.
KDnuggets
MAY 12, 2022
Learn about the free and easy-to-deploy hosting platform for your machine learning projects.
KDnuggets
MAY 11, 2022
SQL Notes for Professionals: The Free eBook Review; How To Structure a Data Science Project: A Step-by-Step Guide; Everything You Need to Know About Tensors; Free University Data Science Resources; Image Classification with Convolutional Neural Networks (CNNs).
Advertisement
Are you trying to decide which entity resolution capabilities you need? It can be confusing to determine which features are most important for your project. And sometimes key features are overlooked. Get the Entity Resolution Evaluation Checklist to make sure you’ve thought of everything to make your project a success! The list was created by Senzing’s team of leading entity resolution experts, based on their real-world experience.
KDnuggets
MAY 9, 2022
Turn your dull Jupyter notebooks into interactive web apps by adding a YAML header and sharing it with your friends and colleagues. You can also use Mercury to create your data science portfolio, which consists of a resume and projects.
KDnuggets
MAY 10, 2022
Good planning and preparation will not only improve productivity, but it will help avoid potential pitfalls and roadblocks that could be encountered during project execution.
KDnuggets
MAY 11, 2022
Save time and effort with this guide, which will show you how to do data join operations in Tableau.
KDnuggets
MAY 9, 2022
Check out this list of free resources, courses, and more to help you become a Data Scientist for free.
Speaker: Anne Steiner and David Laribee
As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.
KDnuggets
MAY 11, 2022
If you aren't familiar with Kaggle, you should be. Hear why from two expert Kagglers in this article.
KDnuggets
MAY 13, 2022
Discover "compound sparsification" and how to apply it to BERT models for 10x compression and GPU-level latency on commodity CPUs.
KDnuggets
MAY 13, 2022
Predict the performance of your model - before the ground truth is available.
KDnuggets
MAY 13, 2022
In this article, we will build a beginner-friendly machine learning model using TensorFlow.
Speaker: Margaret-Ann Seger, Head of Product, Statsig
Experimentation is often seen as an aspirational practice, especially at smaller, fast-moving companies who are strapped for time and resources. So, how can you get your team making decisions in a more data-driven way while continuing to remain lean and maintaining ship velocity? In this webinar, Margaret-Ann Seger, Head of Product at Statsig, will teach you how to build an experimentation culture from the ground-up, graduating from just getting started with data-driven development to operating
Expert insights. Personalized for you.
246 
Let's personalize your content