Trending Articles

article thumbnail

Min rate limits for Apache Kafka

Waitingforcode

I bet you know it already. You can limit the max throughput for Apache Spark Structured Streaming jobs for popular data sources such as Apache Kafka, Delta Lake, or raw files. Have you known that you can also control the lower limit, at least for Apache Kafka?

Kafka 130
article thumbnail

New SQL Practice Problems

Confessions of a Data Guy

New SQL Practice Problems I’m trying something new. I get a lot of questions from folks about getting into the Data Engineering space, how to get better, grow, learn, etc. So I came up with a solution. SQL Practice Problems. Some moons ago I wrote a Data Engineering Practice repo on GitHub for free, and some 1.2K stars later […] The post New SQL Practice Problems appeared first on Confessions of a Data Guy.

SQL 100
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Data News — Week 24.07

Christophe Blefari

Italy Sora ( credits ) Hey you, time for the Data News. Because I did not send the news last week you will get articles from the 2 last weeks. Last few days have been heavily packed with AI News as well. Disclaimer, the 2 events below will be in French. Before jumping to the news there are a few events I want to write about. Next Wednesday I will participate to a Data Night Talk a open discussion about AI & data engineering with other content creators.

Food 130
article thumbnail

Unapologetically Technical Episode 9 – Gunnar Morling

Jesse Anderson

This week on Unapologetically Technical, I had the wonderful pleasure of interviewing Gunnar Morling, the creator of the Billion Row Challenge and Senior Staff Software Engineer at Decodable. In this episode, we talk about why it is so important to stay in a position long enough to gain experience and see the success or failure of decisions. He also shares his experiences at RedHat and working on Debezium.

article thumbnail

Guide to Business Writing

Everything you need to know about better business writing in one place. This is a complete guide to business writing — from a clear business writing definition to tips on how to hone your business writing skills.

article thumbnail

A Roadmap For Your Data Career

KDnuggets

As you design your career in data, you’ve got to avoid getting stuck in your comfort zone or allowing your manager or current situation to determine your path.

Data 122

More Trending

article thumbnail

Is the modern data stack disappearing?

Christophe Blefari

No. This question generated a lot of content last week, and a lot of words were written. I wanted to keep my answer short so as not to burden you with a few thousand more words to read. Modern data stack has been coined by US companies and VCs—mainly Fivetran / dbt Labs—as a word to quickly emphasis a way to build data stack in the cloud related to ELT.

Data 100
article thumbnail

The Abstraction Problem – A Great Evil

Confessions of a Data Guy

There is a great evil Spirit that is haunting the streets of code in the land of programmers. It’s a Spirit of obfuscation and twisting things into what they are not. The Spirit wanders around on the loose looking for someone, and it finds ready victims among the ranks of new programmers and the innocent […] The post The Abstraction Problem – A Great Evil appeared first on Confessions of a Data Guy.

Coding 114
article thumbnail

Access Over 181,000 USGS Historical Topographic Maps

ArcGIS

We recently updated our online USGS historical topographic map collection with over 1,745 new maps for a new total of over 181,000 maps.

article thumbnail

Navigating the Data Revolution: Exploring the Booming Trends in Data Science and Machine Learning

KDnuggets

Dive into transformative trends in data science, encompassing AI-powered automation, NLP, ethical considerations, decentralized computing, and interdisciplinary collaboration.

article thumbnail

How to Build an Experimentation Culture for Data-Driven Product Development

Speaker: Margaret-Ann Seger, Head of Product, Statsig

Experimentation is often seen as an aspirational practice, especially at smaller, fast-moving companies who are strapped for time and resources. So, how can you get your team making decisions in a more data-driven way while continuing to remain lean and maintaining ship velocity? In this webinar, Margaret-Ann Seger, Head of Product at Statsig, will teach you how to build an experimentation culture from the ground-up, graduating from just getting started with data-driven development to operating

article thumbnail

8 Tips for Managing Stakeholder Expectations

Knowledge Hut

Why Stakeholder Management? One of the most critical aspects of project management is doing what’s necessary to develop and control relationships with all individuals that the project impacts. In this article, you will learn techniques for identifying stakeholders, analyzing their influence on the project, and developing strategies to communicate, set boundaries, and manage competing expectations.

article thumbnail

Snowflake’s Data Classification Lets You Identify and Tag Sensitive Data Directly in Snowsight

Snowflake

At Snowflake, we believe in empowering our customers to harness the full potential of their data while maintaining robust compliance standards and safeguarding data privacy. We recognize the critical importance of quickly identifying and safeguarding sensitive data objects, and we consistently strive to provide solutions that help achieve these goals — from advancements such as classification and tag-based policies to the intuitive Data Governance UI.

Data 79
article thumbnail

Data Products, Data Contracts, and Change Data Capture

Confluent

Discover how to build resilient data pipelines with Confluent Data Portal. Learn essential strategies for isolating upstream systems and empowering downstream consumers.

article thumbnail

ArcGIS Pro 3.3 Moves to.NET 8

ArcGIS

ArcGIS Pro 3.3 is planned to be available in May 2024. Install.NET 8 before attempting to install ArcGIS Pro 3.3 for the best user experience!

64
article thumbnail

Reimagined: Building Products with Generative AI

“Reimagined: Building Products with Generative AI” is an extensive guide for integrating generative AI into product strategy and careers featuring over 150 real-world examples, 30 case studies, and 20+ frameworks, and endorsed by over 20 leading AI and product executives, inventors, entrepreneurs, and researchers.

article thumbnail

Master The Art Of Command Line With This GitHub Repository

KDnuggets

Whether you are a beginner or an experienced user, this guide is perfect for familiarizing yourself with basic and advanced command line tools.

article thumbnail

Advantages of Agile Testing Methodology

Knowledge Hut

What is Agile Testing? As the name implies, agile course projects are executed very quickly and with flexibility. Agile methods involve tasks executed in short iterations or sprints. Agile Testing is also iterative and takes place after each sprint, rather than towards the end of the project. Testing courses iteratively helps to validate the client requirements and adapt to changing conditions in a better manner.

Project 97
article thumbnail

Simplify Application Development With Hybrid Tables

Snowflake

We previously announced Snowflake’s Unistore workload , which continues Snowflake’s legacy of breaking down data silos by uniting transactional and analytical data in a consistent and governed platform. Today, we are pleased to announce that Hybrid Tables — the core feature powering Unistore — is in public preview in select AWS regions. Hybrid Tables is a new table type that enables transactional use cases within Snowflake with fast, high-concurrency point operations.

article thumbnail

Creative Spark: The Making of Robinhood’s Gold Campaign

Robinhood

Robinhood was founded on a simple idea: that our financial markets should be accessible to all. With customers at the heart of our decisions, Robinhood is lowering barriers and providing greater access to financial information and investing. Together, we are building products and services that help create a financial system everyone can participate in. … Introducing the creative campaign for Robinhood Gold!

Banking 59
article thumbnail

Entity Resolution Checklist: What to Consider When Evaluating Options

Are you trying to decide which entity resolution capabilities you need? It can be confusing to determine which features are most important for your project. And sometimes key features are overlooked. Get the Entity Resolution Evaluation Checklist to make sure you’ve thought of everything to make your project a success! The list was created by Senzing’s team of leading entity resolution experts, based on their real-world experience.

article thumbnail

Data Engineering Weekly #159

Data Engineering Weekly

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Visit rudderstack.com to learn more. Editor’s Note: DEWCon Next? Aswin and I started DEWCon as a fun experiment. Midway through organizing the conference, fear washed over us.

article thumbnail

Python in Finance: Real Time Data Streaming within Jupyter Notebook

KDnuggets

Learn a modern approach to stream real-time data in Jupyter Notebook. This guide covers dynamic visualizations, a Python for quant finance use case, and Bollinger Bands analysis with live data.

Finance 80
article thumbnail

Strategic Thinking For Business Planning

Knowledge Hut

An organization without strategic planning is like a ship without the captain. Developing strategic thinking into a plan and executing is inevitable for the businesses to compete in the global marketing environment. Strategic thinking, planning, and execution encompass all the concerns pertaining to ambitious goals, prevailing conditions, limitations, challenges, competence, changing customers’ preferences etc.

BI 98
article thumbnail

Top 3 Data + AI Predictions for Retail and Consumer Goods in 2024

Snowflake

Nearly every facet of society has felt the impact of AI since it burst into the mainstream in late 2022 with the public launch of ChatGPT. In 2024, the retail and consumer goods industry is expected to experience massive upheaval due to the proliferation of generative AI (gen AI) tools as well as changes in customer engagement and the general manner in which products are now sold.

Retail 69
article thumbnail

The Big Payoff of Application Analytics

Outdated or absent analytics won’t cut it in today’s data-driven applications – not for your end users, your development team, or your business. That’s what drove the five companies in this e-book to change their approach to analytics. Download this e-book to learn about the unique problems each company faced and how they achieved huge returns beyond expectation by embedding analytics into applications.

article thumbnail

Announcing the General Availability of Azure Private Link and Azure Storage firewall support for Databricks SQL Serverless

databricks

We are excited to announce the upcoming general availability of Azure Private Link support for Databricks SQL (DBSQL) Serverless, planned in April 2024.

SQL 58
article thumbnail

The Power of Geospatial Intelligence and Similarity Analysis for Data Mapping

Towards Data Science

Strategically enhancing address mapping during data integration using geocoding and string matching Many individuals in the big data industry may encounter the following scenario: Is the acronym “TIL” equivalent to the phrase “Today I learned” when extracting these two entries from distinct systems? Your program might get confused too when records come in with different names even though it means the same thing.

Food 62
article thumbnail

Introduction to Memory Profiling in Python

KDnuggets

So where did all the memory go? To figure out, learn how to profile your Python code for memory usage using the memory-profiler package.

Python 98
article thumbnail

Essential Hacks To Become A CBAP Certified Professional

Knowledge Hut

Certified Business Analysis Professional ( CBAP ®) offered by the International Institute of Business Analysis (IIBA®) is the most prestigious professional certification that a business analyst can do. In order to tackle and conquer the exam you need meticulous planning, diligent and honest preparation and confidence in facing the exam. This article intends to provide guidance on passing the examination from the time of filling up the application form.

article thumbnail

The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data and AI

Speaker: Aindra Misra, Sr. Staff Product Manager of Data & AI at BILL (Previously PM Lead at Twitter/X)

Embark on a transformation journey into the heart of the data ecosystem! This webinar is your gateway to a deeper comprehension of the foundations that drive the data industry and will equip you with the knowledge needed to navigate the evolving landscape. Delve into the diverse use cases where data analytics plays a pivotal role. We’ll explore how these applications are transforming with the introduction of Gen AI, and discuss the anticipated use cases for 2024 and beyond.

article thumbnail

Delivering Telecom Sustainability Targets Using Autonomous Networks

Snowflake

As the world grapples with the escalating climate crisis, many industries are re-examining their operations to identify and implement sustainable practices. The telecommunications industry is no exception. Telecom companies face growing pressure from consumers, investors and regulators to reduce their carbon footprint and achieve net-zero emissions.

article thumbnail

The Art of Data Buck-Passing 101: Mastering the Blame Game in Data and Analytic Teams

DataKitchen

The Art of Data Buck-Passing 101: Mastering the Blame Game in Data and Analytic Teams Welcome, dear readers, to the hallowed halls of Data Buck-Passing University, where the motto is “ Per Alios Culpa Transfertur ” (Blame is Transferred to Others). In the world of data and analytics, one skill stands timeless and universal: the art of blaming someone else when things go sideways.

Data 64
article thumbnail

Generative AI Tutorial: A Complete Guide for Generative AI

Edureka

In today’s fast-changing digital world, Generative AI is a key player, changing the way we make, design, and use digital stuff. If you’re looking to boost your creativity and solve problems in new ways but don’t know where to start, this Generative AI tutorial is for you. It offers a simple step-by-step guide on how to write AI prompts that bring out the best in text and image AI tools.