Towards Data Science
DECEMBER 16, 2024
Agentic AI, small data, and the search for value in the age of the unstructured datastack. Image credit: MonteCarlo According to industry experts, 2024 was destined to be a banner year for generative AI. Operational use cases were rising to the surface, technology was reducing barriers to entry, and general artificial intelligence was obviously right around thecorner.
Confluent
DECEMBER 19, 2024
Queue support in Apache Kafka 4.0, enabled by share groups, lets you accommodate traditional queue-type workloads through cooperative consumption.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Simon Späti
DECEMBER 20, 2024
What started as a straightforward implementation guide for a declarative data stack quickly evolved into something more fundamental. While attempting to build a system that could define an entire data stack through a single YAML file, I encountered architectural questions that challenged my initial assumptions: Should we generate production-ready code from templates or create a boilerplate repository with best-in-class tools?
Start Data Engineering
DECEMBER 18, 2024
1. Introduction 2. Ways to reuse seed data across multiple dbt projects 2.1. Code setup 2.1.1. Prerequisites 2.1.2. Setup project environment 2.2. Turn the source repo into a dbt package 2.2.1. Define package version in dbt_project.yml 2.2.2. Store your package for other dbt projects to reference 2.3. Use project dependencies (dbt enterprise only) 2.4.
Advertisement
With over 30 million monthly downloads, Apache Airflow is the tool of choice for programmatically authoring, scheduling, and monitoring data pipelines. Airflow enables you to define workflows as Python code, allowing for dynamic and scalable pipelines suitable to any use case from ETL/ELT to running ML/AI operations in production. This introductory tutorial provides a crash course for writing and deploying your first Airflow pipeline.
KDnuggets
DECEMBER 19, 2024
Using Docker for local development brings stability, flexibility, and ease of management of the environment. No matter what operating system you're using. Learn how to use Docker on Windows, Linux, and macOS to simplify your development setup, from creating your first container to managing complex environments with Docker Compose.
Confluent
DECEMBER 18, 2024
Real-time data streaming and messaging are essential for building scalable, resilient, event-driven microservices. Explore integrating the Micronaut framework with Confluent Cloud.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Netflix Tech
DECEMBER 19, 2024
David J. Berg * , David Casler ^, Romain Cledat * , Qian Huang * , Rui Lin * , Nissan Pow * , Nurcan Sonmez * , Shashank Srikanth * , Chaoying Wang * , Regina Wang * , Darin Yu * *: Model Development Team, Machine Learning Platform ^: Content Demand ModelingTeam A month ago at QConSF, we showcased how Netflix utilizes Metaflow to power a diverse set of ML and AI use cases , managing thousands of unique Metaflow flows.
KDnuggets
DECEMBER 16, 2024
In this article, we'll explore 15 Python one-liners that make string manipulation not just efficient but also fun.
Scott Logic
DECEMBER 14, 2024
Advent of Code (AoC) is an annual, christmas-themed, coding competition that has been running for the past years and is something that I participate in at times. This year, while ~~subjecting myself to~~ learning Rust, I decided to see how OpenAIs latest model faired at the challenge. I quickly knocked together a script, and to my astonishment, found that o1-mini gave correct answers to all but one part of the first six days.
databricks
DECEMBER 17, 2024
Large language models are improving rapidly; to date, this improvement has largely been measured via academic benchmarks. These benchmarks, such as MMLU and.
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Confluent
DECEMBER 20, 2024
Learn how predictive analytics, powered by generative AI and Confluent, transforms healthcare by improving outcomes, reducing costs, and enabling real-time decisions.
KDnuggets
DECEMBER 20, 2024
A simple guide for getting hooked to machine learning and building a successful career in the field.
Monte Carlo
DECEMBER 18, 2024
As we turn the corner into 2025, were excited to announce that for the 7th quarter in a row, Monte Carlo has been named G2s #1 Data Observability Platform, as well as #1 in the Data Quality category. This recognition never gets old because G2 bases their rankings on feedback and insights from real customers who work in these tools every day to add value to their business.
databricks
DECEMBER 17, 2024
Were excited to announce the Public Preview of Query Git integration as part of the new SQL Editor. Git support for queries.
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Cloudera
DECEMBER 19, 2024
AWS re:Invent is one of my favorite trade shows. It is one of the biggest technology conferences of the year and is an opportunity to have hundreds of conversations with customers and prospects, listen to their priorities and challenges, hopes, and give them a Cloudera tote bag or a pair of orange sunglasses. What follows is a collection of just a few things I learned and observed during my week in Las Vegas.
KDnuggets
DECEMBER 18, 2024
Writing Python code thats clean and easy to understand isnt just for experts — learn how to avoid common pitfalls and write like a pro from the start!
Engineering at Meta
DECEMBER 19, 2024
Were sharing details about Glean , Metas open source system for collecting, deriving and working with facts about source code. In this blog post well talk about why a system like Glean is important, explain the rationale for Gleans design, and run through some of the ways were using Glean to supercharge our developer tooling at Meta. In August 2021 we open-sourced our code indexing system Glean.
databricks
DECEMBER 19, 2024
Staying competitive in Major League Soccer (MLS) demands building and maintaining a strong squad through strategic roster planning and smart, effective navigation of.
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
Cloudera
DECEMBER 16, 2024
In the last year, weve seen the explosion of AI in the enterprise, leaving organizations to consider the infrastructure and processes for AI to successfullyand securelydeploy across an organization. As we head into 2025, its clear that next year will be just as exciting as past years. Here, Cloudera experts share their insights on what to expect in data and AI for the enterprise in 2025.
KDnuggets
DECEMBER 17, 2024
Check out this beginner's guide to cleaning and preparing data efficiently with Python.
Engineering at Meta
DECEMBER 18, 2024
Meta has been on a years-long undertaking to translate our entire Android codebase from Java to Kotlin. Today, despite having one of the largest Android codebases in the world, we’re well past the halfway point and still going. We’re sharing some of the tradeoffs we’ve made to support automating our transition to Kotlin, seemingly simple transformations that are surprisingly tricky, and how we’re collaborating with other companies to capture hundreds more corner cases.
databricks
DECEMBER 18, 2024
Czech savings bank esk spoitelna , a division of Austrias Erste Group , recently collaborated with AI solution builder DataSentics to explore the.
Advertisement
Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.
Cloudera
DECEMBER 17, 2024
Since 5G networks began rolling out commercially in 2019, telecom carriers have faced a wide range of new challenges: managing high-velocity workloads, reducing infrastructure costs, and adopting AI and automation. Because data management is a key variable for overcoming these challenges, carriers are turning to hybrid cloud solutions, which provide the flexibility and scalability needed to adapt to the evolving landscape 5G enables.
KDnuggets
DECEMBER 16, 2024
Ever wondered how to handle large data without slowing down your computer? Lets learn about Dask, a tool that helps you work with large data quickly.
Engineering at Meta
DECEMBER 18, 2024
How did the Threads iOS team maintain the app’s performance during its incredible growth? Here’s how Meta’s Threads team thinks about performance, including the key metrics we monitor to keep the app healthy. We’re also diving into some case studies that impact publish reliability and navigation latency. When Meta launched Threads in 2023, it became the fastest-growing app in history, gaining 100 million users in only five days.
databricks
DECEMBER 17, 2024
Book at meeting wtih Databricks at NRF 2025! As we approach January 2025, the retail industry is gearing up for another groundbreaking Retail's.
Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
Confluent
DECEMBER 20, 2024
Learn how to integrate Salesforce with Snowflake to streamline CRM data, improve analytics, and explore common use cases.
KDnuggets
DECEMBER 17, 2024
Learn how to enhance RAG models by combining text and visual inputs using Hugging Face Transformers.
Precisely
DECEMBER 16, 2024
Key Takeaways: Centralized visibility of data is key. Modern IT environments require comprehensive data for successful AIOps, that includes incorporating data from legacy systems like IBM i and IBM Z into ITOps platforms. Predictive of AIOps capabilities will revolutionize IT operations. The shift from reactive to proactive IT operations is driven by AI-powered analysis, automation and insights.
databricks
DECEMBER 19, 2024
Introduction Databricks has joined forces with the Virtue Foundation through Databricks for Good, a grassroots initiative providing pro bono professional services to drive.
Advertisement
Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.
Expert insights. Personalized for you.
69 
Let's personalize your content