Sat.Oct 21, 2023 - Fri.Oct 27, 2023

article thumbnail

Defining A Strategy For Your Data Products

Data Engineering Podcast

Summary The primary application of data has moved beyond analytics. With the broader audience comes the need to present data in a more approachable format. This has led to the broad adoption of data products being the delivery mechanism for information. In this episode Ranjith Raghunath shares his thoughts on how to build a strategy for the development, delivery, and evolution of data products.

BI 162
article thumbnail

Code Review on Printed Paper: an Excerpt from the Twitoons Comic Book

The Pragmatic Engineer

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover two out of seven topics from today’s full issue on The Man Behind the Big Tech Comics. To get full issues twice a week, subscribe here.

Coding 183
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

6 Steps to Avoid Messy Data in Your Warehouse

Start Data Engineering

1. Introduction 2. Six Steps for a Clean Data Warehouse 2.1. Understand the business 2.2. Make data easy to use with the appropriate data model 2.3. Good input data is necessary for a good data warehouse 2.4. Define Source of Truth (SOT) and trace its usage 2.5. Keep stakeholders in the loop for a more significant impact 2.6. Watch out for org-level red flags ?

article thumbnail

What's new in Apache Spark 3.5.0 - Structured Streaming

Waitingforcode

It's time to start the series covering Apache Spark 3.5.0 features. As the first topic I'm going to cover Structured Streaming which has got a lot of RocksDB improvements and some major API changes.

IT 130
article thumbnail

15 Modern Use Cases for Enterprise Business Intelligence

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

article thumbnail

Drag, Drop, Analyze: The Rise of No-Code Data Science

KDnuggets

No-code or low-code functionalities in data science have gained significant traction in recent years. These solutions are well-proven and matured, and they make data science more accessible to a wider range of people.

article thumbnail

Snowflake To Acquire Ponder, Boosting Python Capabilities In the Data Cloud

Snowflake

Python’s popularity has more than doubled in the past decade¹ and it is quickly becoming the preferred language for development across machine learning, application development, pipelines, and more. One of our goals at Snowflake is to ensure we continue to deliver a best-in-class platform for Python developers. Snowflake customers are already harnessing the power of Python through Snowpark , a set of runtimes and libraries that securely deploy and process non-SQL code directly in Snowflake.

Python 141

More Trending

article thumbnail

Automating dead code cleanup

Engineering at Meta

Meta’s Systematic Code and Asset Removal Framework (SCARF) has a subsystem for identifying and removing dead code. SCARF combines static and dynamic analysis of programs to detect dead code from both a business and programming language perspective. SCARF automatically creates change requests that delete the dead code identified from the program analysis, minimizing developer costs.

Coding 132
article thumbnail

5 Free Books to Master Machine Learning

KDnuggets

Machine Learning is one of the most exciting fields in computer science today. In this article, we will take a look at the five best yet free books to learn machine learning in 2023.

article thumbnail

Announcing Apache Flink 1.18

Confluent

Read updates and improvements in Apache Flink 1.18, including dynamic fine-grained rescaling via REST API, Java 17 support, and faster rescaling & batch performance improvements.

Java 125
article thumbnail

Introducing Predictive Optimization: Faster Queries, Cheaper Storage, No Sweat

databricks

Predictive Optimization intelligently optimizes your Lakehouse table data layouts for peak performance and cost-efficiency - without you needing to lift a finger.

Data 113
article thumbnail

Prepare Now: 2025s Must-Know Trends For Product And Data Leaders

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

article thumbnail

5 Things you didn’t know about Buck2

Engineering at Meta

Meta has a very large monorepo, with many different programming languages. To optimize build and performance, we developed our own build system called Buck , which was first open-sourced in 2013. Buck2 is the recently open-sourced successor. In our internal tests at Meta, we observed that Buck2 completed builds approximately 2x as fast as Buck1. Below are five interesting facts you might not have known about Buck2.

article thumbnail

The Top 5 Cloud Machine Learning Platforms & Tools

KDnuggets

What are the top 5 cloud machine learning platforms in the market today. Our list will help provide some vital insights into which platform might best cater to your specific machine learning needs. See what KDnuggets recommends.

article thumbnail

High resolution data updates to Living Atlas World Elevation Layers and Tools (October 2023)

ArcGIS

In October 2023, elevation layers have been updated with high-res datasets of France, New Zealand, USA, Italy along with global bathymetry.

Datasets 131
article thumbnail

ThoughtSpot announces our 2023 Partner Award winners

ThoughtSpot

To our entire partner ecosystem, I want to personally thank each of you for your incredible contributions over the past year. Our partners play a vital role in driving ThoughtSpot’s mission of becoming a more fact-driven world. Together, we help organizations leverage AI and natural language search to discover insights and make data-driven decisions for their businesses.

article thumbnail

Changing the Game with MES: Cut Costs, Drive Efficiency, & Achieve Sustainability Goals!

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

In an era where efficiency is king, are you leveraging the right tools to transform your manufacturing processes? A Manufacturing Execution System (MES) is critical for enhancing operational efficiency, reducing waste, and optimizing energy usage—key factors for improving your bottom line and lowering your carbon footprint. Join Nikhil Joshi, a manufacturing technology expert with 18+ years of hands-on experience, in this new webinar as he uncovers the secrets of MES and how to best utilize thes

article thumbnail

Learn How to Build Airtight Data Pipelines for your AI Initiatives

databricks

"I can't think of anything that's been more powerful since the desktop computer." — Michael Carbin, Associate Professor, MIT, and Founding Advisor, MosaicML A.

article thumbnail

KDnuggets News, October 27: 5 Free Books to Master Data Science • 7 Steps to Mastering LLMs

KDnuggets

This week on KDnuggets: Go from learning what large language models are to building and deploying LLM apps in 7 steps • Check this list of free books for learning Python, statistics, linear algebra, machine learning and deep learning • And much, much more!

article thumbnail

Top 10 Six Sigma Black Belt Project Examples & Ideas

Knowledge Hut

A certified Six Sigma Black Belt expert is a professional who knows and can explain and implement the Six Sigma principles and philosophies. These include tools and supportive systems. A Black Belt professional must have impeccable leadership skills and understand team dynamics. They work in a collaborative manner to assign team members and give them roles and responsibilities.

Project 98
article thumbnail

Kubernetes And Kernel Panics

Netflix Tech

How Netflix’s Container Platform Connects Linux Kernel Panics to Kubernetes Pods By Kyle Anderson With a recent effort to reduce customer (engineers, not end users) pain on our container platform Titus , I started investigating “orphaned” pods. There are pods that never got to finish and had to be garbage collected with no real satisfactory final status.

article thumbnail

Improving the Accuracy of Generative AI Systems: A Structured Approach

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

article thumbnail

Werner Gains Advanced Geospatial Capabilities with Snowflake and CARTO

Snowflake

Founded nearly 70 years ago, Werner Enterprises is a North American transportation and logistics leader that operates a fleet of almost 8,300 trucks and 30,000 trailers out of 16 terminals across the United States. The company generates a massive amount of data on the constantly changing, real-time location of each of its assets. Collecting and analyzing this geospatial data is vital for smart decision-making.

article thumbnail

Greening AI: 7 Strategies to Make Applications More Sustainable

KDnuggets

The article delves into a comprehensive methodology that sheds light on how to accurately estimate the carbon footprint associated with AI applications. It explains the environmental impact of AI, a crucial consideration in today's world.

IT 118
article thumbnail

Top 15 Software Engineer Projects 2023 [Source Code]

Knowledge Hut

In today's fast-paced technological environment, software engineers are continually seeking innovative projects to hone their skills and stay ahead of industry trends. Engaging in software engineering projects not only helps sharpen your programming abilities but also enhances your professional portfolio. To further amplify your skillset, consider enrolling in Programming training course to leverage online programming courses from expert trainers and grow with mentorship programs.

article thumbnail

Build AI-driven near-real-time operational analytics with Amazon Aurora zero-ETL integration with Amazon Redshift and ThoughtSpot

ThoughtSpot

Every business that analyzes their operational (or transactional) data needs to build a custom data pipeline involving several batch or streaming jobs to extract transactional data from relational databases , transform it, and load it into the data warehouse. In this post, we show how you can leverage Amazon Aurora zero-ETL integration with Amazon Redshift and ThoughtSpot for GenAI driven near real-time operational analytics.

article thumbnail

The Ultimate Guide To Data-Driven Construction: Optimize Projects, Reduce Risks, & Boost Innovation

Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network

In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.

article thumbnail

3 Questions Marketers Should Ask When Evaluating AI Solutions

Snowflake

AI. It’s on everyone’s mind—and marketers are no exception. You’ve likely heard about it from co-workers, vendors and peers, and if you had a nickel for every AI mention you heard … well, you get the point. With the release of ChatGPT late last year, OpenAI supercharged the conversation around large language models (LLMs), marking 2023 as “the year of AI.

article thumbnail

Generative AI: The First Draft, Not Final

KDnuggets

This article gives a high-level overview of how LLMs work and their attendant limitations with accessible explanations and anecdotes throughout the piece. We also present advice on how people can introduce them into their workflows.

article thumbnail

Top 20+ Cyber Security Projects for 2023 [With Source Code]

Knowledge Hut

Cybersecurity has become an integral component of every industry as the world advances technologically. In recent years, an increasing number of young professionals have shown interest in this field. If you are pursuing a course in this field, you should complete a project on cybersecurity as your area of competence. Beginners with theoretical knowledge should not undertake an impossible endeavor.

Coding 98
article thumbnail

How Providence Health Built a Model marketplace using Databricks?

databricks

Providence's MLOps Platform Providence is a healthcare organization with 120,000 caregivers serving over 50 hospitals and 1,000 clinics across seven states. Providence is.

article thumbnail

Business Intelligence 101: How To Make The Best Solution Decision For Your Organization

Speaker: Evelyn Chou

Choosing the right business intelligence (BI) platform can feel like navigating a maze of features, promises, and technical jargon. With so many options available, how can you ensure you’re making the right decision for your organization’s unique needs? 🤔 This webinar brings together expert insights to break down the complexities of BI solution vetting.

article thumbnail

Transforming Telco with Trusted AI Everywhere

Cloudera

The AI technologies of today—including not just large language models (LLMs) but also deep learning, reinforcement learning, and natural-language processing (NLP) tools—will equip telcos with powerful new automation and analytics capabilities. AI-powered automation is already driving significant margin growth by reducing costs. But to truly drive transformation telcos must ensure AI models are driven by accurate, high-quality, trusted data, and determine how to manage and govern massive volume

article thumbnail

Windows on Snapdragon Brings Hybrid AI to Apps at the Edge

KDnuggets

Let’s take a closer look at Hybrid AI, how you can take advantage of it, and how Snapdragon brings hybrid AI to apps at the edge.

IT 125
article thumbnail

SAFe Scrum Master Roles and Responsibilities

Knowledge Hut

With the steep upward trend in the adoption of agile practices across the IT industry, various frameworks have gained momentum. This has led to an appetite for exploring new ways of working and setting performance benchmarks. More and more organizations are looking for people who can help them effectively run in a new environment comprising frameworks based on Agile and its variants.

article thumbnail

Announcing GA of Predictive I/O for Updates: Faster DML Queries, Right Out of the Box

databricks

Announcing GA of Predictive I/O for Updates, which harnesses Photon and AI atop Deletion Vectors in order to significantly speed up MERGE, UPDATE and DELETE operations.

83
article thumbnail

Driving Responsible Innovation: How to Navigate AI Governance & Data Privacy

Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL

Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.