Precisely

NOVEMBER 20, 2023

AI transforms how we interact with technology, make decisions, and solve complex problems. It has been at the heart of many innovations over the past two years, powering everything from the chatbots that enhance our customer experiences to the predictive analytics engines that help us make financial decisions. What defines a successful AI initiative, and how can your organization ensure that your investments and hard work deliver maximum value for your organization?

Raw Data 121

Raw Data Insurance Data Portfolio 121

The Pragmatic Engineer

NOVEMBER 21, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover thee out of nine topics from today’s subscriber-only issue: The Past and Future of Modern Backend Practices.

Engineering 325

Engineering Bytes Cloud Computing AWS 325

Start Data Engineering

NOVEMBER 14, 2023

1. Introduction 2. What is an Open Table Format (OTF) 3. Why use an Open Table Format (OTF) 3.0. Setup 3.1. Evolve data and partition schema without reprocessing 3.2. See previous point-in-time table state, aka time travel 3.3. Git like branches & tags for your tables 3.4. Handle multiple reads & writes concurrently 4. Conclusion 5. Further reading 6.

Data 322

Data 322

Analytics Vidhya

NOVEMBER 8, 2023

Introduction Imagine yourself in command of a sizable cargo ship sailing through hazardous waters. It is your responsibility to deliver precious cargo to its destination safely. Determine success by the precision of your charts, the equipment’s dependability, and your crew’s expertise. A single mistake, glitch, or slip-up could endanger the trip. In the data-driven world […] The post Monitoring Data Quality for Your Big Data Pipelines Made Easy appeared first on Analytics Vidhya.

Big Data 253

Big Data Data Pipeline Data IT 253

Speaker: Ben Epstein, Stealth Founder & CTO | Tony Karrer, Founder & CTO, Aggregage

When tasked with building a fundamentally new product line with deeper insights than previously achievable for a high-value client, Ben Epstein and his team faced a significant challenge: how to harness LLMs to produce consistent, high-accuracy outputs at scale. In this new session, Ben will share how he and his team engineered a system (based on proven software engineering approaches) that employs reproducible test variations (via temperature 0 and fixed seeds), and enables non-LLM evaluation m

Software Engineer

KDnuggets

NOVEMBER 23, 2023

Are you excited to learn about and build machine learning models? Start learning today with these free machine learning courses.

Machine Learning 158

Machine Learning Building 158

Jesse Anderson

NOVEMBER 30, 2023

Lately, I’ve been learning how to trade options. Although there’s data and programming involved in options trading, it isn’t as technical as data engineering or software engineering. However, it reflects the current state of learning, whether that’s data engineering or options trading. It gave me a look into learning a skill using videos. Each lesson I learned will directly apply to your learning or skill improvement.

Data Engineering 147

Data Engineering Data Engineer Algorithm Engineering 147

The Pragmatic Engineer

NOVEMBER 9, 2023

The below topic was sent out to full subscribers of The Pragmatic Engineer , three weeks ago, in The Pulse #66. I have received several messages from people asking if they can pay to “unlock” this information for others, given how vital it is for software engineers. It is vital, and so I’m sharing this with all readers, without a paywall.

Software Engineer 251

Software Engineer Software Engineering Engineering Coding 251

Confessions of a Data Guy

NOVEMBER 25, 2023

Ok. Get off your high horse. You are human just like the rest of us. Just like your ancient ancestors who were throwing rocks and sticks at each other a thousand years ago … you are looking for a leg up on the competition. Isn’t that the world we live in? At the end of […] The post How to be Better Than Everyone Else appeared first on Confessions of a Data Guy.

Data 147

Data Data Engineering Data Engineer Engineering 147

Analytics Vidhya

NOVEMBER 26, 2023

In this Leading with Data episode, explore the analytics landscape with Dr. Swati Jain, a seasoned leader boasting over two decades of experience. From her unforeseen foray into analytics to steering EXL Analytics’ India business, Dr. Jain imparts invaluable insights into the ever-evolving world of data science. Read on to know more about her career, […] The post Unlocking the Power of Analytics with Dr.

Data Science 221

Data Science Data Analysis Data Data Engineering 221

KDnuggets

NOVEMBER 24, 2023

Data scientists love it! See how ChatGPT creates jaw-dropping data viz with just a few words - it's almost unfair how easy it is.

Data 158

Data IT Data Science 158

Advertisement

Apache Airflow® is the open-source standard to manage workflows as code. It is a versatile tool used in companies across the world from agile startups to tech giants to flagship enterprises across all industries. Due to its widespread adoption, Airflow knowledge is paramount to success in the field of data engineering.

Datasets

Jesse Anderson

NOVEMBER 14, 2023

A Guest Post by Ole Olesen-Bagneux In this blog post I would like to describe a new data team, that I call ‘the data discovery team’. It’s a team that connects naturally into the constellation of the three data teams Operations team Data engineering team Data Science team as described in Jesse Anderson’s book Data Teams (2020) Before I explain what the data discovery team should do, it is necessary to add a bit of context on the concept of data discovery itself.

Metadata 147

Metadata Data Science Big Data Data 147

Data Engineering Podcast

NOVEMBER 19, 2023

Summary The dbt project has become overwhelmingly popular across analytics and data engineering teams. While it is easy to adopt, there are many potential pitfalls. Dustin Dorsey and Cameron Cyr co-authored a practical guide to building your dbt project. In this episode they share their hard-won wisdom about how to build and scale your dbt projects.

Project 147

Project Data Lake SQL High Quality Data 147

databricks

NOVEMBER 15, 2023

The observation that "software is eating the world" has shaped the modern tech industry. Today, software is ubiquitous in our lives, from the.

Data 145

Data 145

ArcGIS

NOVEMBER 7, 2023

From oriented imagery to engaging thematic map series, there is something for everyone in this release of ArcGIS Pro 3.2.

143

143

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Zalando Engineering

NOVEMBER 8, 2023

Introduction This blog post describes a recent contribution from Zalando to the Postgres JDBC driver to address a long-standing issue with the driver’s integration with Postgres’ logical replication that resulted in runaway Write-Ahead Log (WAL) growth. We will describe the issue, how it affected us at Zalando, and detail the fix made upstream in the JDBC driver that fixes the issue for Debezium and all other clients of the Postgres JDBC driver.

PostgreSQL 139

PostgreSQL Java Database Bytes 139

KDnuggets

NOVEMBER 1, 2023

This list of machine learning algorithms is a good place to start your journey as a data scientist. You should be able to identify the most common models and use them in the right applications.

Machine Learning 157

Machine Learning Algorithm Data 157

Confluent

NOVEMBER 27, 2023

The top 7 free online courses, tutorials, get started guides, and examples for the easiest way to learn Apache Kafka.

Kafka 132

Kafka 132

Data Engineering Podcast

NOVEMBER 12, 2023

Summary Software development involves an interesting balance of creativity and repetition of patterns. Generative AI has accelerated the ability of developer tools to provide useful suggestions that speed up the work of engineers. Tabnine is one of the main platforms offering an AI powered assistant for software engineers. In this episode Eran Yahav shares the journey that he has taken in building this product and the ways that it enhances the ability of humans to get their work done, and when t

Software Engineer 147

Software Engineer Software Engineering Engineering Data Lake 147

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

databricks

NOVEMBER 21, 2023

We recently announced our AI-generated documentation feature, which uses large language models (LLMs) to automatically generate documentation for tables and columns in Unity.

Data Science 145

Data Science Engineering Data 145

ArcGIS

NOVEMBER 21, 2023

Contextual keyboard shortcuts open up a whole new world of customization possibilities in ArcGIS Pro 3.2.

134

134

Snowflake

NOVEMBER 2, 2023

Snowflake’s single, cross-cloud governance model has always been a powerful differentiator, enabling customers to manage their increasingly complex data ecosystems with simplicity and ease. As a result, Snowflake is enhancing its governance capabilities that thousands of customers already rely on through Snowflake Horizon. Snowflake Horizon is Snowflake’s built-in governance solution with a unified set of compliance, security, privacy, interoperability, and access capabilities.

Metadata 131

Metadata Government AWS Medical 131

KDnuggets

NOVEMBER 7, 2023

No, they’re not the same jobs! Learn what responsibilities, skills, and tools used make them different. Then, choose the right career path for you.

Data Science 156

Data Science Data Engineering Data Engineer Engineering 156

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Confluent

NOVEMBER 28, 2023

Learn how to write code that produces messages via librdkafka, how it will behave during error situations, and how your application should detect and respond to them.

Coding 131

Coding IT Kafka 131

Data Engineering Podcast

NOVEMBER 5, 2023

Summary Databases are the core of most applications, but they are often treated as inscrutable black boxes. When an application is slow, there is a good probability that the database needs some attention. In this episode Lukas Fittl shares some hard-won wisdom about the causes and solution of many performance bottlenecks and the work that he is doing to shine some light on PostgreSQL to make it easier to understand how to keep it running smoothly.

PostgreSQL 147

PostgreSQL Data Lake SQL High Quality Data 147

databricks

NOVEMBER 6, 2023

We are excited to announce that we have completed our acquisition of Arcion, a leading provider for real-time data replication technologies. Arcion’s capabilities w.

Data 139

Data Technology 139

Waitingforcode

NOVEMBER 28, 2023

In March I wrote a blog showing how to use accumulators to know the application of each filter statement. Turns out, the solution may not be perfect as mentioned by Aravind in one of the comments. I bet you already have an idea but if not, keep reading. Everything will be clear in the end!

130

130

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Seattle Data Guy

NOVEMBER 28, 2023

Data warehousing would be easy if all data were structured and formatted in the data source. Maybe we wouldn’t even need to build a data warehouse. But as anyone who has worked with data from more than one source knows, that’s rarely the case. Businesses today need to pull data from a plethora of sources,… Read more The post Finding The Right ETL/ELT Solution – What Is Estuary And Should You Use It?

IT 130

IT Data Warehouse Building Data 130

KDnuggets

NOVEMBER 29, 2023

From ANI to AGI and Beyond: Deciphering AI's Evolutionary Path.

154

154

Snowflake

NOVEMBER 15, 2023

BUILD 2023 is where AI gets real. Join our two-day virtual global conference and learn how to build with the app dev innovations you heard about at Snowflake Summit and Snowday. We have more demos and hands-on virtual labs than ever before—and you won’t find a bunch of slideware here. The focus is on tools and capabilities that are generally available or in public and private preview, so you can leave BUILD and put your new skills into action immediately.

Building 130

Building Accessible Accessibility IT 130

ArcGIS

NOVEMBER 30, 2023

Learn how to filter coordinate systems based on a spatial extent, GCS, or projection property.

Project 129

Project Systems 129

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing