Seattle Data Guy

SEPTEMBER 20, 2024

We are still in the early days of data and the value it can add to companies. You’ll read plenty of statistics about how much value data can drive and how far behind companies that aren’t using data are. And as a data consultant, I have helped companies find that value in their data. It… Read more The post How To Modernize Your Data Strategy And Infrastructure For 2025 appeared first on Seattle Data Guy.

Consulting 130

Consulting Data IT Data Warehouse 130

The Pragmatic Engineer

SEPTEMBER 19, 2024

This is a follow-up to the article Paying down tech debt , written by industry veteran Lou Franco. Lou has been in the software business for over 30 years as an engineer, EM, and executive. He’s also worked at four startups and the companies that later acquired them; most recently Atlassian as a Principal Engineer on the Trello iOS app. Later this year, he’s publishing a book on tech debt.

Recruitment 133

Recruitment Java Coding Project 133

Start Data Engineering

SEPTEMBER 18, 2024

1. Introduction 2. Setup 3. Parts of data engineering 3.1. Requirements 3.1.1. Understand input datasets available 3.1.2. Define what the output dataset will look like 3.1.3. Define SLAs so stakeholders know what to expect 3.1.4. Define checks to ensure the output dataset is usable 3.2. Identify what tool to use to process data 3.3. Data flow architecture 3.

Project 130

Project Building Datasets Architecture 130

Confessions of a Data Guy

SEPTEMBER 19, 2024

It is a Brave New World out there these days. The new tools and features come out faster than your mom on Sunday morning getting you ready for church. The same goes for the context and advice being produced on a myriad of platforms, the ole’ Like and Subscribe, and all that bit. It does […] The post Data Modeling in the Brave New Lakehouse World appeared first on Confessions of a Data Guy.

Data 113

Data IT Data Engineer Data Engineering 113

Speaker: Evelyn Chou

Choosing the right business intelligence (BI) platform can feel like navigating a maze of features, promises, and technical jargon. With so many options available, how can you ensure you’re making the right decision for your organization’s unique needs? 🤔 This webinar brings together expert insights to break down the complexities of BI solution vetting.

BI

databricks

SEPTEMBER 18, 2024

The Databricks Generative AI Startup Challenge offers $1M+ in prizes for innovative startups building Generative AI use cases on Databricks. Apply by November 1, 2024!

Building 136

Building 136

KDnuggets

SEPTEMBER 19, 2024

In Python, functions often require multiple arguments, and you may find yourself repeatedly passing the same values for certain parameters. This is where partial functions can help. Python’s built-in functools module allows you to create partial functions.

Python 119

Python 119

Snowflake

SEPTEMBER 19, 2024

Data is increasingly becoming critical for the public sector — from guiding decisions in higher education to enhancing citizen services and streamlining government operations. Government agencies are overwhelmed with data, whether it be structured, like incident logs, or unstructured, like satellite images. Harnessing the vast amount of data can become a burden for any organization, yet the insights have the potential to significantly improve quality of life and strengthen national security.

Government 93

Government Education Data Analytics Engineering 93

Confessions of a Data Guy

SEPTEMBER 13, 2024

Did you know there are only 3 types of Data Engineers? It’s true. I hope you are the right one. The post The 3 Types of Data Engineers. appeared first on Confessions of a Data Guy.

Data Engineering 100

Data Engineering Data Engineer Engineering Data 100

databricks

SEPTEMBER 18, 2024

Transformer models, the backbone of modern language AI, rely on the attention mechanism to process context when generating output. During inference, the attention.

Process 96

Process 96

KDnuggets

SEPTEMBER 18, 2024

Medical imaging has been revolutionized by the adoption of deep learning techniques. The use of this branch of machine learning has ushered in a new era of precision and efficiency in medical image segmentation, a central analytical process in modern healthcare diagnostics and treatment planning. By harnessing neural networks, deep learning algorithms are able.

Medical 120

Medical Deep Learning Healthcare Algorithm 120

Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL

Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.

Government

Cloudera

SEPTEMBER 18, 2024

According to recent survey data from Cloudera, 88% of companies are already utilizing AI for the tasks of enhancing efficiency in IT processes, improving customer support with chatbots, and leveraging analytics for better decision-making. More and more enterprises are leveraging pre-trained models for various applications, from natural language processing to computer vision.

Datasets 75

Datasets Retail Healthcare Finance 75

Snowflake

SEPTEMBER 16, 2024

As one of the most widely used libraries in the Python ecosystem, pandas helps developers analyze, load and transform data across data science, data engineering and machine learning. The flexibility and ease of use of the pandas API have driven rapid growth in popularity, with pandas being used by one in every five developers , according to the StackOverflow 2024 Developer Survey.

Python 86

Python Government Data Pipeline Data 86

Precisely

SEPTEMBER 18, 2024

Businesses around the world are facing major challenges due to higher manufacturing costs, disruptive new technologies like artificial intelligence (AI), and tougher global competition. This means it’s more important than ever to make data-driven decisions, cut costs, and improve efficiency. But this is all easier said than done, as evidenced by key findings from this year’s 2025 Outlook: Data Integrity Trends and Insights report, published in partnership between Precisely and the Center for App

Data Integration 68

Data Integration Data Governance Government Data Programming 68

databricks

SEPTEMBER 17, 2024

Today, we are excited to announce the support for named parameter markers in the SQL editor. This feature allows you to write parameterized.

SQL 107

SQL 107

Speaker: Christophe Louvion, Chief Product & Technology Officer of NRC Health and Tony Karrer, CTO at Aggregage

Christophe Louvion, Chief Product & Technology Officer of NRC Health, is here to take us through how he guided his company's recent experience of getting from concept to launch and sales of products within 90 days. In this exclusive webinar, Christophe will cover key aspects of his journey, including: LLM Development & Quick Wins 🤖 Understand how LLMs differ from traditional software, identifying opportunities for rapid development and deployment.

Designing

KDnuggets

SEPTEMBER 16, 2024

Image by Editor | Ideogram Let’s learn how to perform time series data aggregation in Pandas. Preparation We would need the Pandas and Numpy packages installed, so we can install them using the following code: pip install pandas numpy With the packages installed, let’s jump into the article. Time Series.

Data 127

Data Coding 127

Confluent

SEPTEMBER 17, 2024

Confluent for VS Code streamlines workflows, accelerates development cycles, and enhances real-time data processing, all within a unified environment.

Building 72

Building Management Coding Data 72

Snowflake

SEPTEMBER 18, 2024

As the frenzied hype around generative AI cools off and as we get into the year of ideation, earlier adopters of AI are starting to see the results of initial experimentation. And these conversations are increasingly shifting to a more problem-oriented mentality. A lot of people were understandably swept up in the excitement of all that AI can do, only to find that some use cases were too risky or that those problems could be solved with traditional methods that were less costly.

Building 65

Building Process IT Data 65

Yelp Engineering

SEPTEMBER 18, 2024

Machine Learning Feature Stores ML Feature Store at Yelp Many of Yelp’s core capabilities such as business search, ads, and reviews are powered by Machine Learning (ML). In order to ensure these capabilities are well supported, we have built a dedicated ML platform. One of the pillars of this infrastructure is the Feature Store, which is a centralized data store for ML Features that are the input of ML models.

Machine Learning 59

Machine Learning Data Governance Government Data 59

Advertisement

Entity Resolution Sometimes referred to as data matching or fuzzy matching, entity resolution, is critical for data quality, analytics, graph visualization and AI. Learn what entity resolution is, why it matters, how it works and its benefits. Advanced entity resolution using AI is crucial because it efficiently and easily solves many of today’s data quality and analytics problems.

IT

databricks

SEPTEMBER 19, 2024

Mosaic AI Model Training now supports fine-tuning up to 131K context length for Llama 3.1 models. More efficient training at long sequence lengths is made possible by several optimizations highlighted in this post.

76

76

KDnuggets

SEPTEMBER 19, 2024

ggplot2 is a tool in R for making charts. You can create charts with dots, bars, or lines. You can also add layers to show more details. This article will help you learn how to use ggplot2 to create visualizations. Getting started with ggplot2 Before using ggplot2, you need to install it and.

Data 108

Data IT 108

Data Engineering Weekly

SEPTEMBER 15, 2024

Uber: DataMesh - How Uber laid the foundations for the Data Lake Cloud migration Many companies are slowly adopting DataMesh, and Uber writes about adopting the data mesh principle. Whether or not Data Mesh is a separate product is debatable, but it is certainly an impactful framework for scaling data platforms. I expect more and more frameworks to be built on top of the existing compute and orchestration layer.

Data Engineering 67

Data Engineering Data Engineer Engineering Data Lake 67

DataKitchen

SEPTEMBER 18, 2024

A DataOps Approach to Data Quality The Growing Complexity of Data Quality Data quality issues are widespread, affecting organizations across industries, from manufacturing to healthcare and financial services. According to DataKitchen’s 2024 market research, conducted with over three dozen data quality leaders, the complexity of data quality problems stems from the diverse nature of data sources, the increasing scale of data, and the fragmented nature of data systems.

Data 60

Data Data Lake Manufacturing Machine Learning 60

Advertisement

The complexity of financial data, the need for real-time insight, and the demand for user-friendly visualizations can seem daunting when it comes to analytics - but there is an easier way. With Logi Symphony, we aim to turn these challenges into opportunities. Our platform empowers you to seamlessly integrate advanced data analytics, generative AI, data visualization, and pixel-perfect reporting into your applications, transforming raw data into actionable insights.

Raw Data

DareData

SEPTEMBER 20, 2024

In software development and other information technology fields, technical debt (also known as design debt or code debt ) is the implied cost of future reworking because a solution prioritizes expedience over long-term design. Analogous with monetary debt , if technical debt is not repaid, it can accumulate "interest", making it harder to implement changes.

Software Engineer 52

Software Engineer Software Engineering Data Engineering Data Engineer 52

databricks

SEPTEMBER 19, 2024

The value of Generative AI, the deepened investment Databricks has made in the space, and how customers have benefited from the certification.

Certification 96

Certification 96

KDnuggets

SEPTEMBER 18, 2024

Heaps and heaps of new technology are entering the market, and these new tools and software are making our lives easier. However, although our day-to-day tasks have become easier, there has been an increase in the number of cyber threats. There is a need for the right people to come in and identify and mitigate.

Technology 103

Technology 103

Hevo

SEPTEMBER 20, 2024

As developers and data engineers build complex applications in Snowflake, monitoring performance is essential for ensuring smooth operation and a positive customer experience. Snowflake operations can be tracked using Snowsight, which provides tools for managing costs, tracking query history, monitoring data loading and transformations, and overseeing data governance activities.

Data Governance 52

Data Governance Government Data Engineering Data Engineer 52

Speaker: Jamie Bernard

Success in product management goes beyond delivering great features - it’s about achieving measurable financial outcomes that resonate across the organization. By connecting your product’s journey with the company’s financial success, you’ll ensure that every feature, release, and innovation contributes to the bottom line, driving both customer satisfaction and business growth.

Management

Precisely

SEPTEMBER 17, 2024

In the evolving landscape of digital finance, the importance of robust cybersecurity measures cannot be overstated. The European Union’s Digital Operational Resilience Act (DORA) represents a pivotal step towards safeguarding the financial sector against the growing complexities of cyber threats. If your organization operates within the financial services ecosystem or provides ICT services to this sector, understanding DORA is crucial.

IT 58

IT Finance Management Systems 58

DareData

SEPTEMBER 20, 2024

In software development and other information technology fields, technical debt (also known as design debt [1] or code debt ) is the implied cost of future reworking because a solution prioritizes expedience over long-term design. Analogous with monetary debt , if technical debt is not repaid, it can accumulate "interest", making it harder to implement changes.

Management 52

Management Software Engineer Software Engineering Data Engineering 52

databricks

SEPTEMBER 18, 2024

At Databricks, we know that data is one of your most valuable assets. Our product and security teams work together to deliver an enterprise-grade Data Intelligence Platform that enables you to defend against security risks and meet your compliance obligations. In this blog, we'll explain how you can leverage our platform's security features to establish a robust defense-in-depth posture that protects your data and AI assets from risks.

Data 85

Data 85

KDnuggets

SEPTEMBER 20, 2024

Rapid development is happening around us, and one of the most interesting aspects of this evolution is artificial intelligence's ability to communicate through natural language with humans. Suppose you want to communicate with some LLM running on your computer without switching between applications or windows, just by using a voice hotkey. This is exactly what.

89

89

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

Data Science