ArcGIS

AUGUST 23, 2024

With a new GIS mapping tool you can map the most visited national parks (and much more!) to explore your spatial data even further.

Designing 145

Designing Data 145

KDnuggets

AUGUST 23, 2024

Learn large language models with these free resources from Deeplearning.AI, Microsoft, AWS, and more.

AWS 139

AWS 139

databricks

AUGUST 22, 2024

Today, we're excited to announce the launch of Data Warehouse Brickbuilder Migration Solutions. This is an expansion to the Brickbuilder Program , which.

Data Warehouse 132

Data Warehouse Data Programming 132

Waitingforcode

AUGUST 22, 2024

Code organization and assertions flow are both important but even them, they can't guarantee your colleagues' adherence to the unit tests. There are other user-facing attributes to consider as well.

Coding 130

Coding 130

Speaker: Jason Chester, Director, Product Management

In today’s manufacturing landscape, staying competitive means moving beyond reactive quality checks and toward real-time, data-driven process control. But what does true manufacturing process optimization look like—and why is it more urgent now than ever? Join Jason Chester in this new, thought-provoking session on how modern manufacturers are rethinking quality operations from the ground up.

Manufacturing

ArcGIS

AUGUST 21, 2024

Does your map need a north arrow? It depends.

IT 137

IT Designing 137

KDnuggets

AUGUST 22, 2024

Have you ever wanted to create your library in Python? Well, it’s achievable and surprisingly quite easy!

Python 130

Python 130

Confessions of a Data Guy

AUGUST 20, 2024

Recently, for some unknown reason, I was pursuing the new Stackoverflow … called Reddit, for Data Engineering … and I ran across an interesting question … more or less it was related to “what makes a good Software Engineer … in a Data Engineering context.” This isn’t the first time this idea has come up […] The post What is a “Good” Data or Software Engineer?

Software Engineer 113

Software Engineer Software Engineering Engineering Data Engineering 113

ArcGIS

AUGUST 21, 2024

Discover how ArcGIS empowers users at all levels to harness the potential of geospatial AI. Whether you're leveraging pre-trained models for quick insights or building custom AI solutions, ArcGIS offers flexible, powerful tools for every workflow. Explore simplified and advanced AI capabilities across desktop, enterprise, and cloud environments, designed to make geospatial intelligence accessible to everyone.

Cloud 126

Cloud Designing Accessibility Accessible 126

KDnuggets

AUGUST 19, 2024

Improve your Pandas processing with the NumPy package.

Data Analysis 119

Data Analysis Data Process Python 119

databricks

AUGUST 20, 2024

Special thanks to Kevin Glover, Martin Ko, Kuber Sharma and the team at Tableau for their valuable insights and contributions to this blog.

116

116

Speaker: Kenten Danas, Senior Manager, Developer Relations

ETL and ELT are some of the most common data engineering use cases, but can come with challenges like scaling, connectivity to other systems, and dynamically adapting to changing data sources. Airflow is specifically designed for moving and transforming data in ETL/ELT pipelines, and new features in Airflow 3.0 like assets, backfills, and event-driven scheduling make orchestrating ETL/ELT pipelines easier than ever!

Data Engineering

Engineering at Meta

AUGUST 20, 2024

Delivering new AI technologies at scale also means rethinking every layer of our infrastructure – from silicon and software systems and even our data center designs. For the second year in a row, Meta’s engineering and infrastructure teams returned for the AI Infra @ Scale conference, where they discussed the challenges of scaling up an infrastructure for AI as well as work being done on our large-scale GPU clusters , open hardware designs for next-generation data center hardware, and how Meta i

Engineering 109

Engineering Designing Technology Systems 109

ArcGIS

AUGUST 21, 2024

Did you catch the Huff model demo at UC 2024? Read this article for step-by-step instructions on the Business Analyst workflow.

Business Analyst 116

Business Analyst 116

KDnuggets

AUGUST 22, 2024

The article highlights five top free machine learning courses to enhance your skills.

Machine Learning 117

Machine Learning 117

databricks

AUGUST 19, 2024

We are excited to announce the latest addition to the Databricks developer experience: the PyCharm Professional Integration with Databricks ! This new plugin.

107

107

Advertisement

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data Workflow

Engineering at Meta

AUGUST 23, 2024

Learn about new PyTorch advancements for LLMs and how PyTorch is enhancing every aspect of the LLM lifecycle. In this talk from AI Infra @ Scale 2024 , software engineers Wanchao Liang and Evan Smothers are joined by Meta research scientist Kimish Patel to discuss our newest features and tools that enable large-scale training, memory efficient fine-tuning, and on-device LLM capabilities.

Software Engineer 108

Software Engineer Software Engineering Algorithm Architecture 108

Edureka

AUGUST 19, 2024

If you’re curious about learning DevOps Roadmap but don’t know where to start, you’re in the right place! I’ll guide you step-by-step on your journey to becoming a DevOps engineer. First and foremost, we will start with the basic skills required to become a DevOps engineer, then gradually explore the major milestones you need to reach to succeed in this field.

Engineering 98

Engineering Programming Language Python Cloud 98

KDnuggets

AUGUST 20, 2024

Learn how to build the recommendation system with advanced technology.

Systems 117

Systems Building Technology Machine Learning 117

databricks

AUGUST 22, 2024

Databricks Workflows is the cornerstone of the Databricks Data Intelligence Platform, serving as the orchestration engine that powers critical data and AI workloads.

Engineering 85

Engineering Data 85

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

Engineering at Meta

AUGUST 21, 2024

Llama 3 is Meta’s most capable openly-available LLM to date and the recently-released Llama 3.1 will enable new workflows, such as synthetic data generation and model distillation with unmatched flexibility, control, and state-of-the-art capabilities that rival the best closed source models. At AI Infra @ Scale 2024 , Meta engineers discussed every step of how we built and brought Llama 3 to life, from data and training to inference.

Software Engineer 99

Software Engineer Software Engineering Engineering Cloud 99

Edureka

AUGUST 18, 2024

In the 21st century, Full Stack Web Development has undoubtedly transformed the internet. Notably, it is the driving force behind all the sites we see today, as well as mobile applications installed on our devices. Therefore, To succeed in web development, it’s important for anyone hoping to make a career out of it to understand and excel in the full stack developer skills required according to the current market trend.

Programming Language 98

Programming Language Java Non-relational Database Relational Database 98

KDnuggets

AUGUST 20, 2024

This article explains the basics of time series analysis. Learn to prepare your data and visualize trends in R.

Data 114

Data Programming 114

Cloudera

AUGUST 20, 2024

By now, every organization, regardless of industry, has at least explored the use of AI, if not already embraced it. In today’s market, the AI imperative is firmly here, and failing to act quickly could mean getting left behind. But even as adoption soars, struggles remain, and scalability continues to be a major issue. Organizations are quick to adopt AI, but getting it established across the organization brings a unique set of challenges that come into play.

Government 82

Government Data Lake Data Governance Data Architecture 82

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

Engineering at Meta

AUGUST 22, 2024

In this talk from AI Infra @ Scale 2024 , Joel Colburn, a software engineer at Meta, technical lead Junqiang Lan, and software engineer Jack Montgomery discuss the second generation of MTIA , Meta’s in-house training and inference accelerator. They cover the co-design process behind building the second generation of Meta’s first-ever custom silicon for AI workloads, including the PyTorch software ecosystem, and the model architectures for Meta’s key applications.

Designing 88

Designing Software Engineer Software Engineering Utilities 88

Rock the JVM

AUGUST 22, 2024

By Riccardo Cardin At Rock the JVM, we deeply understand the power of the Kotlin Arrow library and the Raise DSL, and we’ve previously shared our insights in our article on Functional Error Handling in Kotlin, Part 3: The Raise DSL. Now, we’re ready to introduce the crucial topic of testing applications that use the Raise DSL. To fully grasp the concepts we’ll be discussing, it’s essential to have a solid understanding of Kotlin, Arrow library, and the Raise DSL.

Portfolio 81

Portfolio Coding Scala Programming 81

KDnuggets

AUGUST 20, 2024

Looking to make your data visuals stand out? Check out these five tips for effective data visualization.

Data 109

Data Data Science 109

Towards Data Science

AUGUST 23, 2024

Expert techniques to elevate your analysis Continue reading on Towards Data Science »

Data Science 75

Data Science SQL Data Data Engineer 75

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data Workflow

Snowflake

AUGUST 22, 2024

Uncertainty is the new norm for today’s brand advertisers and advertising agencies. Google has once again changed its stance on third-party cookies, keeping them for at least the time being. With the future of cookies still uncertain, many advertisers have already been preparing to use alternative targeting strategies, such as first-party data and contextual targeting.

Entertainment 75

Entertainment Media Algorithm Data Collection 75

databricks

AUGUST 21, 2024

Special thanks to David Gray @Epsilon, Tanishq Bhalla @HealthVerity, Itai Weiss @ Nimble, JB Kole @ Mostly.ai for their valuable insights and contributions.

Data 75

Data 75

KDnuggets

AUGUST 19, 2024

Learn how to harness Hugging Face pre-trained language models to classify the emotions behind tweets.

94

94

Towards Data Science

AUGUST 21, 2024

Getting to the bottom of what structuring your data responsibly really means Continue reading on Towards Data Science »

Data Science 75

Data Science Data Structured Data Data Architecture 75

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline