Azure Data Engineering
NOVEMBER 21, 2021
During some scenarios in Azure Data Factory, we may want to intentionally stop the execution of the pipeline. An example could be when we want to check the existence of a file or folder using Get Metadata activity. We may want to fail the pipeline if the file/folder does not exist. To achieve this, we could use the Fail Activity. Invoking the Fail Activity ensures that the pipeline execution will be stopped.
KDnuggets
NOVEMBER 23, 2021
Sure, we all make mistakes -- which can be a bit more painful when we are trying to get hired -- so check out these typical errors applicants make while answering SQL questions during data science interviews.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Data Engineering Podcast
NOVEMBER 20, 2021
Summary The technology for scaling storage and processing of data has gone through massive evolution over the past decade, leaving us with the ability to work with massive datasets at the cost of massive complexity. Nick Schrock created the Dagster framework to help tame that complexity and scale the organizational capacity for working with data. In this episode he shares the journey that he and his team at Elementl have taken to understand the state of the ecosystem and how they can provide a f
Teradata
NOVEMBER 22, 2021
Teradata's Martin Willcox recently passed 17 years at Teradata and a quarter of a century in the industry. Here are the ten things he's learned about data analytics in those 20-odd years.
Speaker: Timothy Chan, PhD., Head of Data Science
Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.
Cloudera
NOVEMBER 25, 2021
Advances in the performance and capability of Artificial Intelligence (AI) algorithms has led to a significant increase in adoption in recent years. In a February 2021 report by IDC, they estimate that world-wide revenues from AI will grow by 16.4% in 2021 to USD $327 billion. Furthermore, AI adoption is becoming increasingly widespread and not just concentrated within a small number of organisations.
KDnuggets
NOVEMBER 23, 2021
Also: How I Redesigned over 100 ETL into ELT Data Pipelines; Where NLP is heading; Don’t Waste Time Building Your Data Science Network; Data Scientists: How to Sell Your Project and Yourself.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Teradata
NOVEMBER 22, 2021
Teradata's Martin Willcox recently passed 17 years at Teradata and a quarter of a century in the industry. Here are the ten things he's learned about data analytics in those 20-odd years.
Cloudera
NOVEMBER 24, 2021
For underserved youth, the lack of educational opportunity can seriously hinder their development and future career prospects. Many are deprived of early childhood chances at experiencing the professional world, so a career in science, finance, IT, or marketing is a pipe dream. . Unless someone shows them it’s possible. At the Middle Tennessee and Peninsula chapters of the Boys & Girls Clubs, high school students are receiving an introduction into a new world of possibilities.
KDnuggets
NOVEMBER 24, 2021
Maintaining a centralized data repository can simplify your business intelligence initiatives. Here are four data integration tools that can make data more valuable for modern enterprises.
Pipeline Data Engineering
NOVEMBER 24, 2021
Most data professionals realise very early in their journey that accessing the knowledge that they really need to solve data engineering problems is hard to come by. The other thing they don’t necessarily see is how short-sighted a lot of courses are, and how most of the technical content they provide is going to be rendered useless in a year or two.
Speaker: Anne Steiner and David Laribee
As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.
ProjectPro
NOVEMBER 24, 2021
“Is data science hard to learn?”, “Is data science a hard job?”, “Is it hard to get a data science job?” Are you a data science enthusiast who believes data science is hard and keeps thinking about such questions? Allow us to challenge your thoughts and read this blog as we will help you answer all those questions.
Preset
NOVEMBER 23, 2021
In this tutorial, we'll learn how to build a metrics dashboard with Apache Superset, a modern and open-source data exploration and visualization platform. We'll also use Cube, an open-source metrics store, as the data source for Superset.
KDnuggets
NOVEMBER 22, 2021
PyTorch and TensorFlow are the two leading AI/ML Frameworks. In this article, we take a look at their on-device counterparts PyTorch Mobile and TensorFlow Lite and examine them more deeply from the perspective of someone who wishes to develop and deploy models for use on mobile platforms.
RudderStack
NOVEMBER 23, 2021
In this update, we cover our new high-performance JavaScript SDK, announce a new destination integration, and highlight our Event Stream pricing promotion.
Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage
Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.
ProjectPro
NOVEMBER 23, 2021
Towards the end of the 2000s, complex neural networks and model-based deep learning saw a huge upsurge in demand with revolutionary results in the fields of computer vision and natural language processing. While reinforcement learning has been around the corner from the same time, it was overshadowed by its counterparts for decades. It first became the talk of the town when in 2016, Google Deepmind’s AlphaGo defeated the World Champion in the Chinese game of Go.
Preset
NOVEMBER 22, 2021
This tutorial post will cover all of the steps needed to make your first code contribution to the Apache Superset project.
KDnuggets
NOVEMBER 24, 2021
Companies are racing to use AI, but despite its vast potential, most AI projects fail. Examining and resolving operational issues upfront can help AI initiatives reach their full potential.
Data Engineering Podcast
NOVEMBER 20, 2021
Summary One of the perennial challenges posed by data lakes is how to keep them up to date as new data is collected. With the improvements in streaming engines it is now possible to perform all of your data integration in near real time, but it can be challenging to understand the proper processing patterns to make that performant. In this episode Ori Rafael shares his experiences from Upsolver and building scalable stream processing for integrating and analyzing data, and what the tradeoffs are
Advertisement
Are you trying to decide which entity resolution capabilities you need? It can be confusing to determine which features are most important for your project. And sometimes key features are overlooked. Get the Entity Resolution Evaluation Checklist to make sure you’ve thought of everything to make your project a success! The list was created by Senzing’s team of leading entity resolution experts, based on their real-world experience.
DataKitchen
NOVEMBER 20, 2021
What makes an effective DataOps Engineer? A DataOps Engineer shepherds process flows across complex corporate structures. Organizations have changed significantly over the last number of years and even more dramatically over the previous 12 months, with the sharp increase in remote work. A DataOps engineer runs toward errors. You might ask what that means.
Cloudera
NOVEMBER 25, 2021
As data continues to grow at an exponential rate, our customers are increasingly looking to advance and scale operations through digital transformation and the cloud. These modern digital businesses are also dealing with unprecedented rates of data volume, which is exploding from terabytes to petabytes and even exabytes which could prove difficult to manage.
KDnuggets
NOVEMBER 22, 2021
This article is the second article of an ongoing series on using Dask in practice. Each article in this series will be simple enough for beginners, but provide useful tips for real work. The next article in the series is about parallelizing for loops, and other embarrassingly parallel operations with dask.delayed.
ProjectPro
NOVEMBER 22, 2021
PySpark has exploded in popularity in recent years, and many businesses are capitalizing on its advantages by producing plenty of employment opportunities for PySpark professionals. According to the Businesswire report , the worldwide big data as a service market is estimated to grow at a CAGR of 36.9% from 2019 to 2026, reaching $61.42 billion by 2026.
Speaker: Margaret-Ann Seger, Head of Product, Statsig
Experimentation is often seen as an aspirational practice, especially at smaller, fast-moving companies who are strapped for time and resources. So, how can you get your team making decisions in a more data-driven way while continuing to remain lean and maintaining ship velocity? In this webinar, Margaret-Ann Seger, Head of Product at Statsig, will teach you how to build an experimentation culture from the ground-up, graduating from just getting started with data-driven development to operating
AltexSoft
NOVEMBER 23, 2021
Nowadays, all organizations need real-time data to make instant business decisions and bring value to their customers faster. But this data is all over the place: It lives in the cloud, on social media platforms, in operational systems, and on websites, to name a few. Not to mention that additional sources are constantly being added through new initiatives like big data analytics , cloud-first, and legacy app modernization.
Cloudera
NOVEMBER 23, 2021
Concepts. What is Cloudera Operational Database (COD)? Operational Database is a relational and non-relational database built on Apache HBase and is designed to support OLTP applications, which use big data. The operational database in Cloudera Data Platform has the following components: . Apache Phoenix provides a relational model facilitating massive scalability.
KDnuggets
NOVEMBER 25, 2021
You can call Mito into your Jupyter Environment and each edit you make will generate the equivalent Python in the code cell below.
Rockset
NOVEMBER 23, 2021
We built Rockset with the mission to make real-time analytics easy and affordable in the cloud. We put our users first and obsess about helping our users achieve speed, scale and simplicity in their modern real-time data stack (some of which I discuss in depth below). But we, as a team, still take performance benchmarks seriously. Because they help us communicate that performance is one of the core product values at Rockset.
Speaker: David Bard, Principal at VP Product Coaching
In the fast-paced world of digital innovation, success is often accompanied by a multitude of challenges - like the pitfalls lurking at every turn, threatening to derail the most promising projects. But fret not, this webinar is your key to effective product development! Join us for an enlightening session to empower you to lead your team to greater heights.
KDnuggets
NOVEMBER 24, 2021
Until November 29th, you can join over 1.5 million students around the globe and gain the skills of successful data science professionals with unlimited annual access to the 365 Data Science Program at 72% OFF. Read on to learn more!
KDnuggets
NOVEMBER 22, 2021
Read some of the key things the author has learned during the infamous job seeking stage.
KDnuggets
NOVEMBER 25, 2021
A classic KDnuggets Thanksgiving cartoon examines the predicament of one group of fowl Data Scientists.
KDnuggets
NOVEMBER 26, 2021
Learn to use custom Named Entity Recognition and Relation Extraction models.
Advertisement
“Reimagined: Building Products with Generative AI” is an extensive guide for integrating generative AI into product strategy and careers featuring over 150 real-world examples, 30 case studies, and 20+ frameworks, and endorsed by over 20 leading AI and product executives, inventors, entrepreneurs, and researchers.
Expert insights. Personalized for you.
130 
Let's personalize your content