Cloudera

MARCH 4, 2024

In this article, we will walk you through the process of implementing fine grained access control for the data governance framework within the Cloudera platform. In a good data governance strategy, it is important to define roles that allow the business to limit the level of access that users can have to their strategic data assets.

Metadata 52

Metadata Data Governance Government Management 52

Tweag

APRIL 26, 2023

Computing is all about transforming data. Moreover, these steps can be combined in different ways, perhaps omitting some or changing the order of others, producing different data processing pipelines tailored to a particular task at hand. Even then, however, GHC will not complain if we write myPipeline = monomorphize.

Data Process 98

Data Process Process Programming Data 98

Snowflake

SEPTEMBER 11, 2023

Data privacy regulations are sweeping across the globe, with some 71% of countries worldwide adopting data protection and privacy legislation. The European Union’s General Data Protection Regulation (GDPR) , one of the more well-known and far-reaching of these privacy regulations, went into effect on May 25, 2018.

Cloud 74

Cloud Electronics Government Data Governance 74

Monte Carlo

JANUARY 29, 2024

It’s one thing to get your data into a modern data cloud. Monte Carlo is thrilled to be part of the Snowflake Horizon partner ecosystem as we leverage many of the pre-built features Snowflake provides in order to help organizations reduce their data downtime and improve data quality at scale.

Metadata 64

Metadata High Quality Data Data Pipeline Machine Learning 64

Snowflake

SEPTEMBER 27, 2023

At Snowflake, we are dedicated to helping our customers effectively mobilize their data while upholding stringent standards for compliance and data governance. We understand the importance of quick and proactive identification of objects requiring governance, as well as the implementation of protective measures using tags and policies.

Data Governance 60

Data Governance Government SQL Database 60

Towards Data Science

JUNE 16, 2023

How to Identify Your Business-Critical Data Practical steps to identifying business-critical data models and dashboards and drive confidence in your data Source: synq.io This article has been co-written with Lindsay Murphy Not all data is created equal. There’s a good reason for this. Source: synq.io

BI 80

BI Data ETL Tools Machine Learning 80

RudderStack

NOVEMBER 2, 2021

Modern teams need a more robust data integration solution than GTM. Here are 4 of the reasons GTM and data engineers don’t get along plus a better solution.

Data Engineering 40

Data Engineering Data Engineer Engineering Management 40

Christophe Blefari

SEPTEMBER 28, 2023

Make your data stack take-off ( credits ) Hello, another edition of Data News. This week, we're going to take a step back and look at the current state of data platforms. What are the current trends and why are people fighting around the concept of the modern data stack. Is the modern data stack dying?

Cloud Storage 147

Cloud Storage Big Data Hadoop SQL 147

Lyft Engineering

OCTOBER 3, 2023

High-quality data is necessary for the success of every data-driven company. It is now the norm for tech companies to have a well-developed data platform. This makes it easy for engineers to generate, transform, store, and analyze data at the petabyte scale. What and Where is Data Quality?

Big Data 84

Big Data Metadata Data Warehouse Data 84

Towards Data Science

MARCH 1, 2023

Similarly, our flatMapWithGroupState will accumulate tags (evaluated true/false Sigma expressions) and later release them. Our Flux Capacitor function is easy to configure and let’s the user specify how and when each individual tag is stored and retrieved. This evaluator is a no-op, it simply passes the current tag value through.

Designing 76

Designing Scala Data Science Process 76

DoorDash Engineering

AUGUST 1, 2023

Just when we most needed observability data, the system would leave us in the lurch. Challenges Faced With StatsD StatsD was a great asset for our early observability needs, but we began encountering constraints such as losing metrics during surge events, difficulties with naming/standardized tags, and a lack of reporting tools.

AWS 82

AWS Transportation Programming Language Government 82

KDnuggets

MAY 24, 2022

An ML system requiring thousands of tagged samples is fundamentally different from the mind of a child, which can learn from just a few experiences of untagged data.

Machine Learning 160

Machine Learning Systems Data 160

Rockset

APRIL 18, 2023

Organizations have continued to accumulate large quantities of unstructured data, ranging from text documents to multimedia content to machine and sensor data. Comprehending and understanding how to leverage unstructured data has remained challenging and costly, requiring technical depth and domain expertise.

Unstructured Data 105

Unstructured Data Metadata Machine Learning SQL 105

Precisely

DECEMBER 6, 2023

The European Union’s General Data Protection Regulation (commonly known as GDPR) came into effect on the 25th May 2018. It is important to any user or processor of personally identifiable information (PII) as this new law marks a seismic change in the way that organizations must protect consumers’ data.

Retail 64

Retail Data Systems Data Integration 64

Towards Data Science

OCTOBER 25, 2023

Not too long ago, almost all data architectures and data team structures followed a centralized approach. As a data or analytics engineer, you knew where to find all the transformation logic and models because they were all in the same codebase. There was only one data team, two at most.

Data Pipeline 83

Data Pipeline SQL Data Architecture Data 83

Marc Lamberti

SEPTEMBER 29, 2023

Then you will set up and run your local development environment using the Astro CLI to create your first data pipeline. Concretely, you must create data pipelines to produce valuable data for later analytics or machine learning. To create, schedule, and monitor this kind of data pipeline you need a tool.

Data Pipeline 130

Data Pipeline Python AWS Project 130

Knowledge Hut

JULY 26, 2023

It's portrayed as a digital shield that keeps our data safe from prying eyes. It involves special tags and readers. The tags, which are like small chips, send out information when they come close to a reader. RFID technology works through the interaction between RFID tags, readers, and a backend system.

IT 52

IT Electronics Data Collection Technology 52

François Nguyen

JANUARY 23, 2021

or why you should have a look at Data Observability ! This article is the second part on how Airbnb is managing data quality : “Part 2 — A New Gold Standard” The first part can be found here and it was just good principles about roles and responsabilities. Welcome to this strange way of managing data quality !

Finance 130

Finance Certification Data Pipeline Data 130

Netflix Tech

NOVEMBER 11, 2022

Data analysis and machine learning techniques are great candidates to help secure large-scale streaming platforms. We present a systematic overview of the unexpected streaming behaviors together with a set of model-based and data-driven anomaly detection strategies to identify them.

Machine Learning 124

Machine Learning Digital Media Datasets Media 124

Cloudera

OCTOBER 26, 2020

Just like when it comes to data access in business. Enabling data access for end-users so they can drive insight and business value is a typical area of compromise between IT and users. Data access can either be very secure but restrictive or very open yet risky. Quickly onboard data. Multi-tenant data access.

Data Security 116

Data Security IT Unstructured Data Government 116

Data Engineering Podcast

DECEMBER 21, 2020

Summary Data governance is a term that encompasses a wide range of responsibilities, both technical and process oriented. One of the more complex aspects is that of access control to the data assets that an organization is responsible for managing. If you hand a book to a new data engineer, what wisdom would you add to it?

Data Governance 130

Data Governance Government Data Lake Banking 130

Knowledge Hut

MARCH 18, 2024

In the field of computer science and programming, whether it's managing data, executing algorithms, or optimizing memory usage, I always required streamlined processes. The stack data structure is a fundamental tool that embodies simplicity and efficiency in data management. What is Stack Data Structure?

Algorithm 52

Algorithm Computer Science Python Software Engineer 52

Cloudera

DECEMBER 21, 2020

In this blog we will take you through a persona-based data adventure, with short demos attached, to show you the A-Z data worker workflow expedited and made easier through self-service, seamless integration, and cloud-native technologies. In our data adventure we assume the following: . Company data exists in the data lake.

Banking 97

Banking Data Lake Data Data Warehouse 97

Engineering at Meta

FEBRUARY 7, 2023

This global, multi-scale map serves as a foundational layer — showing a variety of geographic features onto which we can blend other data, such as local points of interest or locally important features. Recently, we partnered with the Daylight team at Meta to create Earth Tables, an extension of Daylight that simplifies OSM tags.

Entertainment 133

Entertainment Transportation Data Schemas AWS 133

Snowflake

APRIL 4, 2023

At Snowflake, we like data. And we appreciate people who also like data—and like working with it. Snowflake Marketplace is designed with that in mind: giving people and organizations a place to easily find, try, and buy data and applications that can help them build more and do more with data.

Cloud 52

Cloud Accessible Accessibility Project 52

Cloudera

OCTOBER 30, 2023

The legacy IT infrastructure to run the business operations — mainly data centers — has a deadline to shift to cloud-based services. The public cloud is increasingly becoming the preferred platform to host data analytics – related projects, such as business intelligence, machine learning (ML), and AI applications.

Cloud 74

Cloud Finance Cloud Computing Government 74

DareData

NOVEMBER 28, 2023

In today's data-driven world, machine learning has emerged as a transformative force, empowering organizations to extract valuable insights from vast amounts of data. As the scope of the models and the data continues to scale, the role of a Data Scientist has evolved accordingly in the last years. But that is not all!

Data Science 52

Data Science Medical Machine Learning Data 52

Data Engineering Weekly

JUNE 4, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack provides data pipelines that make it easy to collect data from every application, website, and SaaS platform, then activate it in your warehouse and business tools. Do you need data/ metrics at all? Sign up free to test out the tool today.

Data Engineering 52

Data Engineering Data Engineer Engineering Medical 52

ProjectPro

JANUARY 24, 2023

SpaCy comes with two powerful functionalities, namely, Parts-of-speech (POS) Tagging and Named-Entity Recognition Tagging. Project Idea: NLP Project to Build a Resume Parser in Python using Spacy Gensim Gensim is the Python library used for vectorizing textual data before passing the data at the input of a machine learning model.

Python 52

Python Project Programming Language Data Mining 52

Christophe Blefari

MARCH 1, 2023

dbt Core is an open-source framework that helps you organise data warehouse SQL transformation. dbt was born out of the analysis that more and more companies were switching from on-premise Hadoop data infrastructure to cloud data warehouses. This switch has been lead by modern data stack vision. Enter the ELT.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

Knowledge Hut

FEBRUARY 1, 2024

Today, we’ll talk about how Machine Learning (ML) can be used to build a movie recommendation system - from researching data sets & understanding user preferences all the way through training models & deploying them in applications. Let’s begin with all the aspects of movie recommendation using ML.

Systems 98

Systems Entertainment Datasets Algorithm 98

AltexSoft

AUGUST 29, 2023

In 2010, a transformative concept took root in the realm of data storage and analytics — a data lake. The term was coined by James Dixon , Back-End Java, Data, and Business Intelligence Engineer, and it started a new era in how organizations could store, manage, and analyze their data. What is a data lake?

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

Snowflake

APRIL 29, 2024

Identification of potential threats through data analysis and risk assessment processes to shift toward more proactive risk management. This strategy must incorporate business impact analyses as well as backup and recovery plans in the event of a security incident or loss of access to data. or better) for data in transit.

Transportation 80

Transportation Data Governance Government Consulting 80

The Pragmatic Engineer

JANUARY 12, 2023

We cover one and a half out of eight topics in today’s subscriber-only issue, Inside Pollen's Transparent Compensation Data. In this week’s The Scoop, I analyzed this information and dissected it, going well beyond the raw data. To get this newsletter every week, subscribe here. Tech compensation numbers.

Software Engineer 183

Software Engineer Software Engineering Engineering Raw Data 183

Snowflake

APRIL 3, 2023

The newly offered SnowPro Advanced: Data Analyst Certification allows data analysts to showcase their Snowflake expertise. This certification is intended for Snowflake practitioners who have at least one year of Snowflake Data Cloud analytics experience, including practical, hands-on use of Snowflake. Ready to hone your skills?

Certification 52

Certification Consulting Data Analysis Cloud 52

dbt Developer Hub

JANUARY 23, 2023

Testing the quality of data in your warehouse is an important aspect in any mature data pipeline. One of the biggest blockers for developing a successful data quality pipeline is aggregating test failures and successes in an informational and actionable way. However, ensuring actionability can be challenging. on BigQuery.

Metadata 52

Metadata High Quality Data SQL Data Integration 52

Snowflake

AUGUST 21, 2023

With demonstrable success across a range of industries, organizations are increasingly pursuing cutting-edge data mesh architectures to enhance self-service data use. Data-as-a-product: By considering data resources through a product lens, teams can adopt practices centered around quality and ease of use.

Pipeline-centric 52

Pipeline-centric Architecture Government Data Architect 52