Precisely

JANUARY 9, 2025

Disclaimer: Throughout this post, I discuss a variety of complex technologies but avoid trying to explain how these technologies work. The goal of this post is to understand how data integrity best practices have been embraced time and time again, no matter the technology underpinning. Then came Big Data and Hadoop!

Data Integration 119

Data Integration Hadoop Data Warehouse Data Lake 119

Engineering at Meta

JANUARY 22, 2025

Data lineage is an instrumental part of Metas Privacy Aware Infrastructure (PAI) initiative, a suite of technologies that efficiently protect user privacy. To address this limitation, we utilize Privacy Probes , a key component of our PAI lineage technologies. helping inform the right places to apply privacy controls.

Data Warehouse 109

Data Warehouse SQL Programming Language Data 109

Data Engineering Podcast

MAY 18, 2024

In Code Comments, host Jamie Parker, Red Hatter and experienced engineer, shares the journey of technologists from across the industry and their hard-won lessons in implementing new technologies. Code Comments Podcast Logo]([link] Putting new technology to use is an exciting prospect. Closing Announcements Thank you for listening!

Building 278

Building Data Lake High Quality Data Business Intelligence 278

Data Engineering Weekly

DECEMBER 28, 2024

Conclusion: Embrace, Adapt, and Innovate Looking ahead to 2025, enterprises and professionals must embrace the opportunities presented by AI agents, adapt their skill sets to align with new technologies, and innovate to stay ahead in a competitive landscape.

Unstructured Data 112

Unstructured Data Metadata Data Government 112

Engineering at Meta

MARCH 4, 2025

QLC technology addresses these challenges by forming a middle tier between HDDs and TLC SSDs. QLC flash as a technology has been around since 2009. Pure Storages DFMs can allow scaling up to 600TB with the same NAND package technology. Hardware considerations for adopting QLC While E1.S E3 doesnt bring additional value over U.2

Bytes 101

Bytes Media Data Technology 101

Snowflake

NOVEMBER 25, 2024

The twin revolutions in cloud technology and artificial intelligence are producing more data and analysis than ever before. By sharing knowledge — listening closely to people on the ground — and using technology, organizations can work toward meeting the UN’s Global Goals. These gaps are everywhere.

Machine Learning 98

Machine Learning Data Government Education 98

Analytics Vidhya

MARCH 26, 2023

There may be a number of reasons why you’d want to bring this rising technology […] The post How to Choose a Machine Learning Consulting Firm in 2023? Introduction Artificial intelligence (AI) and machine learning (ML) are in the best swing to help businesses sharpen their edge over their competitors in the market.

Consulting 239

Consulting Machine Learning Technology Data Analysis 239

The Pragmatic Engineer

NOVEMBER 21, 2023

Joshua is currently VP of Product & Strategy at VMware, a cloud computing and virtualization technology company. Joshua also writes an excellent Substack newsletter about how to design products which customers love, how to operate live services at scale, grow and optimize your technology orgs, and the history of the tech industry.

Engineering 325

Engineering Bytes Cloud Computing AWS 325

Monte Carlo

NOVEMBER 26, 2024

Operational use cases were rising to the surface, technology was reducing barriers to entry, and general artificial intelligence was obviously right around the corner. So, if it’s not generating revenue, AI needs to be cutting costs—and in that regard, this budding technology has certainly found some footing. Well, sort of.

Data Engineer 105

Data Engineer Data Engineering Engineering Unstructured Data 105

Snowflake

DECEMBER 4, 2024

In the enterprise technology space, both the greatest certainties and the most significant potential surprises come from one area: the rapidly advancing field of artificial intelligence. But businesses will continue to hesitate to put in front of customers a technology that may display bias or provide inaccurate responses.

Unstructured Data 97

Unstructured Data Data Lake Deep Learning Structured Data 97

The Pragmatic Engineer

JUNE 1, 2023

Juraj created a systems design explainer on how he built this project, and the technologies used: The systems design diagram for the Rides application The app uses: Node.js I also wanted to get better at setting up a full-stack project from scratch and understand why certain technologies are used. path-finding) from scratch.

Education 363

Education Project PostgreSQL Software Engineering 363

Analytics Vidhya

OCTOBER 6, 2023

Enter Microsoft Fabric, a cutting-edge solution designed to revolutionize how we interact with technology. In today’s rapidly evolving digital landscape, seamless data, applications, and device integration are more pressing than ever.

Designing 268

Designing Technology Data Lake Business Intelligence 268

Snowflake

JANUARY 8, 2025

Though AI is (still) the hottest technology topic, its not the overriding issue for enterprise security in 2025. Security teams will have to standardize their approaches to new AI technologies to make sure theyre as secure as their general enterprise infrastructure.

Data Lake 105

Data Lake Data Security Machine Learning Technology 105

Christophe Blefari

MARCH 15, 2024

AI News 🤖 Mira Murati answers the Wall Street Journal about OpenAI Sora — OpenAI CTO has been asked a few questions about the underlying technology in Sora. The technology under this, is, Cityvision. With yato you give a folder with SQL queries and it guesses the DAG and runs the queries in the right order.

Metadata 272

Metadata Data Datasets Data Warehouse 272

Snowflake

NOVEMBER 6, 2024

But as technology speeds forward, organizations of all sizes are realizing that generative AI isn’t just aspirational: It’s accessible and applicable now. You can begin to recognize that this technology, built with Cortex AI, can even start driving care, including suggesting and creating orders.

Hospitality 97

Hospitality Medical Government Software Engineering 97

Analytics Vidhya

FEBRUARY 23, 2023

Introduction Data analysts with the technological know-how to tackle challenging problems are data scientists. They collect, analyze, interpret data, and handle statistics, mathematics, and computer science. They are accountable for providing insights that go beyond statistical analyses.

Retail 258

Retail Computer Science Data Technology 258

Analytics Vidhya

MARCH 5, 2023

HDInsight works seamlessly with the Hadoop ecosystem, which includes technologies like MapReduce, Hive, […] The post Top 6 Microsoft HDFS Interview Questions appeared first on Analytics Vidhya. It is a fully managed cloud-based environment for analyzing and processing enormous volumes of data.

Hadoop 254

Hadoop Data Collection Cloud Technology 254

Analytics Vidhya

FEBRUARY 21, 2023

Introduction Nowadays, the corporate environment changes according to technology. Organizations are converting them to cloud-based technologies for the convenience of data collecting, reporting, and analysis.

Data Warehouse 211

Data Warehouse IT Data Collection Data 211

Precisely

NOVEMBER 18, 2024

A long-term approach to your data strategy is key to success as business environments and technologies continue to evolve. The rapid pace of technological change has made data-driven initiatives more crucial than ever within modern business strategies. Overall, AI success truly depends on a business outcome-driven approach. “We

Data Analytics 72

Data Analytics Data Governance Data Integration Government 72

Analytics Vidhya

APRIL 17, 2023

Introduction Well, hold onto your seats because the DataHour sessions are here to revolutionize how you learn about data-driven technologies. If you’re tired of boring, dry sessions that put you to sleep faster than a lullaby, you’re in for a treat.

Data Science 231

Data Science Data Technology Deep Learning 231

Snowflake

FEBRUARY 4, 2025

For the retail industry where change takes time the challenge is this: How do you harness a technology that is changing faster than you can adopt it? AI and advanced technologies can significantly transform inventory management. At this point, the pace of AI evolution is outstripping the news cycle.

Retail 98

Retail Deep Learning Technology Designing 98

Christophe Blefari

MARCH 1, 2023

Also dbt only does a pass-through to your underlying data compute technology, there is not any kind of processing within dbt. There are multiple technological reasons for this, but technology is rarely the real reason. Actually dbt can be seen as an orchestrator with no scheduling capabilities.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

Snowflake

MAY 22, 2024

TruEra’s technology helps evaluate the quality of inputs, outputs, and intermediate results of LLM apps. TruEra’s technology also provides detailed, actionable insights to improve ML model performance and accuracy by revealing anomalies in model metrics and providing a specific root cause analysis for rapid debugging.

Cloud 136

Cloud Data Governance Technology Machine Learning 136

Towards Data Science

DECEMBER 16, 2024

Operational use cases were rising to the surface, technology was reducing barriers to entry, and general artificial intelligence was obviously right around thecorner. So, if its not generating revenue, AI needs to be cutting costsand in that regard, this budding technology has certainly found somefooting. So did any of thathappen?

Unstructured Data 80

Unstructured Data Data Food Data Engineer 80

Snowflake

MARCH 12, 2025

Sports entity data teams are often mighty but small making complex technology solutions unrealistic to leverage. Legacy systems further complicate the situation, as outdated technologies lack the agility and data-sharing capabilities necessary for secure, seamless data collaboration across systems.

Media 70

Media Cloud Programming Data Collection 70

Data Engineering Podcast

FEBRUARY 18, 2024

Contact Info LinkedIn dain on GitHub Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Contact Info LinkedIn dain on GitHub Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Analytics Vidhya

MARCH 14, 2023

Introduction In today’s world, technology has increased tremendously, and many people are using the internet. This results in the generation of so much data daily. This generated data is stored in the database and will maintain it. SQL is a structured query language used to read and write these databases.

SQL 204

SQL Database Technology Data 204

Data Engineering Podcast

APRIL 21, 2024

Summary Generative AI has rapidly transformed everything in the technology sector. Contact Info LinkedIn Blog Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? When Andrew Lee started work on Shortwave he was focused on making email more productive.

Data Lake 182

Data Lake High Quality Data Machine Learning Data Pipeline 182

Snowflake

FEBRUARY 13, 2025

Lastly, companies have historically collaborated using inefficient and legacy technologies requiring file retrieval from FTP servers, API scraping and complex data pipelines. To remain nimble and have the flexibility to use the best tools for different workstreams, customers should not be locked into a specific vendor or technology.

Management 92

Management Government Cloud Unstructured Data 92

The Pragmatic Engineer

SEPTEMBER 22, 2023

Technological innovation rarely happens in a vacuum; it builds on previous technologies. The license of a project is key to understanding how ‘risky’ it is to use. Bun uses an extremely permissive MIT license that minimizes any risk.

Programming Language 281

Programming Language Project Coding Engineering 281