Edureka

FEBRUARY 8, 2024

Introduction to AWS Instances Selecting the right AWS instance type is a critical decision that can significantly influence the success of your cloud-based applications and infrastructure. Enroll for the AWS Training today to learn more about its instances in detail.

AWS 52

AWS NoSQL Deep Learning Datasets 52

Rockset

FEBRUARY 2, 2022

MongoDB is a top database choice for application development. Developers choose this database because of its flexible data model and its inherent scalability as a NoSQL database. MongoDB wasn’t originally developed with an eye on high performance for analytics. Yet, analytics is now a vital part of modern data applications.

MongoDB 52

MongoDB NoSQL Data Warehouse BI 52

Data Engineering Podcast

SEPTEMBER 11, 2022

Summary Any business that wants to understand their operations and customers through data requires some form of pipeline. Building reliable data pipelines is a complex and costly undertaking with many layered requirements. Data stacks are becoming more and more complex. Sifflet also offers a 2-week free trial.

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Data Engineering Podcast

AUGUST 21, 2022

Summary Data has permeated every aspect of our lives and the products that we interact with. In this episode Shruti Bhat gives her view on the state of the ecosystem for real-time data and the work that she and her team at Rockset is doing to make it easier for engineers to build those experiences.

Lambda Architecture 100

Lambda Architecture MongoDB Scala MySQL 100

Data Engineering Podcast

JUNE 12, 2022

Summary Unstructured data takes many forms in an organization. From a data engineering perspective that often means things like JSON files, audio or video recordings, images, etc. Another category of unstructured data that every business deals with is PDFs, Word documents, workstation backups, and countless other types of information.

Unstructured Data 130

Unstructured Data MongoDB Scala MySQL 130

Data Engineering Podcast

NOVEMBER 6, 2022

Summary Despite the best efforts of data engineers, data is as messy as the real world. Entity resolution and fuzzy matching are powerful utilities for cleaning up data from disconnected sources, but it has typically required custom development and training machine learning models.

MongoDB 130

MongoDB Scala MySQL Machine Learning 130

U-Next

MARCH 1, 2023

Introduction Data Engineer is responsible for managing the flow of data to be used to make better business decisions. A solid understanding of relational databases and SQL language is a must-have skill, as an ability to manipulate large amounts of data effectively. In 2022, data engineering will hold a share of 29.8%

Data Engineering 75

Data Engineering Data Engineer Non-relational Database Engineering 75

Data Engineering Podcast

JULY 24, 2022

Summary The current stage of evolution in the data management ecosystem has resulted in domain and use case specific orchestration capabilities being incorporated into various tools. In this episode Nick Schrock discusses the importance of orchestration and a central location for managing data systems, the road to Dagster’s 1.0

MongoDB 100

MongoDB Scala MySQL Data Lake 100

Data Engineering Podcast

OCTOBER 16, 2022

Summary The "data lakehouse" architecture balances the scalability and flexibility of data lakes with the ease of use and transaction support of data warehouses. Enter Metaplane, the industry’s only self-serve data observability tool. Data teams are increasingly under pressure to deliver.

Data Lake 100

Data Lake Food MongoDB Scala 100

Data Engineering Podcast

NOVEMBER 13, 2022

Summary CreditKarma builds data products that help consumers take advantage of their credit and financial capabilities. To make that possible they need a reliable data platform that empowers all of the organization’s stakeholders. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

MongoDB 100

MongoDB Scala MySQL Google Cloud 100

Data Engineering Podcast

JULY 3, 2022

Summary The perennial challenge of data engineers is ensuring that information is integrated reliably. In order to quickly identify if and how two data systems are out of sync Gleb Mezhanskiy and Simon Eskildsen partnered to create the open source data-diff utility. does exactly that.

Data Integration 100

Data Integration MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 25, 2022

Summary Data integration from source systems to their downstream destinations is the foundational step for any data product. With the increasing expecation for information to be instantly accessible, it drives the need for reliable change data capture. Data teams are increasingly under pressure to deliver.

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

JULY 17, 2022

Summary Data engineering is a large and growing subject, with new technologies, specializations, and "best practices" emerging at an accelerating pace. RudderStack helps you build a customer data platform on your warehouse or data lake. Data teams are increasingly under pressure to deliver. In fact, while only 3.5%

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Data Engineering Podcast

JULY 10, 2022

Summary Building and maintaining reliable data assets is the prime directive for data engineers. While it is easy to say, it is endlessly complex to implement, requiring data professionals to be experts in a wide range of disparate topics while designing and implementing complex topologies of information workflows.

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Data Engineering Podcast

JUNE 5, 2022

Summary The best way to make sure that you don’t leak sensitive data is to never have it in the first place. The team at Skyflow decided that the second best way is to build a storage system dedicated to securely managing your sensitive information and making it easy to integrate with your applications and data systems.

Data Security 100

Data Security Metadata MongoDB Scala 100

Data Engineering Podcast

AUGUST 13, 2022

Summary Data is useless if it isn’t being used, and you can’t use it if you don’t know where it is. Data catalogs were the first solution to this problem, but they are only helpful if you know what you are looking for. Data stacks are becoming more and more complex. Sifflet also offers a 2-week free trial.

Metadata 100

Metadata MongoDB Scala MySQL 100

Data Engineering Podcast

OCTOBER 2, 2022

Summary Data lineage is something that has grown from a convenient feature to a critical need as data systems have grown in scale, complexity, and centrality to business. Alvin is a platform that aims to provide a low effort solution for data lineage capabilities focused on simplifying the work of data engineers.

IT 100

IT Food PostgreSQL MongoDB 100

Data Engineering Podcast

JUNE 26, 2022

Summary The most complicated part of data engineering is the effort involved in making the raw data fit into the narrative of the business. Master Data Management (MDM) is the process of building consensus around what the information actually means in the context of the business and then shaping the data to match those semantics.

Data Management 100

Data Management Management MongoDB Scala 100

Rockset

JULY 8, 2022

June was a month packed with big data and analytics conferences, and we kicked the summer off with the trifecta of MongoDB World in New York, Snowflake Summit in Las Vegas and The Databricks Data+AI Summit in San Francisco. You won't believe your ability to build apps in weeks, not months, regardless of the shape of the data.

MongoDB 52

MongoDB AWS Data Lake Big Data 52

Cloud Academy

JANUARY 27, 2022

Data engineering is the process of designing and implementing solutions to collect, store, and analyze large amounts of data. The data then gets prepared in formats to be used by people such as business analysts, data analysts, and data scientists. What does a data engineer do?

Certification 52

Certification Data Engineering Data Engineer Engineering 52

Striim

AUGUST 26, 2023

While Confluent is a well-known option for data streaming platforms, its complexity can pose significant challenges for businesses. This complexity not only has technical repercussions but also impacts business operations by leading to unreliable or missing data, which can be both costly and time-consuming to rectify.

MongoDB 52

MongoDB Google Cloud Kafka AWS 52

Data Engineering Podcast

AUGUST 28, 2022

Summary AirBnB pioneered a number of the organizational practices that have become the goal of modern data teams. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs.

Building 100

Building MongoDB Scala MySQL 100

Knowledge Hut

APRIL 25, 2023

The tremendous growth in data generation, then the rise in data engineer jobs - there’s no arguing the fact that the big data industry is at its best pace and you, as an aspiring data engineer, have a lot to learn and make out of it - including some tools! While they go about it - enter big data data engineer tools.

Data Engineering 52

Data Engineering Data Engineer Engineering Google Cloud 52

Monte Carlo

NOVEMBER 20, 2023

Far from a mere “copy-paste” job, data migrations can be deceptively tricky and one of the most challenging aspects of a data engineer’s job. But as businesses pivot and technologies advance, data migrations are—regrettably—unavoidable. Table of Contents What makes data migrations complex? Set clear objectives.

MongoDB 52

MongoDB ETL Tools Database Data 52

ProjectPro

JULY 23, 2021

AWS (Amazon Web Services) is the world’s leading and widely used cloud platform, with over 200 fully featured services available from data centers worldwide. This blog presents some of the most unique and innovative AWS projects from beginner to advanced levels. Table of Contents What is AWS?

AWS 52

AWS Project Amazon Web Services Cloud Computing 52

Data Engineering Podcast

AUGUST 13, 2022

Summary Data engineers have typically left the process of data labeling to data scientists or other roles because of its nature as a manual and process heavy undertaking, focusing instead on building automation and repeatable systems. Data stacks are becoming more and more complex. Sifflet also offers a 2-week free trial.

Machine Learning 100

Machine Learning Pipeline-centric Database-centric MongoDB 100

Data Engineering Podcast

OCTOBER 30, 2022

Summary One of the most impactful technologies for data analytics in recent years has been dbt. It’s hard to have a conversation about data engineering or analysis without mentioning it. Despite its widespread adoption there are still rough edges in its workflow that cause friction for data analysts.

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Podcast

JULY 3, 2022

Summary The ecosystem for data tools has been going through rapid and constant evolution over the past several years. These technological shifts have brought about corresponding changes in data and platform architectures for managing data and analytical workflows. Atlan is the metadata hub for your data ecosystem.

Architecture 100

Architecture Metadata MongoDB MySQL 100

Workfall

APRIL 6, 2023

Clients can subscribe to specific data events on the server using GraphQL Subscriptions, such as the creation of a new resource or the updating of an existing one. Hands-On The repository for the code used in this blog is at [link] Pre-requisites First of all, you have to ensure that MongoDB is installed on your machine.

MongoDB 59

MongoDB Building Transportation PostgreSQL 59

Data Engineering Podcast

DECEMBER 11, 2022

Summary One of the most critical aspects of software projects is managing its data. Managing the operational concerns for your database can be complex and expensive, especially if you need to scale to large volumes of data, high traffic, or geographically distributed usage.

Database 100

Database MySQL Data Lake MongoDB 100

Data Engineering Podcast

OCTOBER 9, 2022

In this episode Purvi Shah, the VP of Enterprise Big Data Platforms at American Express, explains how they have invested in the cloud to power this visibility and the complex suite of integrations they have built and maintained across legacy and modern systems to make it possible. Data teams are increasingly under pressure to deliver.

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 4, 2022

Sust Global was created to provide curated data sets for organizations to be able to analyze climate information in the context of their business needs. Data stacks are becoming more and more complex. All thanks to 50+ quality checks, extensive column-level lineage, and 20+ connectors across the Data Stack.

MongoDB 100

MongoDB Scala MySQL Data Ingestion 100

Data Engineering Podcast

SEPTEMBER 18, 2022

He discusses the challenges of making data collection purposeful and efficient without distracting employees from their primary duties and how business owners can use the provided analytics to support their staff in their duties. RudderStack helps you build a customer data platform on your warehouse or data lake.

Hospitality 100

Hospitality Food MongoDB Scala 100

Data Engineering Podcast

AUGUST 6, 2022

Summary The optimal format for storage and retrieval of data is dependent on how it is going to be used. For analytical systems there are decades of investment in data warehouses and various modeling techniques. Data stacks are becoming more and more complex.

Machine Learning 100

Machine Learning Database MySQL PostgreSQL 100

Knowledge Hut

JUNE 23, 2023

Data engineers make a tangible difference with their presence in top-notch industries, especially in assisting data scientists in machine learning and deep learning. Let us understand here the complete big data engineer roadmap to lead a successful Data Engineering Learning Path.

Data Engineering 52

Data Engineering Data Engineer Engineering Non-relational Database 52

Data Science Blog: Data Engineering

DECEMBER 23, 2022

Warum Data Engineering der Data Science in Bedeutung und Berufschancen längst die Show stiehlt, dabei selbst ebenso einem stetigen Wandel unterliegt. Was ein Data Engineer wirklich können muss Der Data Scientist als sexiest Job des 21. Data Engineer, der gefragteste Job des 21sten Jahrhunderts? Jahrhunderts?

Data Engineering 52

Data Engineering Data Engineer Engineering Business Intelligence 52

Knowledge Hut

JULY 26, 2023

Database applications have become vital in current business environments because they enable effective data management, integration, privacy, collaboration, analysis, and reporting. Database applications also help in data-driven decision-making by providing data analysis and reporting tools. What are Database Applications?

Database 52

Database NoSQL Telecommunication MongoDB 52