dbt Developer Hub

NOVEMBER 3, 2022

Let’s discuss how to convert events from an event-driven microservice architecture into relational tables in a warehouse like Snowflake. Quality problems lead to first responders unable to check into disaster sites or parents unable to access ESA funds. So our solution was to start using an intentional contract: Events.

Kafka 52

Kafka ETL Tools BI Database 52

Workfall

SEPTEMBER 26, 2023

Reading Time: 9 minutes In this blog, we will cover: What are Server-Sent Events? Why Stream Data Using Server-Sent Events (SSE)? Hands-On Conclusion What are Server-Sent Events? These events can carry information like notifications, updates, or alerts. This makes it accessible to a wider range of developers and projects.

Python 85

Python Bytes Coding Data 85

Data Engineering Podcast

NOVEMBER 11, 2019

Summary The practice of data management is one that requires technical acumen, but there are also many policy and regulatory issues that inform and influence the design of our systems. Upcoming events include the combined events of the Data Architecture Summit and Graphorum, the Data Orchestration Summit, and Data Council in NYC.

Designing 100

Designing Data Pipeline Programming Language Data 100

Data Engineering Podcast

AUGUST 10, 2020

Summary Event based data is a rich source of information for analytics, unless none of the event structures are consistent. The team at Iteratively are building a platform to manage the end to end flow of collaboration around what events are needed, how to structure the attributes, and how they are captured.

Data Collection 100

Data Collection Data Engineering Data Engineer Education 100

Knowledge Hut

MARCH 19, 2024

There are a lot of opportunities waiting for UX design enthusiasts, provided they consider moving to Germany. Upon moving here, you will experience a diversified pool of opportunities in digital product companies, design agencies, startups, etc. Who is a UX Designer?

Designing 52

Designing Portfolio Certification Project 52

Data Engineering Podcast

FEBRUARY 18, 2024

Join in with the event for the global data community, Data Council Austin. Don't miss out on their only event this year! What are the differences in terms of pipeline design/access and usage patterns when using a Trino/Iceberg lakehouse as compared to other popular warehouse/lakehouse structures?

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Rockset

OCTOBER 4, 2022

Event streams, such as clickstreams, IoT data and other time series data, are common sources of data into these apps. The broad adoption of Apache Kafka has helped make these event streams more accessible. In this post, we compare two options for real-time analytics on event and CDC streams: Rockset and ClickHouse.

MySQL 52

MySQL Kafka Aggregated Data Architecture 52

Data Engineering Podcast

APRIL 23, 2023

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. What are the shifts that have made them more accessible to a wider variety of teams? RudderStack also supports real-time use cases.

Data Lake 277

Data Lake Kafka Machine Learning Data Warehouse 277

Data Engineering Podcast

FEBRUARY 25, 2024

Join us at the top event for the global data community, Data Council Austin. Don't miss out on our only event this year! What were the design goals and constraints that led you to this architecture? Data Council Logo]([link] Join us at the top event for the global data community, Data Council Austin.

Database 162

Database Technology Data Lake High Quality Data 162

Data Engineering Podcast

MARCH 10, 2024

Join us at the top event for the global data community, Data Council Austin. Don't miss out on our only event this year! What are the core problems/complexities that Nessie is designed to solve? How have the design and goals of the project changed since it was first created?

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

Knowledge Hut

APRIL 27, 2023

A UI/UX designer is a professional who is responsible for creating intuitive and engaging interfaces for digital products such as websites, mobile applications, software, and other digital platforms. A skilled UI/UX designer is able to create a design that not only looks great but is also user-friendly, accessible, and functional.

Designing 52

Designing Portfolio Education Digital Media 52

Data Engineering Podcast

NOVEMBER 26, 2023

Summary Building a data platform that is enjoyable and accessible for all of its end users is a substantial challenge. Developing event-driven pipelines is going to be a lot easier - Meet Functions! Memphis Logo]([link] Developing event-driven pipelines is going to be a lot easier - Meet Functions!

Architecture 147

Architecture Data Lake High Quality Data SQL 147

Snowflake

JUNE 28, 2023

External Network Access (PrPr) – Allows users to seamlessly connect to external endpoints from their Snowpark code (UDFs/UDTFs and Stored procedures) while maintaining high security and governance. Python Tasks API (PrPr Soon) – Provides first class Python APIs for creating and managing Snowflake Tasks/DAGs.

Python 52

Python Accessible Accessibility Pipeline-centric 52

Data Engineering Podcast

MARCH 3, 2024

Join us at the top event for the global data community, Data Council Austin. Don't miss out on our only event this year! What are the foundational/fundamental principles that you need to design around to allow for future flexibility? Don't miss out on our only event this year! Data Council : ![Data

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Rockset

FEBRUARY 24, 2022

Event-based architectures have been gaining popularity for some time. With increased adoption has come a flood of options for aggregating and analyzing events. Which databases are optimized for ingesting streaming events and analyzing them in real time? Supporting separation of data access (read/write) mechanisms like CQRS.

AWS 52

AWS Amazon Web Services Kafka SQL 52

Snowflake

APRIL 29, 2024

This strategy must incorporate business impact analyses as well as backup and recovery plans in the event of a security incident or loss of access to data. ESA will designate critical ICT providers in January 2025. All designated critical ICT third-party service providers will be subject to direct oversight by ESA.

Transportation 79

Transportation Data Governance Government Consulting 79

Data Engineering Podcast

DECEMBER 24, 2023

Summary Kafka has become a ubiquitous technology, offering a simple method for coordinating events and data across different systems. In the event of these different cluster errors, what are the strategies for mitigating and recovering from those failures? Operating it at scale, however, is notoriously challenging.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Knowledge Hut

MARCH 22, 2024

These locks are designed to automatically unlock during emergencies or power outages, ensuring swift evacuation or access for first responders. These sophisticated locks operate on the principle of maintaining secure access control, even in the event of power failure or system malfunction.

Systems 105

Systems Electronics Hospitality Architecture 105

Snowflake

APRIL 11, 2024

At Snowflake, we believe in making the power of data accessible to all. Seamless AI within Snowflake Copilot is a complete solution designed to seamlessly integrate with your existing Snowflake workflow. Please note: Public preview access for Snowflake Copilot is currently rolling out for AWS accounts.

SQL 134

SQL AWS Data Analysis High Quality Data 134

Robinhood

MARCH 26, 2024

The event is available to livestream on go.robinhood.com/presents at 7:15 pm ET on March 26 and will be available for replay. It’s why we started Robinhood and gave our customers access to features like commission free trading and the 24 Hour Market,” said Vlad Tenev, Co-Founder and CEO of Robinhood. © 2024 Robinhood.

Banking 120

Banking Insurance Finance Programming 120

Data Engineering Podcast

JANUARY 28, 2024

Summary Monitoring and auditing IT systems for security events requires the ability to quickly analyze massive volumes of unstructured log data. What are the data acquisition paths/sources that you are designed to work with?- How have the design and goals of the product changed since you first started working on it?

Data Lake 147

Data Lake Building High Quality Data AWS 147

Confluent

FEBRUARY 21, 2019

During a recent talk titled Hunters ATT&CKing with the Right Data , which I presented with my brother Jose Luis Rodriguez at ATT&CKcon, we talked about the importance of documenting and modeling security event logs before developing any data analytics while preparing for a threat hunting engagement. Yeah…I can do that already!

Process 80

Process Kafka Datasets SQL 80

The Pragmatic Engineer

NOVEMBER 21, 2023

Joshua also writes an excellent Substack newsletter about how to design products which customers love, how to operate live services at scale, grow and optimize your technology orgs, and the history of the tech industry. ARPANET was designed to maintain communications during outages. Subscribe here.

Engineering 283

Engineering Bytes Cloud Computing AWS 283

Towards Data Science

FEBRUARY 18, 2024

In the first section, I present an introductory overview of stream processing concepts, drawing extensively from the book Designing Data-Intensive Applications [1]. An event is a small, self-contained object that contains the details of something happened at some point in time e.g. user interaction. online dashboard).

Kafka 80

Kafka Python Process Google Cloud 80

Snowflake

APRIL 17, 2024

Yet while SQL applications have long served as the gateway to access and manage data, Python has become the language of choice for most data teams, creating a disconnect. While the Python API connector remains available for specific SQL use cases, the new API is designed to be your go-to solution.

Data Pipeline 100

Data Pipeline Python Data Engineering Data Engineer 100

Data Engineering Podcast

MARCH 24, 2024

What are the most interesting, unexpected, or challenging lessons that you have learned while working on the design and launch of Dagster+? What are the most interesting, unexpected, or challenging lessons that you have learned while working on the design and launch of Dagster+? When is Dagster+ the wrong choice?

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

The Pragmatic Engineer

JANUARY 12, 2023

Pollen was an events tech startup founded in 2015, which raised more than $200M in funding and employed about 600 people by 2022. It defied gravity by appearing to thrive at the same time as the Covid-19 pandemic shut down swathes of the events industry, worldwide. To get this newsletter every week, subscribe here.

Software Engineer 181

Software Engineer Software Engineering Engineering Raw Data 181

Data Engineering Podcast

MARCH 18, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Join in with the event for the global data community, Data Council Austin. Don't miss out on their only event this year! How has the scope and complexity of implementing security controls on data systems changed in recent years?

Data Security 130

Data Security Machine Learning Data Science Metadata 130

Knowledge Hut

DECEMBER 22, 2023

Voice Search Benefits: Improved User Experience Accessibility Hands-free Interaction Faster Search Results Examples: Integrating voice-enabled search boxes or chatbots on websites to allow users to perform voice-based searches. Event websites featuring event details, schedules, and registration forms on a single page.

Entertainment 105

Entertainment Utilities Machine Learning AWS 105

Snowflake

FEBRUARY 12, 2024

One way telecom companies stay ahead of the curve is by attending the world’s largest and most influential telecom and connectivity event: Mobile World Congress (MWC) 2024, held February 26 – 29 in Barcelona, Spain.

Python 78

Python Manufacturing Cloud Government 78

Netflix Tech

MARCH 10, 2023

Studio applications use this service to store their media assets, which then goes through an asset cycle of schema validation, versioning, access control, sharing, triggering configured workflows like inspection, proxy generation etc. This pattern grows over time when we need to access and update the existing assets metadata.

Management 94

Management Kafka Metadata Media 94

Knowledge Hut

JANUARY 10, 2024

Whether you're a startup founder, a seasoned developer, or an aspiring cloud enthusiast, having access to AWS credits can be a game-changer. This includes access to AWS activate credits, technical support, Cloud Engineer training , and other valuable resources. But how can you secure these valuable resources without breaking the bank?

AWS 52

AWS Amazon Web Services Education Cloud Computing 52

Snowflake

JANUARY 29, 2024

This new connector offers immediate access to your Google Analytics data without the hassle, complexity and expense of manual integration via API endpoints and patched systems workarounds. Once your data reaches its final destination, access to the ingested data is governed by Snowflake’s built-in access controls.

Raw Data 105

Raw Data Aggregated Data Data Government 105

The Pragmatic Engineer

APRIL 9, 2024

On top of the ability to post jobs, they had access to The Pragmatic Engineer Talent Collective. As I analyzed this event in The Pragmatic Engineer: Every hiring manager I talked with told me how Stack Overflow Jobs has been the place where job ads got good candidates.

Software Engineer 256

Software Engineer Software Engineering Engineering Recruitment 256

Snowflake

APRIL 18, 2024

Legacy security information and event management (SIEM) solutions, like Splunk, are powerful tools for managing and analyzing machine-generated data. Snowflake allows security teams to store all their data in a single platform and maintain it all in a readily accessible state, with virtually unlimited cloud data storage capacity.

Data Lake 75

Data Lake Data Ingestion Bytes Cloud Computing 75

Zalando Engineering

NOVEMBER 8, 2023

Postgres Logical Replication at Zalando Builders at Zalando have access to a low-code solution that allows them to declare event streams that source from Postgres databases. At the time of writing, there are hundreds of these Postgres-sourced event streams out in the wild at Zalando. Simple, right?

PostgreSQL 139

PostgreSQL Java Database Bytes 139

ThoughtSpot

JANUARY 16, 2024

ThoughtSpot User Groups were designed to help users connect—a place where you can share stories and get new ideas to empower your organization with data. As a member, you also gain access to a wealth of information, tips, and tricks built to enhance your ThoughtSpot user experience.

Business Intelligence 76

Business Intelligence Consulting Certification Utilities 76