The Pragmatic Engineer

MARCH 19, 2024

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover one out of three topics from last week’s subscriber-only The Pulse issue. Today, full subscribers got access to a comprehensive Senior-and-above tech compensation research.

Software Engineer 255

Software Engineer Software Engineering Programming Language Media 255

Data Engineering Podcast

MARCH 17, 2024

Summary A significant portion of data workflows involve storing and processing information in database engines. Validating that the information is stored and processed correctly can be complex and time-consuming, especially when the source and destination speak different dialects of SQL. In this episode Gleb Mezhanskiy, founder and CEO of Datafold, discusses the different error conditions and solutions that you need to know about to ensure the accuracy of your data.

Database 147

Database Data Lake High Quality Data Data Workflow 147

Christophe Blefari

MARCH 22, 2024

Friday routine ( credits ) It's Friday and it's Data News. I don't go into too much detail about the magic of Data News, but every Friday is the same. At first, I'm: oh s**t, here we go again and 10 minutes later I'm lost in reading the content and picking too many articles to fit into a thousand word edition. Usually all the process takes me a whole Friday.

Electronics 130

Electronics Media Data Python 130

Waitingforcode

MARCH 22, 2024

Apache Spark leverages the observer design pattern for the framework-to-code communication. One of the consumers' implementations is StreamingQueryListener.

Coding 130

Coding Designing 130

Advertisement

Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.

Database

Confluent

MARCH 19, 2024

Seamlessly integrate Apache Kafka data into your lakehouse as Apache Iceberg tables, bridging the operational and analytical divide, with Tableflow. Read more in our blog post.

Kafka 132

Kafka Data 132

Robinhood

MARCH 20, 2024

Robinhood Wallet is a portal to the world of web3, giving users full ownership and control of their crypto Today, we are excited to release Robinhood Wallet to all eligible Android users globally, expanding on our mission to make Robinhood the most trusted and easiest way to use crypto. All customers who previously joined the waitlist can download and get started today.

Finance 130

Finance Insurance Accessible Accessibility 130

KDnuggets

MARCH 20, 2024

Are you curious about what it takes to become a professional data scientist? Look no further! By following these guides, you can transform yourself into a skilled data scientist and unlock endless career opportunities.

Data Analysis 124

Data Analysis SQL Python Data 124

Engineering at Meta

MARCH 21, 2024

Threads has entered the fediverse! As part of our beta experience, now available in a few countries, Threads users aged 18+ with public profiles can now choose to share their Threads posts to other ActivityPub-compliant servers. People on those servers can now follow federated Threads profiles and see, like, reply to, and repost posts from the fediverse.

Media 124

Media Building Government Engineering 124

Snowflake

MARCH 21, 2024

Snowflake is committed to helping our customers unlock the power of artificial intelligence (AI) to drive better decisions, improve productivity and reach more customers using all types of data. Large Language Models ( LLMs ) are a critical component of generative AI applications, and multimodal models are an exciting category that allows users to go beyond text and incorporate images and video into their prompts to get a better understanding of the context and meaning of the data.

Government 116

Government Technology Cloud Building 116

Confessions of a Data Guy

MARCH 18, 2024

Want to know how to grow to the Senior Engineering position? Take a look. The post The Path To Senior Engineer appeared first on Confessions of a Data Guy.

Engineering 100

Engineering Data Data Engineering Data Engineer 100

Speaker: Scott Sehlhorst

We know we want to create products which our customers find to be valuable. Whether we label it as customer-centric or product-led depends on how long we've been doing product management. There are three challenges we face when doing this. The obvious challenge is figuring out what our users need; the non-obvious challenges are in creating a shared understanding of those needs and in sensing if what we're doing is meeting those needs.

Certification

KDnuggets

MARCH 22, 2024

GenAI has enabled new search engine platforms with unique features and advantages, challenging Google's dominance.

Engineering 139

Engineering 139

Cloudera

MARCH 22, 2024

As enterprise AI technologies rapidly reshape our digital environment, the foundation of your cloud infrastructure is more critical than ever. That’s why Cloudera and Red Hat , renowned for their open-source solutions, have teamed up to bring Red Hat Enterprise Linux ( RHEL ) to Cloudera on public cloud as the operating system for all of our public cloud platform images.

Cloud 112

Cloud Certification Systems Architecture 112

Confluent

MARCH 21, 2024

Learn the best practices for using Confluent Schema Registry, including using schema IDs, understanding subjects and versions, using data contracts, pre-registering schemas, and more.

Data 109

Data 109

Snowflake

MARCH 19, 2024

A company’s data strategy is always in motion. Since the explosion of interest in generative AI and large language models (LLMs), that is more true than ever, with business leaders discussing how quickly they should adopt these technologies to stay competitive. Some emerging approaches may be seen in our newly released Snowflake Data Trends 2024 , looking at how users in the Data Cloud are working with their data.

Data Governance 109

Data Governance Government Unstructured Data Programming Language 109

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

Data Science

Robinhood

MARCH 21, 2024

President and CEO of Americas Society and Council of the Americas joins the Robinhood Board of Directors We’re excited to share that Susan Segal has joined Robinhood Markets’ Board of Directors. “Susan is widely recognized for her work and extensive contributions in Latin America as well as in the financial sector,” said Vlad Tenev, CEO and Co-Founder of Robinhood.

IT 107

IT Banking Finance Accessible 107

KDnuggets

MARCH 19, 2024

Statistics is a must-have skill for data science. And here are 5 free books that’ll help you learn all the statistics you need as a data professional.

Data Science 115

Data Science Data 115

databricks

MARCH 21, 2024

We are excited to announce the release of the Databricks AI Security Framework (DASF) version 1.0 whitepaper! The framework is designed to improve.

Designing 111

Designing 111

WeCloudData

MARCH 20, 2024

Machine learning is revolutionizing traffic prediction, enhancing route planning and reducing congestion in urban commuting. Explore advanced algorithms like Uni-LSTM and BiLSTM for accurate forecasts, along with Google Maps' integration of deep learning for improved ETA accuracy. Discover the practical utility of machine learning in everyday life. The post Navigating your way: Traffic Prediction with Machine Learning appeared first on WeCloudData.

Machine Learning 97

Machine Learning Deep Learning Algorithm Utilities 97

Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage

Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.

Building

Knowledge Hut

MARCH 20, 2024

Employees who are in the profession today will change occupations considerably more much of the time than in past time. Thus, you should be aware and proactive in dealing with your profession. Yet, does this mean you need to yield in different parts of your life that are important to you? Our lives are sufficiently occupied balancing work and family without finding time for making out significant profession improvements also.

Utilities 94

Utilities Building Systems Management 94

KDnuggets

MARCH 20, 2024

Learn data science according to your expertise with these 4 different learning roadmaps.

Data Science 145

Data Science Data 145

databricks

MARCH 21, 2024

Benchmarking for training (dense) models at scale. We demonstrate great performance (very high MFU) and highlight our use of NVIDIA's Transformer Engine, along with PyTorch FSDP and DTensor.

Engineering 95

Engineering 95

Engineering at Meta

MARCH 18, 2024

Systems and application logs play a key role in operations, observability, and debugging workflows at Meta. Logarithm is a hosted, serverless, multitenant service, used only internally at Meta, that consumes and indexes these logs and provides an interactive query interface to retrieve and view logs. In this post, we present the design behind Logarithm, and show how it powers AI training debugging use cases.

Engineering 89

Engineering Metadata Architecture Designing 89

Advertisement

Are you trying to decide which entity resolution capabilities you need? It can be confusing to determine which features are most important for your project. And sometimes key features are overlooked. Get the Entity Resolution Evaluation Checklist to make sure you’ve thought of everything to make your project a success! The list was created by Senzing’s team of leading entity resolution experts, based on their real-world experience.

Project

Knowledge Hut

MARCH 19, 2024

Logical thinking skills play a significant role in developing careers because they help you reason through vital decisions, generate creative ideas, set goals, and solve problems. You may encounter multiple challenges in your life when you enter the job industry or advance your career. Therefore, need strong logical reasoning skills to solve your problems.

Education 94

Education Certification Process Programming 94

KDnuggets

MARCH 19, 2024

Check out this beginner’s guide to understanding the role of Large Language Model Operations for seamless user experiences.

121

121

Robinhood

MARCH 18, 2024

The largest UK brokers typically charge UK investors, with a £10,000 portfolio, an average of £240 per year to invest in US stocks*–Robinhood offers no commission fees and no foreign exchange (FX) fees on trades.** Today, we’ve rolled all eligible customers off our waitlist and Robinhood is now officially available throughout the United Kingdom. With Robinhood, customers simply get more for their money, with no commission fees and no foreign exchange (FX) fees on trades, access to more than 6,00

Insurance 87

Insurance Portfolio Education Banking 87

Engineering at Meta

MARCH 20, 2024

At Meta, we support real-time communication (RTC) for billions of people through our apps, including Messenger, Instagram, and WhatsApp. We’ve seen significant benefits by adopting the AV1 codec for RTC. Here’s how we are improving the RTC video quality for our apps with tools like the AV1 codec, the challenges we face, and how we mitigate those challenges.

Algorithm 86

Algorithm Manufacturing Media Coding 86

Speaker: Anne Steiner and David Laribee

As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.

Engineering

Knowledge Hut

MARCH 22, 2024

When I worked in the hospitality industry, the electricity abruptly went out while we were establishing the network and door locks. To my surprise, the door lock opened like any other door. This was the first time in my life that I had ever seen a fail-safe door lock. I have comprehensively analyzed the area of physical security, particularly the ongoing discussion surrounding fail safe vs fail-safe secure electric strike locking systems.

Systems 92

Systems Electronics Hospitality Architecture 92

KDnuggets

MARCH 21, 2024

MetaGPT's newest agent addition makes running data interpretation and analysis tasks a breeze. Find out more and give it a try for yourself.

Data Solutions 107

Data Solutions Data IT 107

Confluent

MARCH 19, 2024

Confluent Cloud's serverless Flink offering is now available on all major clouds, offering a unified, managed platform for real-time data processing.

Cloud 93

Cloud Data Process Process Management 93

Cloudera

MARCH 22, 2024

We’re excited to announce that Cloudera has been named the Best Medium Workplace in Ireland , one of the Best Workplaces in Costa Rica , and one of Ireland’s Best Workplaces for Women for 2024. These recognitions underscore Cloudera’s ongoing efforts to prioritize employee well-being, professional development, and collaborative work environments.

Finance 81

Finance Cloud Data Management Management 81

Speaker: Margaret-Ann Seger, Head of Product, Statsig

Experimentation is often seen as an aspirational practice, especially at smaller, fast-moving companies who are strapped for time and resources. So, how can you get your team making decisions in a more data-driven way while continuing to remain lean and maintaining ship velocity? In this webinar, Margaret-Ann Seger, Head of Product at Statsig, will teach you how to build an experimentation culture from the ground-up, graduating from just getting started with data-driven development to operating

Building