Start Data Engineering
JULY 21, 2021
Introduction Using Native Python Using Pydantic Pydantic Caveats Conclusion Further reading References Introduction Data type issues are one of the biggest concerns when processing data in python. If you are wondering how to Make sure that a column is of a specific data type ( e.g.
Uber Engineering
JULY 22, 2021
Introduction. As Uber’s business grew, we scaled our Apache Hadoop (referred to as ‘Hadoop’ in this article) deployment to 21000+ hosts in 5 years, to support the various analytical and machine learning use cases. We built a team with varied … The post Containerizing Apache Hadoop Infrastructure at Uber appeared first on Uber Engineering Blog.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Confluent
JULY 20, 2021
We’re pleased to announce ksqlDB 0.19.0! This release includes a new NULLIF function and a major upgrade to ksqlDB’s data modeling capabilities—foreign-key joins. We’re excited to share this highly requested […].
Data Engineering Podcast
JULY 19, 2021
Summary Data quality is a concern that has been gaining attention alongside the rising importance of analytics for business success. Many solutions rely on hand-coded rules for catching known bugs, or statistical analysis of records to detect anomalies retroactively. While those are useful tools, it is far better to prevent data errors before they become an outsized issue.
Speaker: Timothy Chan, PhD., Head of Data Science
Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.
Cloudera
JULY 19, 2021
Meet Veda Kadam. She’s relatively new to the Cloudera family. She started her journey here in June of 2020 when she joined our first ever fully virtual intern program. Now she’s a full time employee working as a Software Engineer on our Data In Motion team. From an early age, Veda knew she wanted to work in the technology industry. Her father worked in pharmaceuticals and her mother worked in accounting.
DataKitchen
JULY 21, 2021
The post Does Your Organization Need a Chief Data Officer? Probably first appeared on DataKitchen.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Teradata
JULY 22, 2021
Learn about the techniques and frameworks needed to build a more resilient, cost-effective, and efficient data & analytic decisioning support capability for the post-pandemic supply chain.
Cloudera
JULY 21, 2021
In the build-up to this year’s Data Impact Awards, we’re looking back at last year’s winners. We are reflecting on their accomplishments, finding out about further developments, and giving you a taste of what it takes to get the judges’ attention. Last year’s awards saw OVO crowned as Data Champions. This is the category for Cloudera customers whose IT administration provides the agility business requires, without putting organizations at risk, and who are embracing a pattern of technology adopt
Rockset
JULY 22, 2021
Social gaming is on the rise. During COVID-19, 29% of consumers reported playing games on a weekly basis and the goal for many players was to connect with friends and family ( Deloitte: Games and Streaming Services Fight it Out During Pandemic from VentureBeat ). One of the challenges that gaming companies face is rapidly building features that can strengthen network effects.
ProjectPro
JULY 21, 2021
In this blog, explore a diverse list of interesting NLP projects ideas, from simple NLP projects for beginners to advanced NLP projects for professionals that will help master NLP skills. As per the Future of Jobs Report released by the World Economic Forum in October 2020, humans and machines will be spending an equal amount of time on current tasks in the companies, by 2025.
Speaker: Anne Steiner and David Laribee
As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.
DataKitchen
JULY 22, 2021
Learn about four data architectures patterns for agility - DataOps, Data Fabric, Data Mesh & Functional Data Engineering - & an example combining all four. The post DataOps: The Foundation for Your Agile Data Architecture first appeared on DataKitchen.
Teradata
JULY 20, 2021
Many Retailers & CPGs are missing huge opportunities to improve their margins & further enhance their customer experience due to broad brush data that lack insight. Read more.
A Cloud Guru: Data Engineering
JULY 20, 2021
In this post, (based on my session from the recent ACG Community Summit) I’m going to lay out what I view as the four pillars of Azure, trends we’re seeing around these, where I think they’re heading, and how you might plan your cloud career around these areas. What are the pillars of Azure? Before […] The post Pillars of Azure: 4 trends to watch in your cloud career appeared first on A Cloud Guru.
Grouparoo
JULY 18, 2021
Update (January 2022) The Grouparoo community is continually working to improve the developer experience for Reverse ETL. Here's our guide to Getting Started with Grouparoo to lead you through installation, configuration, running, and deploying projects. Grouparoo's recommend way to configure the application is through UI Config. An important enhancement to the workflow is the addition of Models.
Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage
Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.
DataKitchen
JULY 22, 2021
Chris Bergh chats with author Randy Bean about his book, Fail Fast, Learn Faster: Lessons in Data-Driven Leadership in an Age of Disruption, Big Data & AI. The post A Chat with Randy Bean on His Book, Fail Fast, Learn Faster first appeared on DataKitchen.
Teradata
JULY 18, 2021
Read more about Teradata's “Sleep Prediction” Hackathon, based on Apple Watch data, to capture different stages of sleep based on heart rate and activity count.
ProjectPro
JULY 23, 2021
Machine Learning and Deep Learning have experienced unusual tours from bust to boom from the last decade. Simmering in research labs, these two verticals of artificial intelligence became a savior for many companies. As there is a famous saying, "the larger, the better." But when it comes to large data sets, determining insights from them through deep learning algorithms and mining them becomes tricky.
Scribd Technology
JULY 19, 2021
Delta Lake is integral to our data platform which is why we have invested heavily in delta-rs to support our non-JVM Delta Lake needs. This year I had the opportunity to share the progress of delta-rs at Data and AI Summit. Delta-rs was originally started by my colleague QP just over a year ago and it has now grown to now a multi-company project with numerous contributors, and downstream projects such as kafka-delta-ingest.
Advertisement
Are you trying to decide which entity resolution capabilities you need? It can be confusing to determine which features are most important for your project. And sometimes key features are overlooked. Get the Entity Resolution Evaluation Checklist to make sure you’ve thought of everything to make your project a success! The list was created by Senzing’s team of leading entity resolution experts, based on their real-world experience.
Rockset
JULY 19, 2021
Apache Druid is a distributed real-time analytics database commonly used with user activity streams, clickstream analytics, and Internet of things (IoT) device analytics. Druid is often helpful in use cases that prioritize real-time ingestion and fast queries. Druid’s list of features includes individually compressed and indexed columns, various stream ingestion connectors and time-based partitioning.
Preset
JULY 21, 2021
Why Preset is the best alternative to Chart.io. Learn how to avoid lock-in with Preset, built on top of Apache Superset.
ProjectPro
JULY 19, 2021
Artificial Intelligence tools and technologies are moving at a rapid pace of innovation, so not to be surprised by the constant emergence of novel artificial intelligence and machine learning job roles like NLP Engineer , Computer Vision Engineer, Machine Learning Engineer, AI Software Engineer, AI Research Engineer, Artificial Intelligence Engineer , Machine Learning Scientist , Data Scientist , and many more to mention.
RudderStack
JULY 21, 2021
Here’s an overview of the specific ways engineering teams support marketing from the data layer with RudderStack.
Speaker: Margaret-Ann Seger, Head of Product, Statsig
Experimentation is often seen as an aspirational practice, especially at smaller, fast-moving companies who are strapped for time and resources. So, how can you get your team making decisions in a more data-driven way while continuing to remain lean and maintaining ship velocity? In this webinar, Margaret-Ann Seger, Head of Product at Statsig, will teach you how to build an experimentation culture from the ground-up, graduating from just getting started with data-driven development to operating
Data Engineering Podcast
JULY 22, 2021
Summary Collecting and cleaning data is only useful if someone can make sense of it afterward. The latest evolution in the data ecosystem is the introduction of a dedicated metrics layer to help address the challenge of adding context and semantics to raw information. In this episode Nick Handel shares the story behind Transform, a new platform that provides a managed metrics layer for your data platform.
Cloudera
JULY 22, 2021
My name is Shanmukha Kota and I am a recent graduate from University at Buffalo. I interned with Cloudera last summer and joined Cloudera as a software engineer a couple of weeks ago and this is my first experience with CDP and CDP Operational Database. For a new hire college graduate in the industry with only academic experience with HBase, I can only say it is very simple and easy to set up and work with CDP Operational Database.
ProjectPro
JULY 17, 2021
Want to become an AI Engineer? Check out this detailed AI Engineer salary guide to understand how much can you make as an AI engineer based on various factors- experience level, companies, and location. Artificial Intelligence (AI) market will be worth 190 Billion USD by 2025. As of June 2022, there are 18,380 open vacancies for AI Engineers in the United States, while India has 2,740 openings for the role of an AI Engineer.
Scribd Technology
JULY 20, 2021
Extracting metadata from our documents is an important part of our discovery and recommendation pipeline, but discerning useful and relevant details from text-heavy user-uploaded documents can be challenging. This is part 2 in a series of blog posts describing a multi-component machine learning system the Applied Research team built to extract metadata from our documents in order to enrich downstream discovery models.
Speaker: David Bard, Principal at VP Product Coaching
In the fast-paced world of digital innovation, success is often accompanied by a multitude of challenges - like the pitfalls lurking at every turn, threatening to derail the most promising projects. But fret not, this webinar is your key to effective product development! Join us for an enlightening session to empower you to lead your team to greater heights.
ProjectPro
JULY 21, 2021
As a big data architect or a big data developer, when working with Microservices-based systems, you might often end up in a dilemma whether to use Apache Kafka or RabbitMQ for messaging. Rabbit MQ vs. Kafka - Which one is a better message broker? You might find some articles across the web that conclude that Apache Kafka is better than RabbitMQ and few others that mention RabbitMQ to be more reliable than Kafka.
Cloudera
JULY 20, 2021
Most businesses, whether you are in Retail, Manufacturing, Specialty Chemicals, Telecommunications, consider a 10% market capitalization increase from 2020 to 2021 outstanding. But what would you say to your shareholders when they found out your competitors’ market capitalization grew 35%? A recent McKinsey report dove into the divergence between retail’s laggards and winners and found if there is one message in the retail sector’s stock market performance since the pandemic’s start, it is
Expert insights. Personalized for you.
130 
Let's personalize your content