KDnuggets

NOVEMBER 30, 2023

Data engineers must prepare and manage the infrastructure and tools necessary for the whole data workflow in a data-driven company.

Data Engineering 91

Data Engineering Data Engineer Engineering Data 91

Analytics Vidhya

NOVEMBER 26, 2023

In this Leading with Data episode, explore the analytics landscape with Dr. Swati Jain, a seasoned leader boasting over two decades of experience. From her unforeseen foray into analytics to steering EXL Analytics’ India business, Dr. Jain imparts invaluable insights into the ever-evolving world of data science. Read on to know more about her career, […] The post Unlocking the Power of Analytics with Dr.

Data Science 215

Data Science Data Data Analysis Data Engineering 215

Jesse Anderson

NOVEMBER 30, 2023

Lately, I’ve been learning how to trade options. Although there’s data and programming involved in options trading, it isn’t as technical as data engineering or software engineering. However, it reflects the current state of learning, whether that’s data engineering or options trading. It gave me a look into learning a skill using videos. Each lesson I learned will directly apply to your learning or skill improvement.

Data Engineering 147

Data Engineering Data Engineer Engineering Algorithm 147

Waitingforcode

NOVEMBER 28, 2023

In March I wrote a blog showing how to use accumulators to know the application of each filter statement. Turns out, the solution may not be perfect as mentioned by Aravind in one of the comments. I bet you already have an idea but if not, keep reading. Everything will be clear in the end!

130

130

Speaker: Dr. Greg Loughnane and Chris Alexiuk

Technology professionals developing generative AI applications are finding that there are big leaps from POCs and MVPs to production-ready applications. They're often developing using prompting, Retrieval Augmented Generation (RAG), and fine-tuning (up to and including Reinforcement Learning with Human Feedback (RLHF)), typically in that order. However, during development – and even more so once deployed to production – best practices for operating and improving generative AI applications are le

Process

Seattle Data Guy

NOVEMBER 28, 2023

Data warehousing would be easy if all data were structured and formatted in the data source. Maybe we wouldn’t even need to build a data warehouse. But as anyone who has worked with data from more than one source knows, that’s rarely the case. Businesses today need to pull data from a plethora of sources,… Read more The post Finding The Right ETL/ELT Solution – What Is Estuary And Should You Use It?

Data Warehouse 130

Data Warehouse IT Data Building 130

KDnuggets

NOVEMBER 29, 2023

Probability is one of the foundational elements of computer science. Some bootcamps will skim over the topic, however, it is integral to your computer science knowledge.

Computer Science 123

Computer Science Data Science IT Data 123

Confluent

NOVEMBER 28, 2023

Learn how to write code that produces messages via librdkafka, how it will behave during error situations, and how your application should detect and respond to them.

Coding 119

Coding Kafka IT 119

Knowledge Hut

NOVEMBER 30, 2023

The job market today emphasizes experience as a major criterion. Employers consider experienced professionals better candidates since they provide more value to the company. Are you interested in knowing how to become a data scientist with no experience  but not sure how to go about it? Here you will learn how to get your first data science job. To make t hings easier for you, here is a quick tip.

Data Science 96

Data Science Data Programming Language Data Engineering 96

Tweag

NOVEMBER 27, 2023

Sponsored by Antithesis (distributed systems reliability testing experts), I’ve developed a new library to filter local files in Nix which I’d like to introduce! This post requires some familiarity with Nix and its language. So if you don’t know what Nix is yet, take a look first, it’s pretty neat. In this post we’re going to look at what source filtering is, why it’s useful, why a new library was needed for it, and the basics of the new library.

Building 107

Building Systems Project IT 107

Advertisement

Are you thinking of adding enhanced data matching and relationship detection to your product or service? Do you need to know more about what to look for when assessing your options? The Senzing Entity Resolution Buyer’s Guide gives you step-by-step details about everything you should consider when evaluating entity resolution technologies. You’ll learn about use cases, technology and deployment options, top ten evaluation criteria and more.

Technology

KDnuggets

NOVEMBER 28, 2023

Want to support the behavior of built-in functions and method calls in your Python classes? Magic methods in Python let you do just that! So let’s uncover the method behind the magic.

Python 107

Python 107

Medium Data Engineering

NOVEMBER 28, 2023

บางครั้งเราก็อยาก process อะไรบางอย่างบน PySpark เช่นการ encrypt ข้อมูล หรือแปลงข้อมูลแบบแปลก ๆ ด&

Process 98

Process Data Engineering Data Engineer Engineering 98

Knowledge Hut

NOVEMBER 28, 2023

Software engineers, on average, get paid $1,13,781 yearly; however, the pay scale usually varies depending on the job location, employer, and demographics. The amount you earn as a working software professional will depend on the number of years of experience, skillsets you have, and demand for that job position in the industry. Experienced software engineers make up to millions a year, and even freelance software developers can earn up to hundreds of thousands of dollars per project.

Software Engineer 98

Software Engineer Software Engineering Engineering Python 98

Airbnb Tech

NOVEMBER 28, 2023

By: Clark Wright Introduction These days, as the volume of data collected by companies grows exponentially, we’re all realizing that more data is not always better. In fact, more data, especially if you can’t rely on its quality, can hinder a company by slowing down decision-making or causing poor decisions. With 1.4 billion cumulative guest arrivals as of year-end 2022, Airbnb’s growth pushed us to an inflection point where diminishing data quality began to hinder our data practitioners.

Data 98

Data Data Warehouse Metadata Certification 98

Edureka

NOVEMBER 28, 2023

Computer vision is an interdisciplinary scientific field that deals with how computers can be made to gain high-level understanding from digital images or videos. OpenCV(open source computer vision library) is an open source computer vision and machine learning software library. OpenCV was build to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.

Python 98

Python Machine Learning Algorithm Programming Language 98

KDnuggets

NOVEMBER 28, 2023

This is a great guide for anyone who wants to learn Web Scraping. It can help you understand the basics of Web Scraping with BeautifulSoup and how to use it.

IT 103

IT 103

Knowledge Hut

NOVEMBER 27, 2023

Is your organization active on social media? Whether you work in big business, a charity, the public sector or somewhere else, chances are your organization has or should have social media accounts. That might be YouTube, SlideShare, Pinterest or LinkedIn (or one of many other social networks), and the right channel is going to largely depend on what you want to get out of your engagement with your social media communities.

Media 96

Media Building Process IT 96

Confluent

NOVEMBER 27, 2023

The top 7 free online courses, tutorials, get started guides, and examples for the easiest way to learn Apache Kafka.

Kafka 118

Kafka 118

DoorDash Engineering

NOVEMBER 28, 2023

It is amusing for a human being to write an article about artificial intelligence in a time when AI systems, powered by machine learning (ML), are generating their own blog posts. DoorDash has been building an internal Machine Learning Workbench over the past year to enhance data operations and assist our data scientists, analysts, and AI/ML engineers.

Machine Learning 91

Machine Learning Data Science Pipeline-centric Engineering 91

databricks

NOVEMBER 29, 2023

Background: Modernizing Data Delivery Today's enterprise data estates are vastly different from 10 years ago. Industries have transitioned their analytics from monolithic data.

Data 86

Data Government Healthcare 86

Knowledge Hut

NOVEMBER 28, 2023

As Data scientists, our focus is on both the quality and quantity of data which can improve the model results. With different sources of data, we can leverage the information to drive good business understanding. Whether you are working on a personal project, learning the concepts, or working with datasets for your company, the primary focus is a data acquisition and data understanding.

Datasets 94

Datasets Data Science Project Data 94

Snowflake

NOVEMBER 30, 2023

At TCS , we help companies shift their enterprise data warehouse (EDW) platforms to the cloud as well as offering IT services. We’re extremely familiar with just how tricky a cloud migration can be, especially when it involves moving historical business data. Choosing a migration approach involves balancing cloud strategy, architecture needs and business priorities.

Data 81

Data Data Warehouse Cloud Raw Data 81

KDnuggets

NOVEMBER 27, 2023

Data analysis is a time-consuming activity. With ChatGPT, we can perform data summary, data preprocessing, data visualization, and many others in a short time.

Data Analysis 98

Data Analysis Data Data Science 98

Lyft Engineering

NOVEMBER 29, 2023

Written by Ritesh Varyani and Jeana Choi at Lyft. Introduction At Lyft, we have used systems like Apache ClickHouse and Apache Druid for near real-time and sub-second analytics. Sub-second query systems allow for near real-time data explorations and low latency, high throughput queries, which are particularly well-suited for handling time-series data.

Kafka 84

Kafka Data Ingestion Architecture Datasets 84

Knowledge Hut

NOVEMBER 28, 2023

As a software engineer , you will be responsible for developing and maintaining software applications. You will also be involved in the testing and debugging of software programs. To be successful in this role, you will need to have strong problem-solving skills, technical skills, and the ability to work independently. They are also constantly innovating and expanding, which creates opportunities for software engineers to grow their skills and careers.

Software Engineer 87

Software Engineer Software Engineering Engineering Python 87

Snowflake

NOVEMBER 28, 2023

ERP systems run the world’s businesses. These stalwart systems are great at managing records and processes for finance, operations, supply chain management and more. But their insights need an upgrade. That’s the case put forward by Maxa , an enterprise-grade startup that has made it their mission to reinvent the way companies access and use ERP data for transformational insights.

Data 88

Data Systems Finance Accessible 88

Medium Data Engineering

NOVEMBER 28, 2023

In this blog post, we share our innovative approach to scoring data quality, Airbnb’s Data Quality Score (“DQ Score”).

Data 98

Data Data Engineering Data Engineer Data Science 98

Netflix Tech

NOVEMBER 29, 2023

by Aditya Mavlankar , Zhi Li , Lukáš Krasula and Christos Bampis High dynamic range ( HDR ) video brings a wider range of luminance and a wider gamut of colors, paving the way for a stunning viewing experience. Separately, our invention of Dynamically Optimized ( DO ) encoding helps achieve optimized bitrate-quality tradeoffs depending on the complexity of the content.

Metadata 82

Metadata Algorithm Software Engineer Software Engineering 82

KDnuggets

NOVEMBER 30, 2023

The blog discusses five platforms designed for data scientists with specialized capabilities in managing large datasets, models, workflows, and collaboration beyond what GitHub offers.

Data Science 78

Data Science Data Datasets Project 78

LinkedIn Engineering

NOVEMBER 27, 2023

Co-Authors: Abhishek Chandak and Ritish Verma We recognize that our 1 billion members and their over 10 billion years’ worth of collective knowledge and insights bring tremendous value to the LinkedIn community. That's why we're committed to enabling our members and customers to safely engage and connect with content. Our Trust & Safety team works diligently to keep harmful content off the platform, allowing members to tap into the real-life, meaningful insights and information shared by the

Machine Learning 85

Machine Learning Engineering Data Science Data 85

Knowledge Hut

NOVEMBER 27, 2023

The Data Science learning path is a collective set of curated courses that comprise a learning plan for achieving the required skills for the data scientist role. While the time limit to complete the learning path to become a data scientist can expect 8-9 months to get through all Data Science courses. It is known that people from diverse backgrounds with zero experience turn out to be good data scientists in just a year through learning smart coding.

Data Science 81

Data Science Data Python Machine Learning 81

Netflix Tech

NOVEMBER 27, 2023

Netflix was thrilled to be the premier sponsor for the 2nd year in a row at the 2023 Conference on Digital Experimentation (CODE@MIT) in Cambridge, MA. The conference features a balanced blend of academic and industry research from some wicked smart folks, and we’re proud to have contributed a number of talks and posters along with a plenary session.

Coding 83

Coding Algorithm Data 83

KDnuggets

NOVEMBER 29, 2023

This week on KDnuggets: Start learning how to build machine learning models today with these free machine learning courses • See how ChatGPT creates jaw-dropping data viz with just a few words • And much, much more!

Machine Learning 82

Machine Learning Data Building 82