phData: Data Engineering

DECEMBER 18, 2023

Data engineering refers to the design of systems that are capable of collecting, analyzing, and storing data at a large scale. In manufacturing, data engineering aids in optimizing operations and enhancing productivity while ensuring curated data that is both compliant and high in integrity. The increased efficiency in data “wrangling” means that more accurate modeling and planning may be done, enabling manufacturers to make stronger data-driven decisions.

Manufacturing 52

Manufacturing Data Engineering Data Engineer Engineering 52

The Pragmatic Engineer

DECEMBER 18, 2023

I get asked every now and then if I offer 1:1 mentoring for either software engineers or engineering managers or leaders. While I used to do this in the past, I don't offer this any more. I collected much of the advice I have to offer for software engineers in The Software Engineer's Guidebook. I also write The Pragmatic Engineer Newsletter where I do cover topics like what it means to be a senior engineer at various companies , how to deal with a low-quality engineering culture , and

Software Engineer 195

Software Engineer Software Engineering Engineering Management 195

Analytics Vidhya

DECEMBER 18, 2023

Introduction Docker containers have emerged as indispensable tools in the fast-evolving landscape of software development and deployment, providing a lightweight and efficient way to package, distribute, and run applications. This article delves into the top 20 Docker containers across various categories, showcasing their features, use cases, and contributions to streamlining development workflows.

231

231

Waitingforcode

DECEMBER 19, 2023

Even though nowadays data processing frameworks and data stores have smart query planners, they don't take our responsibility to correctly design the job logic.

Designing 130

Designing Data Process Process Data 130

Advertisement

Generative AI is upending the way product developers & end-users alike are interacting with data. Despite the potential of AI, many are left with questions about the future of product development: How will AI impact my business and contribute to its success? What can product managers and developers expect in the future with the widespread adoption of AI?

Data

Engineering at Meta

DECEMBER 19, 2023

On July 5, 2023, Meta launched Threads, the newest product in our family of apps, to an unprecedented success that saw it garner over 100 million sign ups in its first five days. A small, nimble team of engineers built Threads over the course of only five months of technical work. While the app’s production launch had been under consideration for some time, the business finally made the decision and informed the infrastructure teams to prepare for its launch with only two days’ advance notice.

Programming Language 141

Programming Language Engineering Database Coding 141

The Pragmatic Engineer

DECEMBER 21, 2023

2023 was the second full year of The Pragmatic Engineer Newsletter , and this newsletter is now almost two and a half years old; the first issue came out on 26 August 2021. Thank you for being a reader, I greatly value your support. This year, 102 newsletter issues were published, and this is number 103. You received a deepdive issue on Tuesdays, and every Thursday it was  “The Pulse”  – formerly The Scoop.

Engineering 176

Engineering Software Engineer Software Engineering Project 176

KDnuggets

DECEMBER 20, 2023

We've partnered with Springboard, the leading data science bootcamp offering personalized 1:1 mentorship, dedicated career support, proven outcomes, and an unbeatable money-back job guarantee, to present a handpicked collection of resources to supercharge your data science journey in the coming year.

Data Science 133

Data Science Data 133

LinkedIn Engineering

DECEMBER 19, 2023

Co-authors: Max Kanat-Alexander and Grant Jenks Today we are open-sourcing the LinkedIn Developer Productivity & Happiness Framework (DPH Framework) - a collection of documents that describe the systems, processes, metrics, and feedback systems we use to understand our developers and their needs internally at LinkedIn. Now more than ever, developers are navigating so much change and new opportunity in this new era of Generative AI, so ensuring teams have the systems, processes, metrics and f

Data Schemas 121

Data Schemas Software Engineer Software Engineering Data Pipeline 121

Scott Logic

DECEMBER 17, 2023

An exploration of the quantitative and qualitative impacts of Generative AI on software development. We’ve undertaken multiple experiments to better understand the impact of GenerativeAI tools (ChatGPT, Copilot) on developer productivity. Our quantitative results show a 37% improvement in productivity (speed), however, this result is a misrepresentation of what it means to be productive as a developer.

Programming Language 118

Programming Language Coding Programming Consulting 118

databricks

DECEMBER 21, 2023

Over the past six months, we've been working with NVIDIA to get the most out of their new TensorRT-LLM library. TensorRT-LLM provides an easy-to-use Python interface to integrate with a web server for fast, efficient inference performance with LLMs. In this post, we're highlighting some key areas where our collaboration with NVIDIA has been particularly important.

Python 119

Python 119

Advertisement

Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.

Database

KDnuggets

DECEMBER 22, 2023

Here are our top posts of 2023, including: 5 Free Books to Master Data Science • 5 Free Courses to Master Machine Learning • 3 Ways to Access GPT-4 for Free • and much more!

Machine Learning 127

Machine Learning Data Science Accessible Accessibility 127

Snowflake

DECEMBER 18, 2023

When businesses share sensitive first-party data with outside partners or customers, they must do so in a way that meets strict governance requirements around security and privacy. Data clean rooms have emerged as the technology to meet this need, enabling interoperability where multiple parties can collaborate on and analyze sensitive data in a governed way without exposing direct access to the underlying data and business logic.

Cloud 115

Cloud Building Entertainment Government 115

Engineering at Meta

DECEMBER 19, 2023

HawkEye is the powerful toolkit used internally at Meta for monitoring, observability, and debuggability of the end-to-end machine learning (ML) workflow that powers ML-based products. HawkEye supports recommendation and ranking models across several products at Meta. Over the past two years, it has facilitated order of magnitude improvements in the time spent debugging production issues.

Data Pipeline 108

Data Pipeline Algorithm Architecture Machine Learning 108

databricks

DECEMBER 21, 2023

We are excited to announce that Gartner has recognized Databricks as a Leader for a third consecutive year in the 2023 Gartner® Magic.

Systems 135

Systems Database Cloud Management 135

Speaker: Scott Sehlhorst

We know we want to create products which our customers find to be valuable. Whether we label it as customer-centric or product-led depends on how long we've been doing product management. There are three challenges we face when doing this. The obvious challenge is figuring out what our users need; the non-obvious challenges are in creating a shared understanding of those needs and in sensing if what we're doing is meeting those needs.

Certification

KDnuggets

DECEMBER 19, 2023

There are many data-learning materials locked up behind expensive books. These cheap books would bolster your skills without blowing up your savings.

Data Science 125

Data Science Data 125

Knowledge Hut

DECEMBER 22, 2023

What is SAFe® ? Traditional Agile has always had an assumption of small teams. In recent years, several approaches (Large-scale Scrum (LeSS), Scrum @ Scale and Scaled Agile Framework (SAFe®) have evolved. The Scaled Agile Framework (SAFe®) is a set of organization and workflow patterns intended to guide enterprises in scaling lean and agile practices.

Certification 98

Certification Portfolio Education Programming 98

ArcGIS

DECEMBER 20, 2023

Check out this blog to see how we enhanced several geoprocessing controls with additional features and new designs in ArcGIS Pro 3.2.

Designing 114

Designing 114

databricks

DECEMBER 21, 2023

Today, Databricks is excited to announce support for Mixtral 8x7B in Model Serving. Mixtral 8x7B is a sparse Mixture of Experts (MoE) open.

Engineering 124

Engineering 124

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

Data Science

KDnuggets

DECEMBER 18, 2023

Looking for a great course to learn Artificial Intelligence with Python? Check out this free course from Harvard University.

Python 136

Python 136

Knowledge Hut

DECEMBER 22, 2023

We read and hear about cyber-attacks and hacking threats all the time. However, ethical hackers are the one-stop solution to various cyber threats. Research claims that a malware-based cyber-attack can cost a business 2.6 million dollars. Thus, the ethical hacker’s salary increased significantly over a few years. An ethical hacker must have the computer system's owner's written consent before accessing it, safeguard the organization's privacy, disclose all vulnerabilities found

Consulting 98

Consulting Certification Banking Education 98

ThoughtSpot

DECEMBER 19, 2023

2023 has been a year of breakthrough innovation for many, and a deer-in-headlights moment for others. I keep flashing back to the 90s when the Internet created new businesses and destroyed others—LLMs are doing the same, only with more velocity. From CDAOs to VCs alike, the rate of creative destruction is faster, but there is also an intense focus on value.

Entertainment 97

Entertainment Data Big Data Banking 97

LinkedIn Engineering

DECEMBER 19, 2023

Co-authors: Arjun Mohnot , Jenchang Ho , Anthony Quigley , Xing Lin , Anil Alluri , Michael Kuchenbecker LinkedIn operates one of the world’s largest Apache Hadoop big data clusters. These clusters are the backbone for storing and processing extensive data volumes, empowering us to deliver essential features and services to members, such as personalized recommendations, enhanced search functionality, and valuable insights.

Big Data 92

Big Data Hadoop Metadata Data 92

Advertisement

Start-ups & SMBs launching products quickly must bundle dashboards, reports, & self-service analytics into apps. Customers expect rapid value from your product (time-to-value), data security, and access to advanced capabilities. Traditional Business Intelligence (BI) tools can provide valuable data analysis capabilities, but they have a barrier to entry that can stop small and midsize businesses from capitalizing on them.

Data Analysis

KDnuggets

DECEMBER 22, 2023

Learn how to integrate advanced AI multimodal models into your project using a simple Python API.

Accessible 146

Accessible Accessibility Python Project 146

Knowledge Hut

DECEMBER 22, 2023

Every business now incorporates data science into their operations, especially those that recognize the value of data and the potential applications of that knowledge. A data scientist's main responsibility is to draw practical conclusions from complicated data so that you may make informed business decisions. This career guide can assist you in taking the first steps toward a rewarding career in data science if you're wondering how to become a data scientist.

Portfolio 98

Portfolio Data Science Programming Language Scala 98

databricks

DECEMBER 19, 2023

Request a meeting with Databricks executives/thought leaders at NRF! Each January, thousands of leaders from retailers around the globe gather at Javits Center.

Retail 99

Retail Data 99

ArcGIS

DECEMBER 19, 2023

Here's how to fake an isometric 3D topo terrain in 2D! And stuff.

138

138

Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage

Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.

Building

KDnuggets

DECEMBER 19, 2023

Soon AI will become embedded into daily business processes, including cybersecurity controls. The author explains how to assess which processes make sense to automate.

Process 110

Process 110

Knowledge Hut

DECEMBER 22, 2023

We live in a fast-paced world, where now, changes are happening within moments, not months, and definitely not years. This rapidly changing world urges us to use dynamic approaches for better outcomes within less time. Among one of these approaches, Scrum stands out! Considered as one of the most agile frameworks by businesses for obvious reasons – Scrum helps to enhance productivity and offer practical customer value.

Certification 98

Certification Education Project Designing 98

Cloudera

DECEMBER 19, 2023

Without a doubt, 2023 has shaped up to be generative AI’s breakout year. Less than 12 months after the introduction of generative AI large language models such as ChatGPT and PaLM, image generators like Dall-E, Midjourney, and Stable Diffusion, and code generation tools like OpenAI Codex and GitHub CoPilot, organizations across every industry, including government, are beginning to leverage generative AI regularly to increase creativity and productivity.

Government 85

Government Education Unstructured Data Datasets 85

databricks

DECEMBER 22, 2023

Introduction Anomaly detection is widely applied across various industries, playing a significant role in the enterprise sector. This blog focuses on its application.

Machine Learning 91

Machine Learning Building IT 91

Speaker: Anne Steiner and David Laribee

As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.

Engineering