Confessions of a Data Guy
JUNE 20, 2024
After 10 years of Data Engineering work, I think it’s time to hang up the proverbial hat and ride off into the sunset, never to be seen again. I wish. Everything has changed in 10 years, yet nothing has changed in 10 years, how is that even possible? Sometimes I wonder if I’ve learned anything […] The post What I’ve Learned After A Decade Of Data Engineering appeared first on Confessions of a Data Guy.
Waitingforcode
JUNE 17, 2024
One of the big challenges in streaming Delta Lake is the inability to handle in-place changes, like updates, deletes, or merges. There is good news, though. With a little bit of effort on your data provider's side, you can process a Delta Lake table as you would process Apache Kafka topics, hence without in-place changes.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Rockset
JUNE 21, 2024
I’m excited to share that OpenAI has completed the acquisition of Rockset. We are thrilled to join the OpenAI team and bring our technology and expertise to building safe and beneficial AGI. From the start, our vision at Rockset was to fundamentally transform the way data-driven applications were built. We developed our search and analytics database, taking full advantage of the cloud, to eliminate the complexity inherent in the data infrastructure needed for these apps.
KDnuggets
JUNE 20, 2024
Image by author Model deployment is the process of trained models being integrated into practical applications. This includes defining the necessary environment, specifying how input data is introduced into the model and the output produced, and the capacity to analyze new data and provide relevant predictions or categorizations.
Speaker: Jay Allardyce
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
Cloudera
JUNE 17, 2024
Pride Month is underway and we at Cloudera are looking forward to joining the global celebration of diversity, equity and the ongoing effort for LGBTQ+ ( L esbian, G ay, B isexual, T ransgender, Q ueer/ Q uestioning) rights and recognition. Pride Month serves as a reminder that the fight for equality and equity for members of the LGBTQ+ community is not over.
Data Engineering Weekly
JUNE 16, 2024
Experience Enterprise-Grade Apache Airflow Astro augments Airflow with enterprise-grade features to enhance productivity, meet scalability and availability demands across your data pipelines, and more. Learn More → Databricks: Open Sourcing Unity Catalog This week brought many exciting developments, with Snowflake and Databricks announcing open-source catalogs.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
KDnuggets
JUNE 20, 2024
Understanding LLMs is pivotal in unlocking the full potential of AI-driven solutions across various domains. As we navigate the process of building AI-driven solutions, it is essential to approach the development and deployment of LLMs with a focus on responsible AI practices.
Snowflake
JUNE 20, 2024
In the age of AI, enterprises are increasingly looking to extract value from their data at scale but often find it difficult to establish a scalable data engineering foundation that can process the large amounts of data required to build or improve models. Designed for processing large data sets, Spark has been a popular solution, yet it is one that can be challenging to manage, especially for users who are new to big data processing or distributed systems.
ArcGIS
JUNE 17, 2024
Productivity Update! Learn how to override default parameter values for geoprocessing tools in ArcGIS Pro 3.3. Override Geoprocessing Tool Defaults in ArcGIS Pro 3.
Cloudera
JUNE 21, 2024
Juneteenth holds profound significance in the history of freedom and equality for Black Americans. Also known as Freedom Day or Emancipation Day, Juneteenth commemorates the anniversary of June 19, 1865, when news of the Emancipation Proclamation reached Galveston, Texas, finally declaring freedom for enslaved Americans held in the Confederacy–more than two years after the proclamation was issued in on January 1, 1863.
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
In an era where efficiency is king, are you leveraging the right tools to transform your manufacturing processes? A Manufacturing Execution System (MES) is critical for enhancing operational efficiency, reducing waste, and optimizing energy usage—key factors for improving your bottom line and lowering your carbon footprint. Join Nikhil Joshi, a manufacturing technology expert with 18+ years of hands-on experience, in this new webinar as he uncovers the secrets of MES and how to best utilize thes
KDnuggets
JUNE 21, 2024
Want to learn AI from the best of resources? Check out these free AI courses from top universities.
databricks
JUNE 20, 2024
We are excited to announce that Gartner has recognized Databricks as a Leader in the 2024 Gartner® Magic Quadrant™ for Data Science and.
Snowflake
JUNE 20, 2024
Recent Snowflake workshops and roundtables have started with the question: “Does your data strategy match your AI ambition?” It certainly sparks customer engagement, but is that the right question to ask? Right now, it seems appropriate with all of the interest — dare I say “hype” — around AI. But it merely reflects the current darling of the tech world, focusing on the technology itself, rather than the ultimate goal.
Netflix Tech
JUNE 20, 2024
A summary of sessions at the first Data Engineering Open Forum at Netflix on April 18th, 2024 The Data Engineering Open Forum at Netflix on April 18th, 2024. At Netflix, we aspire to entertain the world, and our data engineering teams play a crucial role in this mission by enabling data-driven decision-making at scale. Netflix is not the only place where data engineers are solving challenging problems with creative solutions.
Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage
When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.
KDnuggets
JUNE 19, 2024
Perform data integrity tests and generate model evaluation reports by writing a few lines of code.
databricks
JUNE 21, 2024
In the insurance sector, customers demand personalized, fast, and efficient service that addresses their needs. Meanwhile, insurance agents must access a large amount.
Zalando Engineering
JUNE 20, 2024
Introduction In Lounge by Zalando, we run an Elasticsearch cluster in Kubernetes to store user facing article descriptions. Our business model is such that we receive about three times the normal load during the busy hour in the morning and therefore we use schedules to automatically scale in and out applications to handle that peak. If scaling out in the morning fails, we face a potential catastrophe.
ArcGIS
JUNE 17, 2024
Discover what's new in ArcGIS Pro 3.3 for CAD and BIM workflows, allowing you to directly read datasets from Autodesk Revit, Civil 3D, and Industry Foundation Classes.
Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network
In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.
KDnuggets
JUNE 18, 2024
Get your own local RAG system up and running in an embarrassingly few lines of code thanks to these 3 Llamas.
Cloudera
JUNE 18, 2024
The world of Generative AI (GenAI) is rapidly evolving, with a wide array of models available for businesses to leverage. These models can be broadly categorized into two types: closed-source (proprietary) and open-source models. Closed-source models, such as OpenAI’s GPT-4o, Anthropic’s Claude 3, or Google’s Gemini 1.5 Pro, are developed and maintained by private and public companies.
Snowflake
JUNE 18, 2024
Despite the seemingly nonstop conversation surrounding AI, the data suggests that bringing AI into enterprises is still easier said than done. There’s so much potential and plenty of value to be captured — if you have the right models and tools. Implementing advanced AI today requires a solid data foundation and often a set of solutions, each demanding its own tools and complex infrastructure.
ArcGIS
JUNE 17, 2024
Discover what's new in ArcGIS Pro 3.3 for CAD and BIM workflows, allowing you to directly read datasets from Autodesk Revit, Civil 3D, and Industry Foundation Classes.
Speaker: Evelyn Chou
Choosing the right business intelligence (BI) platform can feel like navigating a maze of features, promises, and technical jargon. With so many options available, how can you ensure you’re making the right decision for your organization’s unique needs? 🤔 This webinar brings together expert insights to break down the complexities of BI solution vetting.
KDnuggets
JUNE 18, 2024
Going beyond technical skills; learn how to make a data science profile that stands out and helps you land your dream role.
Monte Carlo
JUNE 20, 2024
No two companies are ever quite the same. Some teams have more security needs. Other teams are concerned about costs or administration requirements. So, when it comes to how organizations choose to deploy new software, there’s never a one-size-fits-all approach. That’s particularly true when you’re working with a customer resource as critical as data.
Hevo
JUNE 20, 2024
ETL (Extract, Transform, and Load) is an emerging topic in all IT Industries. Industries often look for an easy solution to do ETL on their data without spending much effort on coding. If you’re also looking for such a solution, then you’ve landed in the right place.
RandomTrees
JUNE 18, 2024
Customer services are continuously changing significantly. Now, it is not about waiting for hours plus and getting irritating phone menus. For instance, artificial intelligence (AI) chatbots powered by the latest machine learning and natural language processing (NLP) applications have redefined interaction between companies and their customers. The old days, where virtual assistants used to handle simple queries, are gone.
Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL
Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.
KDnuggets
JUNE 21, 2024
Generating a ready-to-use HuggingFace model with FastAPI and Docker
phData: Data Engineering
JUNE 18, 2024
Machine learning (ML) is only possible because of all the data we collect. However, with data coming from so many different sources, it doesn’t always come in a format that’s easy for ML models to understand. Before you can take advantage of everything ML offers, much prep work is involved. In this blog, we’ll explain why you should prepare your data before use in machine learning , how to clean and preprocess the data, and a few tips and tricks about data preparation.
Hevo
JUNE 19, 2024
Oracle is widely used to store, manage, and perform complex operations on data, making it ideal for business-critical operations. You can efficiently scale your business data by hosting Oracle services on the Google Cloud Platform. GCP offers efficient resource utilization, which can be helpful when performing operations like data processing, analysis, and visualization.
Striim
JUNE 18, 2024
In today’s fast-paced world, staying ahead of the competition requires making decisions informed by the freshest data available — and quickly. That’s where real-time data integration comes into play. By seamlessly blending and updating information from numerous sources, businesses can guarantee their AI systems are fueled by the latest, most accurate data.
Speaker: Christophe Louvion, Chief Product & Technology Officer of NRC Health and Tony Karrer, CTO at Aggregage
Christophe Louvion, Chief Product & Technology Officer of NRC Health, is here to take us through how he guided his company's recent experience of getting from concept to launch and sales of products within 90 days. In this exclusive webinar, Christophe will cover key aspects of his journey, including: LLM Development & Quick Wins 🤖 Understand how LLMs differ from traditional software, identifying opportunities for rapid development and deployment.
Expert insights. Personalized for you.
223 
Let's personalize your content