Alex Vitvitskyi

This article by Pere Martra explores a structured pruning technique for state-of-the-art models, that uses a GLU architecture, enabling the creation of smaller and more efficient large language models. #Llama #LLM

5

🚀 Want to employ the power of #PyTorch for your scientific models without a steep learning curve? 👀 Check out our latest blog, where Travis Oliphant discusses how PyTorch's integration with the Array API makes it easy to use with familiar libraries like NumPy and scikit-learn.🔥 https://lnkd.in/dDRkEY3v #Innovation

10

🔍 Curious about optimizer performance? Benjamin Marie's latest article delves into a comparative analysis of AdamW-32bit, its 8-bit variant, and paged AdamW optimizers. Discover their effects on memory consumption, learning curves, and training time. #MachineLearning #Programming

13

We are thrilled to announce that the new Python version of the #ASALI API and the updated version of #ASALIPY are now live! This update brings enhanced features for estimating thermodynamic and transport properties, seamlessly integrating with the latest version of Cantera. Check them out now! #OpenSource #ChemicalEngineering #Python

6

Fine-tuning LLMs with Reinforcement Learning from Human Feedback This blog post explores how Reinforcement Learning from Human Feedback (RLHF) can be used to fine-tune Large Language Models (LLMs) to align with human preferences. It delves into the concept of Proximal Policy Optimization (PPO) and how it is used in this context. The key takeaway is that RLHF offers a powerful framework to train LLMs that can produce human-aligned text, while still maintaining creativity. This paves the way for more natural and nuanced language generation in the future of AI and natural language understanding. Read the article: https://lnkd.in/d9EGf579 #AI #machinelearning #goML

8

🧠 What’s a Semantic Cache in RAG? Here’s What We’ve Learned Semantic cache is a time and cost saving method for Retrieval Augmented Generation (RAG). Instead of always running a full vector search, a semantic cache stores embeddings of frequently asked queries and their responses. This speeds up your RAG system by skipping redundant searches for similar queries. How do you implement it? Step 1: Store history of answered questions, their embeddings and the LLM response in a semantic cache table. Step 2: When a new query comes in, compare it against the cache table using vector similarity. Step 3: If there’s a match with score over a certain threshold, serve the cached response. If not, run a fresh retrieval and add it to the cache. Now, let’s talk vector search methods: 1️⃣ Cosine Similarity: Measures angle between vectors, works well for normalized data. 2️⃣ Euclidean Distance: Measures straight-line distance but can be skewed by vector magnitude. 3️⃣ Dot Product: Our favorite—it factors in vector size and performs best for our use cases. Want to see how semantic caching powers a RAG system in real life? Check out our demo at https://lnkd.in/g2jFuHTT. 🚀

2

This week in #TheBatch: 🎨 All about Claude’s Artifacts feature 🌱 AI growth hinders carbon emissions goals 🧠 GaLore, a memory-efficient method for fine-tuning and pretraining Plus: Andrew Ng shares his concerns about the proposed California regulation SB 1047. Read The Batch now: https://hubs.la/Q02G5YpM0

49

🚀 See how Handshake cut their LLM GPU costs by 50% with Anyscale. Discover how they: 💰 Reduced LLM GPU costs by 50% or more. 📈 Seamlessly scaled large language models (LLMs) without compromising performance. ⏱ Enhanced operational efficiency, enabling faster development cycles. Check out the full story here: https://lnkd.in/gSiKJDaY

37

2 Comments

Combining the power of TensorFlow, NeuralForecast, and zero-shot LLMs, Jinhang Jiang shows how we can generate accurate multi-step quantile forecasts.

11

Alex Punnen's newest article explores how smaller language models (LLMs), like the recently opensourced Meta 1 Billion model, can be effectively utilized to summarize and index large documents, thereby improving the efficiency and scalability of RAG systems. #LLM #RAG

20

SynDL: A Synthetic Test Collection Utilizing Large Language Models to Revolutionize Large-Scale Information Retrieval Evaluation and Relevance Assessment Researchers from University College London, University of Sheffield, Amazon, and Microsoft introduced a new test collection named SynDL. SynDL represents a significant advancement in the field of IR by leveraging LLMs to generate a large-scale synthetic dataset. This collection extends the existing TREC Deep Learning Tracks by incorporating over 1,900 test queries and generating 637,063 query-passage pairs for relevance assessment. The development process of SynDL involved aggregating initial queries from the five years of TREC Deep Learning Tracks, including 500 synthetic queries generated by GPT-4 and T5 models. These synthetic queries allow for a more extensive analysis of query-document relationships and provide a robust framework for evaluating the performance of retrieval systems. The core innovation of SynDL lies in its use of LLMs to annotate query-passage pairs with detailed relevance labels. Unlike previous collections, SynDL offers a deep and wide relevance assessment by associating each query with an average of 320 passages. This approach increases the scale of the evaluation and provides a more nuanced understanding of the relevance of each passage to a given query. SynDL effectively bridges the gap between human and machine-generated relevance judgments by leveraging LLMs’ advanced natural language comprehension capabilities. The use of GPT-4 for annotation has been particularly noteworthy, as it enables high granularity in labeling passages as irrelevant, related, highly relevant, or perfectly relevant..... Read our full take on this: https://lnkd.in/gATdzyyQ Paper: https://lnkd.in/gQGnGbGY

23

Enhancing Self-Supervised Learning with Automatic Data Curation: A Hierarchical K-Means Approach Quick read: https://lnkd.in/giitcGQy Paper: https://lnkd.in/gyN-2sWa

18

Speculative Sampling is the principle of using two language models simultaneously to produce output faster at a reduced cost. https://buff.ly/3WAGKNq Author: https://buff.ly/3JVGHEn

1

An Open-Source Language Model for Evaluating LLMs Researchers have introduced Prometheus 2, a powerful open-source language model specifically designed to evaluate the quality of responses from other language models. This addresses the limitations of proprietary evaluator LMs like GPT-4, offering a transparent, controllable, and affordable alternative. Prometheus 2 surpasses existing open evaluator LMs by: - Achieving high correlation with human judgments and proprietary LMs: Scores closely mirror human evaluations and those of models like GPT-4 and Claude. - Handling both direct assessment and pairwise ranking: Offers flexibility for different evaluation schemes. - Evaluating based on custom criteria: Goes beyond general qualities like helpfulness and harmlessness, allowing for fine-grained assessment tailored to specific needs. Key advancements in Prometheus 2: - Preference Collection: A new dataset with pairwise ranking feedback and 1,000 custom evaluation criteria. - Weight merging: Combines models trained on direct assessment and pairwise ranking data, resulting in a unified evaluator LM that excels in both formats. Prometheus 2 represents a significant step towards a more reliable and accessible evaluation of LLMs, paving the way for further advancements in the field. Read the full paper: https://lnkd.in/dU7sit-Y #Prometheus2 #LLMs #Evaluation #OpenSource #AI ✔️ Click "Follow" on the Cohorte page for daily AI engineering news. Credits: Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, Minjoon Seo

8

1 Comment

Heading into October🍂 - it's been quite a year for txtai. Just this year alone, txtai added: - llama.cpp support - LLM API support (i.e. OpenAI, Claude, Bedrock etc) - Graph RAG - Binary quantization and support for Matryoshka Embeddings - Postgres integration for all vector database components - Streaming LLM, Streaming RAG and chat message support - Significant text extraction improvements There's much more but these are the most crucial changes. It's hard to remember a time when txtai didn't have these features but it wasn't long ago! https://lnkd.in/dxWDeey

8

3 Comments

Advances in Bayesian Deep Neural Network Ensembles and Active Learning for Preference Modeling Quick read: https://lnkd.in/gzZB2VGh Paper 1: https://lnkd.in/gBkwknBZ Paper 2: https://lnkd.in/gNrqnqg9

18