Rivindu Weerasekera

Earlier this year we saw some really exciting research around Mamba, a selective SSM model architecture. So it's no surprise that we're starting to see some additional model architectures that promise to rival transformers in performance and efficiency. Researchers from Microsoft Research and Tsinghua University have developed YOCO (You Only Cache Once), a decoder-decoder architecture for large language models. This architecture appears to show some really exciting performance and efficiency gains over traditional transformers. Here are the top three takeaways: 1. Memory Efficiency: YOCO dramatically reduces GPU memory demands by caching key-value pairs just once. This means these models can handle long sequences without exhausting GPU memory, even managing up to 1M tokens efficiently! 2. Speed & Performance: The design significantly cuts down prefill latency and boosts throughput. YOCO slashes prefilling time by up to 30.3x and enhances throughput by 9.6x, demonstrating extremely fast model response times. 3. Scalability: YOCO excels at scaling up model sizes, training tokens, and context length. It performs competitively with traditional transformers across various metrics. It will be really exciting to see how new architectures like Mamba and YOCO can help in reducing training costs and improving performance. Read the full paper here: https://lnkd.in/g838mSPy #AI #MachineLearning #TechInnovation HOPPR

16

Looking forward to my #keynote tomorrow, titled "Enhancing #Foundation_Models with #Knowledge_Graphs: The Future of Context-Aware AI" at the #IEEE_CSDE Conference in beautiful Perth. In the age of #foundation_models dominating AI research and applications, the question isn’t just how we scale but how we enhance understanding, reasoning, and context-awareness. This keynote will explore: (i) How #Knowledge_Graphs can bring depth, accuracy, and interpretability to foundation models; (ii) The potential for #context_aware AI systems to revolutionize industries such as healthcare, education, and finance; and (iii) The balance between leveraging domain-specific knowledge and maintaining the flexibility of general-purpose AI. Centre for Applied AI: https://lnkd.in/gTvJCYTj Data Science Lab: https://lnkd.in/f6gAqas

107

1 Comment

Day 21: Reinforcement Learning: Shaping AI Through Rewards for #100DaysOfMLChallenge Today, we shift gears to explore reinforcement learning (RL), a fascinating area of AI inspired by how we learn through trial and error. Imagine an agent navigating a maze. In RL, the agent interacts with the environment, taking actions and receiving rewards (positive for good actions, negative for bad ones). Through this feedback loop, the agent learns to optimize its behavior to maximize the rewards. Here's what makes RL unique: No explicit instructions: Unlike supervised learning, RL agents don't receive step-by-step instructions. They learn through exploration and experimentation. Delayed rewards: The consequences of an action might not be immediate, requiring the agent to consider long-term goals. Adapting to dynamic environments: The environment can change, forcing the agent to continuously learn and adapt its strategies. The potential applications of RL are vast, with real-world examples already emerging in: Robotics: Training robots to perform complex tasks in dynamic environments. Game playing: AlphaGo, anyone? RL agents are mastering complex games like chess and Go. Resource management: Optimizing resource allocation in complex systems like logistics or energy grids. What are your thoughts on reinforcement learning? Are there any specific applications that excite you? Share your thoughts in the comments! #ReinforcementLearning #AI #MachineLearning #TrialAndError #Robotics

7

I will be presenting the poster entitled "The Neural-SRP method for robust universal sound source tracking" at ICASSP 2024 tomorrow at 8:30. I realise this is very early, so if science is not a big enough motivator, the first attendees will be getting a Ferrero Rocher 🍫!  I am excited about Neural-SRP, as we show it to be a high performance, low complexity alternative to state-of-the-art sound localization methods while also being able to work on multiple microphone geometries, allowing companies with many different arrays on their product lines to use a single model. Hope to see you soon! https://lnkd.in/eS4edJwc

62

4 Comments

Deep reinforcement learning (DRL) is an artificial intelligence approach that pertains to developing programs that can solve problems that require intelligence. Intelligence is something that we are obsessed with replicating outside our brains. In fact we associate the term artificial intelligence with this capability, and strive to attain Artificial General Intelligence (AGI) at some point. While DRL might still not be what we can call true AGI on its own, but it does mimic some form of intelligence. Specifically the decision making intelligence used by many species. The purpose of this article series is to demystify deep reinforcement learning, one of the most powerful approaches in artificial intelligence that leverages deep learning, and combined with other approaches, can help humans plan businesses, specifically operational and tactical planning, with zero defects. #data #analytics #ai #ml #datascience #designedanalytics #deeplearning #reinforcementlearning #deepreinfrocementlearning https://lnkd.in/geTA4gqJ

2

1 Comment

🔍 Google's Gemma Scope: Illuminating the Inner Workings of Large Language Models 🧠 Google researchers have developed a powerful new tool called Gemma Scope, designed to shed light on how each layer in Gemma 2 large language models responds to input tokens. 📊 Key insights: - Sparse autoencoders (SAEs) can transform embeddings into interpretable representations - Each index in the transformed embedding corresponds to a distinct concept - SAE weights indicate the strength of each concept in the input Gemma Scope enables: - Manual and automatic labeling of concepts in each layer - Steering the model by adjusting SAE outputs to generate concept-specific text This groundbreaking tool paves the way for answering critical questions about LLMs, such as how fine-tuning and chain-of-thought prompting influence a model's internal representations. 🤔 #GemmaScope #LargeLanugageModels #TransformerInterpretability #GoogleAI #MachineLearningResearch

1

LLM Evaluation Metrics: Everything You Need for LLM Evaluation - Confident AI https://bit.ly/3TYhkaP

30

Exciting Research Alert: Is RAG Always the Answer? Team at National Chengchi University and Academia Sinica just published groundbreaking research challenging conventional wisdom in LLM architecture. The big question: Do we always need real-time retrieval (RAG) for knowledge tasks? Findings: Not necessarily! Cache-Augmented Generation (CAG), a simpler yet powerful alternative that: Eliminates retrieval latency Reduces system complexity Achieves comparable or better results than RAG Cuts generation time by up to 40x (94s → 2.3s in tests) Technical Highlights: Architecture: CAG precomputes KV-cache by encoding external knowledge (D → CKV = KV-Encode(D)), eliminating real-time retrieval overhead during inference. Performance Metrics: BERTScore improvements across both SQuAD and HotPotQA benchmarks Generation time reduced from 94.34s to 2.32s on HotPotQA-large Superior context coherence due to holistic document processing Key Innovation: Instead of traditional {query → retrieve → generate} pipeline, CAG leverages extended context windows to preload knowledge and cache runtime parameters, enabling R = M(Q|CKV) direct inference. Experimental Validation: Tested against both sparse (BM25) and dense (OpenAI Indexes) retrieval Consistent outperformance across varying document sizes (21k-85k tokens) Particularly effective for bounded knowledge domains Paper: "Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks" (https://lnkd.in/e8D2d7zz)

57

🚀 MegaParse - The Ultimate Document Parser for RAG MegaParse is a robust Python library tailored to parse diverse document types with ease and efficiency. 🎯 Key Features: 1️⃣ Versatile Parsing: Supports multiple document formats without hassle. 2️⃣ No Data Loss: Ensures that no information is overlooked during parsing. 3️⃣ Fast and Efficient: Optimized for speed and accuracy. 4️⃣ Wide Compatibility: Works seamlessly with Text, PDF, Word, Excel, CSV, PPTX, and more. 5️⃣ Open Source: Fully free to use and contribute to, fostering a community-driven ecosystem. Perfect for projects like Retrieval-Augmented Generation (RAG) or document-intensive workflows. 💡 #megaparse #llm #pdfparsing #aiagents Explore its potential and make your parsing tasks smarter! https://lnkd.in/giNVAUR8

27

🚀 Advancing Information Retrieval with APEER: Automating Prompt Engineering for LLMs! 🚀 I recently delved into a fascinating paper on APEER (Automatic Prompt Engineering Enhances LLM Reranking), a method transforming the way we approach relevance ranking in Information Retrieval (IR) using Large Language Models (LLMs). 🌟 🔍 Challenges in IR with LLMs: Current IR methods with LLMs often rely heavily on human-crafted prompts for zero-shot relevance ranking. This process is time-consuming, subjective, and lacks scalability. The complexities of integrating query and passage pairs and conducting comprehensive relevance assessments further complicate the effectiveness of existing methods. 💡 APEER: A Game-Changer APEER addresses these challenges by automating prompt engineering through iterative feedback and preference optimization. Here's how it stands out: Feedback Optimization: Continuously refine prompts based on performance metrics. Preference Optimization: Improves prompts by learning from positive and negative examples. 🌟 Key Highlights: Reduced Human Effort: APEER minimizes the need for manual prompt crafting, making the process more efficient and less reliant on human expertise. Enhanced Performance: Demonstrates significant improvements in LLM performance on IR tasks, with notable gains in metrics like nDCG@10. For example, APEER achieved an average improvement of 5.29 nDCG@10 on eight BEIR datasets over manual prompts on the LLaMA3 model. Better Transferability: Shows consistent outperformance across diverse datasets and LLM architectures, including GPT-4, LLaMA3, and Qwen2. 📊 Robust Validation: APEER has been rigorously tested on multiple datasets such as MS MARCO, TREC-DL, and BEIR, ensuring its robustness and effectiveness in various IR scenarios. 🌐 Implications: This advancement represents a significant step forward in optimizing LLM prompts for complex relevance ranking tasks. By reducing manual intervention and enhancing the efficiency of LLMs, APEER paves the way for more scalable and accurate applications in real-world IR tasks. 🛠️ Conclusion: APEER's automated approach to prompt engineering is a major leap towards more effective and scalable IR systems. It highlights the potential of integrating iterative feedback and preference optimization to refine LLM performance, offering promising avenues for future research and practical applications. Looking forward to seeing how APEER influences the landscape of Information Retrieval and beyond! #InformationRetrieval #LLM #AI #MachineLearning #NLP #DataScience #Research

1

On data centric vs algorithmic centric rlhf work This year we've had two major projects for our state-of-the-art post training pipelines (Tulu 2.5 and 3 soon) at Ai2. One has been more data focussed and one was focussed on trying to get performance from PPO. It's amazing how much smoother and more predictable progress is when working in a data-centric point of view for post training. For tulu 2.5, the innovation to make PPO more stable or faster require very in depth algorithmic knowledge that falls on one or two people, and it adds a lot of uncertainty into the process. It took months and makes these few people disproportionately stressed. For tulu 3, which we are working on now, we've been able to incorporate way more people into the pipeline and see immediate gains on evals we care about, like GSM8k, MATH, IFEval, etc, by curating and filtering the right data. I suspect for tulu 3 we spend a similar amount of person hours, by including more people in a shorter time window, and get 2x plus the gains in performance. This is why people tell you not to read too into the "DPO vs PPO" in the llama 3.1 paper. They're just slowly honing the blades of their datasets. One week at a time, a few thousand examples at a time, until they have a great end product. Way less risk for the model to focus on data. Way more risk for "academic novelty" to not fiddle with algorithms. Models are more impactful anyways, and for some reason their are fewer people publishing their data recipes in the open.

60

2 Comments

GPT-4o says "9.11 is bigger than 9.9". It is extremely fascinating how AI models which can understand quantum mechanics, have read books from Tolstoy to Principia, and are now achieving super-human scores on knowledge benchmarks can fail at something as trivial as interpreting the decimal point system. It would almost be like if a top scientist made groundbreaking discoveries in physics and then said 1x1=2 *. These issues are at the heart of why AI models are considered unreliable. To build a world where AI models are ubiquitous, we need to figure this out ASAP. I think the most important step here is understanding WHY these mistakes happen. To this end, the recent interpretability work done by Anthropic will be crucial. https://lnkd.in/gvMKkGXX Further thoughts in comments. * Any resemblances to actual people are purely coincidental 😜 #AI #Anthropic #LLMs

58

28 Comments

A new research paper from Apple shows that large language models (LLMs) struggle with basic math problems when irrelevant details are introduced, leading to significant drops in accuracy. The paper demonstrates that adding simple distractions, like a comment about the size of kiwis in a math problem, can confuse AI models, highlighting their inability to perform genuine logical reasoning. Researchers argue that AI models don’t reason like humans but rather replicate observed patterns from their training data, which leads to failures when problems deviate slightly from the norm. Some suggest that better prompts could improve performance, but the need for more context as distractions grow suggests limitations in AI’s current reasoning capabilities. The study sparks further debate on whether LLMs truly "reason" or just mimic the data they have been exposed to, raising concerns about how AI is being marketed and used today.

2

1 Comment

Simple and clear! Cyclical encoding to represent cyclical data (hours, month, moon cycles) through the use of trigonometric functions. https://lnkd.in/gqTiRK3s

1

💡Eureka Insight Day 5: This is a meta point about measurements in Eureka ML Insights. Several datasets in Eureka-Bench were procedurally/dynamically generated, including: GeoMeter, Image Understanding, Vision Language Understanding, Kitab, and Toxigen. This means that it is possible to generate other new samples of a similar nature in the future and run comparative measurements, whenever concerns about memorization arise. Of course, given that training data access is not possible for most models, it is not possible to guarantee that any public benchmark was not used during training. However, these two benchmark selection criteria can be useful while thinking about the composition of collection of benchmarks like Eureka-Bench: 🎯The benchmark should be challenging even for the most capable models to date. 🎯The benchmark can be re generated or transformed in a procedural/dynamic way such that performance can be compared between previous and newer versions of the benchmark to test for saturation and memorization. 🎯Other important criteria for selection may include capability coverage, usefulness for composing more complex and real-word tasks, potential label noise etc. It is important to acknowledge that some of these criteria may be a deal breaker for certain capabilities and applications, so the process of “benchmark deprecation” will become more prevalent as we make more progress in AI and our expectations from AI continue to increase. We look forward to enriching Eureka across all these aspects, and are eager to learn more from the community: how do you all select on which benchmarks to evaluate? 🌱 More about Eureka here: https://lnkd.in/gAqpqZ39

30

2 Comments

Our new Hybrid model is on arxiv!

31

How covariance and correlation differ? Curious about how two variables move together in data? Dive into the key differences between correlation and covariance in this blog! 🔗 https://lnkd.in/gMYPGS6S Rahul Balwan StatisticaHub

16

3 Comments

If you have access to multiple GPUs, such as those available through Australia's National Computational Infrastructure (NCI), leveraging PyTorch’s Distributed Data Parallel (DDP) can significantly enhance your deep learning projects. Efficiently utilizing multiple GPUs is crucial for reducing training times and improving model performance. For researchers aiming to leverage NCI’s resources for multi-GPU setups, the journey can be greatly rewarding. Many companies with large datasets and multiple GPUs require expertise in using multi-GPU setups to train large models effectively.

16