Press enter or click to view image in full size

Discover how major decoding methods and algorithms work and their usages with practical examples

Kuriko Iwai

14 min read

·

6 days ago

--

2

Share

Introduction

Decoding strategies are essential guides used in Large Language Models (LLMs) to transform the conditional probability distributions into coherent, high-quality text.

Different strategies influence the characteristics of the LLM’s output, including its quality, creativity, consistency, and variability.

In this article, I’ll delve into the decoding strategies, analyzing underlying mechanisms, usage, and limitations of major decoding methods and algorithms.

Table of Contents

LLM Decoding Basics: The Probability Distribution

Implementing Decoding Strategies
∘ A Hierarchical Breakdown

Decoding Methods: Controlling Probability Distributions
∘ Sampling Methods
∘ Top-k Sampling
∘ Top-p Sampling (Nucleus Sampling)
∘ Temperature Scaling
∘ Repetition and Frequency Penalties