AI Fundamentals
A curated glossary category for "AI Fundamentals" terms. Practical definitions for generative AI, LLM, RAG, AI agents, DX, and cybersecurity — with examples for executives and IT teams driving real-world adoption.
25 terms
Context Window
A context window refers to the maximum number of tokens an LLM can process at one time, indicating t
Fine-tuning
Fine-tuning refers to the process of providing additional training data to a pre-trained machine lea
Generative AI (Generative AI)
Generative AI is a collective term for AI models capable of autonomously generating content such as
LoRA
LoRA (Low-Rank Adaptation) is a technique that inserts low-rank delta matrices into the weight matri
Multilingual NLP (Multilingual Natural Language Processing)
Multilingual NLP is a natural language processing technology capable of analyzing and generating tex
Multimodal AI (Multimodal AI)
Multimodal AI refers to an AI system capable of integrating, processing, understanding, and generati
PEFT
PEFT (Parameter-Efficient Fine-Tuning) is a collective term for fine-tuning methods that adapt a lar
QLoRA
QLoRA (Quantized LoRA) is a method that combines LoRA with 4-bit quantization, enabling fine-tuning
RLHF
RLHF is a reinforcement learning method that uses human feedback as a reward, while RLVR is a reinfo
SLM (Small Language Model)
SLM (Small Language Model) is a general term for language models with a parameter count limited to a
MoE (Mixture of Experts)
MoE (Mixture of Experts) is an architecture that contains multiple "expert" subnetworks within a mod
LLM (Large Language Model)
LLM (Large Language Model) is a general term for neural network models pre-trained on massive amount
Open-weight model
An open-weight model is a language model whose trained weights (parameters) are publicly released an
Inference-time Scaling (Test-time Compute)
Inference-time scaling is a technique that dynamically increases or decreases the amount of computat
Reasoning Model
A type of large language model that generates an explicit chain of thought before responding, solvin
Sparse Model
A Sparse Model is a general term for neural network architectures that activate only a subset of the
TurboQuant
A memory compression technology for LLMs developed by Google. It reduces memory consumption by up to
Knowledge Distillation (Knowledge Distillation)
A technique that transfers knowledge from a large teacher model to a small student model, creating a
Dense Model (Tightly Coupled Model)
A Dense Model is a neural network architecture in which all of the model's parameters are used for c
Speculative Decoding
A inference acceleration technique in which a small draft model proposes multiple tokens speculative
Token (Token)
A token is the smallest unit used by an LLM when processing text. It is not necessarily a whole word
BPE Tokenizer (Byte-Pair Encoding Tokenizer)
An algorithm that merges text based on frequent patterns and splits it into subword units. It direct
Base Model (Foundation Model)
A base model (Foundation Model) is a general-purpose AI model pre-trained on large-scale datasets. R
Quantization (Quantization)
An optimization technique that compresses model size by reducing parameter precision from 16-bit to
Local LLM
A local LLM refers to an operational model in which a large language model is run directly on one's