Embedding
Embedding is a technique that transforms unstructured data such as text, images, and audio into fixed-length numerical vectors while preserving semantic relationships.
A computer cannot determine from raw strings that "apple" and "orange" are similar. Embedding solves this problem. When "apple" is converted into a vector like [0.23, -0.41, 0.87, ...] with hundreds of dimensions, the vector for "orange" is close by while "automobile" is far away. Semantic closeness becomes numerical closeness.
Embeddings play a core role inside LLMs as well. Input text is first tokenized, and each token is converted into an embedding vector. The Transformer processes this sequence of vectors to generate output.
In practice, sentence-level embeddings are used most frequently. Models such as OpenAI's text-embedding-3-small and Cohere's embed-v4 convert entire sentences into single vectors. Storing these vectors in a vector database enables semantic search and the retrieval layer for RAG.
When selecting a model, dimensionality, supported languages, and cost are the key criteria. For Japanese or Thai language processing, benchmarking multilingual model accuracy beforehand is important.
Articles covering this term
- What is a Vector Database? A Complete Guide to How It Works, Top Product Comparisons, and RAG ApplicationsFrom basic vector database concepts to comparing top products like Pinecone, Weaviate, and pgvector, plus RAG system integration—a clear guide for AI implementation managers.
- What is Hybrid Search? How It Works and Implementation to Improve RAG Accuracy with Vector Search × Full-Text SearchExplains hybrid search using vector search, BM25, and RRF. Practical design patterns and implementation tips to improve RAG system retrieval accuracy.
- What is Semantic Caching? How AI Gateways Reduce LLM Costs and Implementation GuideLearn how to cache AI responses using semantic similarity to reduce LLM calls for duplicate queries. Covers Semantic Cache integration with AI Gateway and cost savings estimates.
- 10 RAG Implementation Failure Patterns and How to Avoid Them — Preventing Production Issues Before They Happen10 common RAG failures before & after production: chunk design errors, retrieval accuracy drops, hallucination fixes & more. Learn key avoidance strategies before you build.
Related Terms
RRF
RRF (Reciprocal Rank Fusion) is a scoring method that integrates ranking results returned by multipl
Agentic RAG
Agentic RAG is an architecture in which an LLM autonomously and iteratively generates search queries
GraphRAG
A next-generation RAG architecture that combines knowledge graphs and vector search, leveraging rela
Gemini Embedding 2
Gemini Embedding 2 is a multimodal embedding model developed by Google, capable of converting text,