
MoE (Mixture of Experts)
MoE (Mixture of Experts) is an architecture that contains multiple "expert" subnetworks within a model, activating only a subset of them for each input, thereby increasing the total number of parameters while keeping inference costs low.











