IQuest Research Proposes New Route for LLM Interpretability: Directly Decomposing Weights with Data Cost Under 1%

IQuest Research, in collaboration with multiple universities, proposes Sparse Weight Decomposition (SWD), a method that directly decomposes pretrained weights into intervenable units without training surrogate networks, reducing data cost to under 1%. The method demonstrates efficient circuit extraction on models like GPT-2 and Qwen, offering a new direction for mechanistic interpretability.

+6