Daily digest — 2026-09-05

YouTube (10)

Papers (10)

  • DistillLens: Symmetric Knowledge Distillation Through Logit Lens — Manish Dhakal; Uthman Jinadu; Anjila Budathoki; Rajshekhar Sunderraman; Yi Ding Standard Knowledge Distillation (KD) compresses Large Language Models (LLMs) by optimizing final outputs, yet it typically treats the teacher’s intermediate layer’s thought process as a black box. While feature-based distillation attempts to bridge this gap, existing methods…
  • Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models — Jinghan Cao; Yu Ma; Xinjin Li; Qingyang Ren; Xiangyun Chen Large Language Models achieve remarkable performance but incur substantial computational costs unsuitable for resource-constrained deployments. This paper presents the first comprehensive task-specific efficiency analysis comparing 16 language models across five diverse NLP…
  • Low-bit Model Quantization for Deep Neural Networks: A Survey — Kai Liu; Qian Zheng; Kaiwen Tao; Zhiteng Li; Haotong Qin; Wenbo Li; Yong Guo; Xianglong Liu; Linghe Kong; Guihai Chen; Yulun Zhang; Xiaokang Yang With unprecedented rapid development, deep neural networks (DNNs) have deeply influenced almost all fields. However, their heavy computation costs and model sizes are usually unacceptable in real-world deployment. Model quantization, an effective weight-lighting technique,…
  • Compensate Quantization Errors+: Quantized Models Are Inquisitive Learners — Yifei Gao; Jie Ou; Lei Wang; Jun Cheng; Mengchu Zhou The quantization of large language models (LLMs) has been a prominent research area aimed at enabling their lightweight deployment in practice. Existing research about LLM’s quantization has mainly explored the interplay between weights and activations, or employing auxiliary…
  • Enhancing Human-Like Responses in Large Language Models — Ethem Yağız Çalık; Talha Rüzgar Akkuş This paper explores the advancements in making large language models (LLMs) more human-like. We focus on techniques that enhance natural language understanding, conversational coherence, and emotional intelligence in AI systems. The study evaluates various approaches,…
  • RecoAtlas: From Semantic Plausibility to Set-Level Utility in LLM Recommendation Agents — Imad Aouali; Flavian Vasile; Otmane Sakhi; Alexandre Gilotte; Benjamin Heymann LLM recommendation agents increasingly produce structured recommendation reports: sets of items accompanied by natural-language justifications. Yet existing evaluations often reduce this setting to reranking small shortlisted candidate sets or judge reports mainly by semantic…
  • Agents at Risk: How Users Unwittingly Undermine LLM Safety — Fengchao Chen; Tingmin Wu; Van Nguyen; Surya. Nepal; Carsten Rudolph Large language model (LLM)-based agents are increasingly deployed in applications, such as trip-planning agents and web-use agents, to perform complex planning and execution tasks. Prior work has shown that LLM-based agents are vulnerable to context confusion, where external…
  • Uncertainty Decomposition for Clarification Seeking in LLM Agents — Gregory Matsnev Recent position papers argue that the classical aleatoric/epistemic uncertainty framework is insufficient for interactive large language model (LLM) agents and call for underspecification-aware, decomposed, and communicable uncertainty representations that can unlock new…
  • Triplet Loss for Knowledge Distillation — Hideki Oki; Motoshi Abe; Junichi Miyao; Takio Kurita In recent years, deep learning has spread rapidly, and deeper, larger models have been proposed. However, the calculation cost becomes enormous as the size of the models becomes larger. Various techniques for compressing the size of the models have been proposed to improve…
  • Small but Significant: On the Promise of Small Language Models for Accessible AIED — Yumou Wei; Paulo Carvalho; John Stamper GPT has become nearly synonymous with large language models (LLMs), an increasingly popular term in AIED proceedings. A simple keyword-based search reveals that 61% of the 76 long and short papers presented at AIED 2024 describe novel solutions using LLMs to address some of…

Hacker News (20)