Alex Xi's Notes
Search
Search
Dark mode
Light mode
Tag: architecture
18 items with this tag.
Jul 06, 2026
tweet-sparse-attention-anns
attention
sparse-attention
anns
retrieval
architecture
Jul 06, 2026
tweet-subq-sub-quadratic-llm
architecture
attention
long-context
efficiency
Jul 06, 2026
tweet-anti-moe-take
moe
architecture
contrarian
Jul 06, 2026
tweet-attention-free-model-14b
architecture
attention-free
transformer
alternative
efficiency
Jul 06, 2026
tweet-causal-jepa-icml
jepa
self-supervised-learning
icml
architecture
Jul 06, 2026
tweet-deepseek-v4-analysis
deepseek
architecture
moe
scaling
Jul 06, 2026
tweet-deepseek-v4-hbm
deepseek
architecture
hbm
hardware
Jul 06, 2026
tweet-gemma4-architecture
architecture
gemma
google
transformer
model-design
Jul 06, 2026
tweet-group-representational-pos-encoding
positional-encoding
architecture
jane-street
Jul 06, 2026
tweet-moe-training-instability
moe
training-stability
architecture
Jul 06, 2026
tweet-parallel-agent-context-search
agents
parallelism
context
search
architecture
Jul 06, 2026
mixture-experts-architectures
mixture-of-experts
moe
architecture
scaling
Jul 06, 2026
openai-reverse-engineered
openai
o1
architecture
reverse-engineering
Jul 06, 2026
recursive-language-models-2026
recursive-models
training
architecture
self-improvement
Jul 06, 2026
differential-transformer
differential-attention
transformer
architecture
attention
Jul 06, 2026
how-attention-residuals-work
transformers
attention
architecture
Jul 06, 2026
attention-wrong-norm
attention
transformers
architecture
normalization
sequence-length
Jul 06, 2026
rag-vs-compiled-knowledge
rag
knowledge-management
llm
architecture