News
AI, cybersecurity, machine learning, scraping & programming news.
14 results on this page · clear filters
Probabilistic Linear Explanations
When a machine learning model denies someone a loan or flags a medical scan, the most natural question is "why?" Formal explainability has spent years trying to answer that question with mathematical rigor. The dominant approach f...
Double descent is the principle of least action
The double descent phenomenon is one of the most counterintuitive findings in modern machine learning. Train a model with a small number of parameters and test error falls as you add more, following the classical bias-variance cur...
Knowledge as Orbit: Finite Collections as Phases of an Exactly Periodic Latent Generator
Machine learning stores knowledge extensionally. Every item in a collection gets its own code, its own latent vector, its own row in a codebook. For X objects, you need X entries. The question this paper asks is whether a finite c...
Type Diversity Enables Transformers to Generalise Compositionally
The compositional generalisation problem has haunted Transformer research for years. Train a seq2seq model on "John drinks coffee" and it learns to translate that just fine. But give it "John drinks coffee on a mat" when only "on...
General Quantification of Covariate and Concept Shifts
Distribution Shift Is a Broken Promise Every deployed model makes an implicit bet: that the data it sees at inference will look like the data it trained on. When that bet fails, performance degrades silently and sometimes catastro...
Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
When Running Out of Data Breaks Sparse Models The internet's text is finite. As language model training scales toward and beyond the limits of human-written corpora, repeating data has become standard practice. Muennighoff et al....
Distance generalization in transformers: why bother with positional encoding?
Positional Encoding May Be Hurting Your Transformer Transformers are everywhere, and positional encoding is considered foundational. Every major production model, from Llama to DeepSeek, uses RoPE (Rotary Position Embeddings) to t...
TART: A Modular Tool for Technique-Aware Audio-to-Tablature Guitar Transcription
Automatic Music Transcription for guitar has long been stuck on the wrong side of the difficulty curve compared to piano. Piano AMT hit high note-level accuracy years ago through CRNN models and large paired datasets, but guitar t...
The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement
Modern AI systems can write code, reason through graduate-level science, and plan multi-step research workflows. But every improvement they undergo still requires humans to decide what to fix, build the resources to fix it, and ju...
Model-Aware Schedules Improve Generation via Fiberwise Optimal Transport
Diffusion models and flow-matching models generate images by progressively denoising a noisy sample over a series of timesteps. How that time is allocated across the denoising process, the schedule, has a direct impact on sample q...