The Future of Software The world of software is undergoing a shift not seen since the advent of compilers in the 1970s. Compilers were the original vibe coding: they automatically generate complex machine code that human programmers had to manually write before. Over time, compilers became fully trusted, nobody has to look under the
deep learning Deep Learning is Powerful Because It Makes Hard Things Easy - Reflections 10 Years On Ten years ago this week, I wrote a post called "Deep Learning is Easy - Learn Something Harder". The post blew up, top spot on HackerNews. Needless to say, it didn't age well.
Discrete Diffusion: Continuous-Time Markov Chains A tutorial explaining some key intuitions behind continuous time Markov chains for machine learners interested in discrete diffusion models: alternative representations, connections to point processes, and the memoryless property.
We may finally crack Maths. But should we? Automating mathematical theorem proving has been a long standing goal of artificial intelligence and indeed computer science. It's one of the areas I became very interested in recently. This is because I feel we may have the ingredients needed to make very, very significant progress: 1. a structured
AGI Mortal Komputation: On Hinton's argument for superhuman AI. Last week in Cambridge was Hinton bonanza. He visited the university town where he was once an undergraduate in experimental psychology, and gave a series of back-to-back talks, Q&A sessions, interviews, dinners, etc. He was stopped on the street by random passers-by who recognised him from the lecture,
Autoregressive Models, OOD Prompts and the Interpolation Regime A few years ago I was very much into maximum likelihood-based generative modeling and autoregressive models (see this, this or this). More recently, my focus shifted to characterising inductive biases of gradient-based optimization focussing mostly on supervised learning. I only very recently started combining the two ideas, revisiting autoregressive models
We May be Surprised Again: Why I take LLMs seriously. "Deep Learning is Easy, Learn something Harder" - I proclaimed in one of my early and provocative blog posts from 2016. While some observations were fair, that post is now evidence that I clearly underestimated the impact simple techniques will have, and probably gave counterproductive advice. I wasn&
Implicit Bayesian Inference in Large Language Models This intriguing paper kept me thinking long enough for me to I decide it's time to resurrect my blogging (I started writing this during ICLR review period, and realised it might be a good idea to wait until that's concluded) * Sang Michael Xie, Aditi Raghunathan, Percy
Eastern European Guide to Writing Reference Letters Excruciating. One phrase I often use to describe what it's like to read reference letters for Eastern European applicants to PhD and Master's programs in Cambridge. Even objectively outstanding students often receive dull, short, factual, almost negative-sounding reference letters. This is a result of (A) cultural
causal inference Causal inference 4: Causal Diagrams, Markov Factorization, Structural Equation Models This post is written with my PhD student and now guest author Patrik Reizinger [https://twitter.com/rpatrik96] and is part 4 of a series of posts on causal inference: * Part 1: Intro to causal inference and do-calculus [https://www.inference.vc/untitled] * Part 2: Illustrating Interventions with a Toy
