This is a post about my takeaways from the DALI workshop on the Goals and Principles of Representation Learning [http://dalimeeting.org/dali2018/workshopRepresentationLearning.html] which we co-organized with DeepMinders Shakir Mohamed [http://shakirm.com/] and Andriy Mnih [https://www.cs.toronto.edu/~amnih/] and Twitter colleague Lucas Theis [http:

I wanted to briefly highlight two recent papers on pruning neural networks (disclaimer, one of them is ours): * Christos Louizos, Max Welling, Diederik P. Kingma (2018) Learning Sparse Neural Networks through $L_0$ Regularization [https://arxiv.org/abs/1712.01312] * Lucas Theis, Iryna Korshunova, Alykhan Tejani, Ferenc Huszár (2018) Faster

After last week's post on the generalization mystery, people have pointed me to recent work connecting the Fisher-Rao norm to generalization (thanks!): * Tengyuan Liang, Tomaso Poggio, Alexander Rakhlin, James Stokes (2017) Fisher-Rao Metric, Geometry, and Complexity of Neural Networks [https://arxiv.org/abs/1711.01530] This short presentation

I set out to write about the following paper I saw people talk about on twitter and reddit: * Hao Li, Zheng Xu, Gavin Taylor, Tom Goldstein Visualizing the Loss Landscape of Neural Nets [https://openreview.net/pdf?id=HkmaTz-0W] It's related to this pretty insightful paper: * Laurent Dinh,

Like many of you, I thoroughly enjoyed Ali Rahimi's NIPS talk [https://www.youtube.com/watch?v=ORHFOnaEzPc] in response to winning the test-of time award for their work on random kitchen sinks. I recommend everyone to watch it if you haven't already. I also read

I wrote this post in response to a challenge by Roger Grosse: > Theorem 2 of my moment averaging paper is one of the most surprising mathematical results I've seen in machine learning. The proof is very short, but I still have no intuition for why it'

In the spirit of thanksgiving, let me start by thanking all the active commenters on my blog: you are always very quick to point out typos, flaws and references to literature I overlooked. This post is basically a follow-up my earlier post, "GANs are broken in more than one

This post is just a quick note on some of the pitfalls we encounter when dealing with high-dimensional problems, even when working with something as simple as a Gaussian distribution. Last week I wrote about mixup [https://www.inference.vc/mixup-data-dependent-data-augmentation/] a new data-augmentation scheme that achieves good performance in

By popular demand, here is my post on mixup, a new data augmentation scheme that was shown to improve generalization and stabilize GAN performance. * H Zhang, M Cisse, YN Dauphin and D Lopez-Paz (2017) mixup: Beyond Empirical Risk Minimization [https://arxiv.org/abs/1710.09412] I have to say I