• Deep Neural Nets: 33 years ago and 33 years from now
    To my knowledge, LeCun et al. 1989 is the earliest real-world application of a neural net trained end-to-end with backpropagation. Can we improve on it using 33 years of progress in deep learning? What does 1989 deep learning look like to someone in 2022, and what will today's deep learning look like to someone in 2055?
  • A from-scratch tour of Bitcoin in Python
    We are going to create, digitally sign, and broadcast a Bitcoin transaction in pure Python, from scratch, and with zero dependencies
  • Short Story on AI: Forward Pass
    A story of an existential crisis under the hood of a humble forward pass.
  • Biohacking Lite
    Dipping toes into biochemistry, energy metabolism and running some biohacking lite experiments.
  • A Recipe for Training Neural Networks
    A collection of practical advice for the process of achieving strong results with neural networks.
  • (started posting on Medium instead)
    Yes I'm still around but, I've started posting on Medium instead of here.
  • A Survival Guide to a PhD
    A collection of tips/tricks for navigating the PhD experience.
  • Deep Reinforcement Learning: Pong from Pixels
    I'll discuss the core ideas, pros and cons of policy gradients, a standard approach to the rapidly growing and exciting area of deep reinforcement learning. As a running example we'll learn to play ATARI 2600 Pong from raw pixels.
  • Short Story on AI: A Cognitive Discontinuity.
    The first part of a short story collection that has been on my mind for a long while. Exciting! :)
  • What a Deep Neural Network thinks about your #selfie
    We will look at Convolutional Neural Networks, with a fun example of training them to classify #selfies as good/bad based on a scraped dataset of 2 million selfies.
  • The Unreasonable Effectiveness of Recurrent Neural Networks
    We'll train and sample from character-level RNN language models that learn to write poetry, latex math and code. We'll also analyze the models and get hints of future research directions.
  • Breaking Linear Classifiers on ImageNet
    There have been a few recent papers that fool ConvNets by taking a correctly classified image and perturbing it in an imperceptible way to produce an image that is misclassified. In this post I show that ConvNets are an overkill: Simple linear classifiers are in fact susceptible to the same fooling strategy.
  • What I learned from competing against a ConvNet on ImageNet
    The latest state of the art Image Classification networks have only 6.7% Hit@5 error on ILSVRC 2014 classification task. How do humans compare?
  • Quantifying Productivity
    Describing a new pet project that tracks active windows and keystroke frequencies over the duration of a day (on Ubuntu/OSX) and creates pretty HTML visualizations of the data. This allows me to gain nice insights into my productivity. Code on Github.
  • Feature Learning Escapades
    Some reflections on the last two years of my research: The Quest for Unsupervised Feature Learning algorithms for visual data. Where it was, where it is, and where it's going. Maybe.
  • Visualizing Top Tweeps with t-SNE, in Javascript
    A writeup of a recent mini-project: I scraped tweets of the top 500 Twitter accounts and used t-SNE to visualize the accounts so that people who tweet similar things are nearby. My final Javascript implementation of t-SNE is released on Github as tsnejs.
  • Switching Blog from Wordpress to Jekyll
    I can't believe I lasted this long on Wordpress. I am switching permanently to Jekyll for hosting my blog, and so should you :) Details inside.
  • Interview with Data Science Weekly on Neural Nets and ConvNetJS
    I gave a (long) interview about my background and perspectives on neural nets.
  • Quantifying Hacker News with 50 days of data
    I scraped Hacker News Front Page and New Page every minute for 50 days and analyzed the results. How do stories rise and fall on Hacker News? What makes a successful post? Find out in this post :)
  • Chrome Extension Programming: Illustrating a Basic Survival Skill with a Twitter Case Study
    I illustrate a very valuable skill (Chrome Extension Programming) using a Twitter Case study. We will give Twitter a face lift, get it to refresh new tweets automatically, and highlight tweets from people who rarely tweet. All with a few lines of Javascript!
  • The state of Computer Vision and AI: we are really, really far away.
    A depressing look at the state of Computer Vision Research and AI in general. For those who like to think that AI is anywhere close.
  • Lessons learned from manually classifying CIFAR-10
    CIFAR-10 is a popular dataset small dataset for testing out Computer Vision Deep Learning learning methods. We're seeing a lot of improvements. But what is the human baseline?