Andrej Karpathy blog
  • Mar 14, 2022 Deep Neural Nets: 33 years ago and 33 years from now
    To my knowledge, LeCun et al. 1989 is the earliest real-world application of a neural net trained end-to-end with backpropagation. Can we improve on it using 33 years of progress in deep learning? What does 1989 deep learning look like to someone in 2022, and what will today's deep learning look like to someone in 2055?
  • Jun 21, 2021 A from-scratch tour of Bitcoin in Python
    We are going to create, digitally sign, and broadcast a Bitcoin transaction in pure Python, from scratch, and with zero dependencies
  • Mar 27, 2021 Short Story on AI: Forward Pass
    A story of an existential crisis under the hood of a humble forward pass.
  • Jun 11, 2020 Biohacking Lite
    Dipping toes into biochemistry, energy metabolism and running some biohacking lite experiments.
  • Apr 25, 2019 A Recipe for Training Neural Networks
    A collection of practical advice for the process of achieving strong results with neural networks.
  • Jan 20, 2018 (started posting on Medium instead)
    Yes I'm still around but, I've started posting on Medium instead of here.
  • Sep 7, 2016 A Survival Guide to a PhD
    A collection of tips/tricks for navigating the PhD experience.
  • May 31, 2016 Deep Reinforcement Learning: Pong from Pixels
    I'll discuss the core ideas, pros and cons of policy gradients, a standard approach to the rapidly growing and exciting area of deep reinforcement learning. As a running example we'll learn to play ATARI 2600 Pong from raw pixels.
  • Nov 14, 2015 Short Story on AI: A Cognitive Discontinuity.
    The first part of a short story collection that has been on my mind for a long while. Exciting! :)
  • Oct 25, 2015 What a Deep Neural Network thinks about your #selfie
    We will look at Convolutional Neural Networks, with a fun example of training them to classify #selfies as good/bad based on a scraped dataset of 2 million selfies.
  • May 21, 2015 The Unreasonable Effectiveness of Recurrent Neural Networks
    We'll train and sample from character-level RNN language models that learn to write poetry, latex math and code. We'll also analyze the models and get hints of future research directions.
  • Mar 30, 2015 Breaking Linear Classifiers on ImageNet
    There have been a few recent papers that fool ConvNets by taking a correctly classified image and perturbing it in an imperceptible way to produce an image that is misclassified. In this post I show that ConvNets are an overkill: Simple linear classifiers are in fact susceptible to the same fooling strategy.
  • Sep 2, 2014 What I learned from competing against a ConvNet on ImageNet
    The latest state of the art Image Classification networks have only 6.7% Hit@5 error on ILSVRC 2014 classification task. How do humans compare?
  • Aug 3, 2014 Quantifying Productivity
    Describing a new pet project that tracks active windows and keystroke frequencies over the duration of a day (on Ubuntu/OSX) and creates pretty HTML visualizations of the data. This allows me to gain nice insights into my productivity. Code on Github.
  • Jul 3, 2014 Feature Learning Escapades
    Some reflections on the last two years of my research: The Quest for Unsupervised Feature Learning algorithms for visual data. Where it was, where it is, and where it's going. Maybe.
  • Jul 2, 2014 Visualizing Top Tweeps with t-SNE, in Javascript
    A writeup of a recent mini-project: I scraped tweets of the top 500 Twitter accounts and used t-SNE to visualize the accounts so that people who tweet similar things are nearby. My final Javascript implementation of t-SNE is released on Github as tsnejs.
  • Jul 1, 2014 Switching Blog from Wordpress to Jekyll
    I can't believe I lasted this long on Wordpress. I am switching permanently to Jekyll for hosting my blog, and so should you :) Details inside.
  • Apr 26, 2014 Interview with Data Science Weekly on Neural Nets and ConvNetJS
    I gave a (long) interview about my background and perspectives on neural nets.
  • Nov 27, 2013 Quantifying Hacker News with 50 days of data
    I scraped Hacker News Front Page and New Page every minute for 50 days and analyzed the results. How do stories rise and fall on Hacker News? What makes a successful post? Find out in this post :)
  • Nov 23, 2013 Chrome Extension Programming: Illustrating a Basic Survival Skill with a Twitter Case Study
    I illustrate a very valuable skill (Chrome Extension Programming) using a Twitter Case study. We will give Twitter a face lift, get it to refresh new tweets automatically, and highlight tweets from people who rarely tweet. All with a few lines of Javascript!
  • Oct 22, 2012 The state of Computer Vision and AI: we are really, really far away.
    A depressing look at the state of Computer Vision Research and AI in general. For those who like to think that AI is anywhere close.
  • Apr 27, 2011 Lessons learned from manually classifying CIFAR-10
    CIFAR-10 is a popular dataset small dataset for testing out Computer Vision Deep Learning learning methods. We're seeing a lot of improvements. But what is the human baseline?