RSS Parrot

Deep Neural Nets: 33 years ago and 33 years from now

http://karpathy.github.io/2022/03/14/lecun1989/

Published: March 14, 2022 07:00

The Yann LeCun et al. (1989) paper Backpropagation Applied to Handwritten Zip Code Recognition is I believe of some historical significance because it is, to my knowledge, the earliest real-world application of a neural net trained end-to-end with…

A from-scratch tour of Bitcoin in Python

http://karpathy.github.io/2021/06/21/blockchain/

Published: June 21, 2021 10:00

I find blockchain fascinating because it extends open source software development to open source + state. This seems to be a genuine/exciting innovation in computing paradigms; We don’t just get to share code, we get to share a running computer, and anyone…

Short Story on AI: Forward Pass

http://karpathy.github.io/2021/03/27/forward-pass/

Published: March 27, 2021 10:00

The inspiration for this short story came to me while reading Kevin Lacker’s Giving GPT-3 a Turing Test. It is probably worth it (though not required) to skim this post to get a bit of a background on some of this story. It was probably around the…

Biohacking Lite

http://karpathy.github.io/2020/06/11/biohacking-lite/

Published: June 11, 2020 10:00

Throughout my life I never paid too much attention to health, exercise, diet or nutrition. I knew that you’re supposed to get some exercise and eat vegetables or something, but it stopped at that (“mom said”-) level of abstraction. I also knew that I can…

A Recipe for Training Neural Networks

http://karpathy.github.io/2019/04/25/recipe/

Published: April 25, 2019 09:00

Some few weeks ago I posted a tweet on “the most common neural net mistakes”, listing a few common gotchas related to training neural nets. The tweet got quite a bit more engagement than I anticipated (including a webinar :)). Clearly, a lot of people have…

(started posting on Medium instead)

http://karpathy.github.io/2018/01/20/medium/

Published: January 20, 2018 11:00

The current state of this blog (with the last post 2 years ago) makes it look like I’ve disappeared. I’ve certainly become less active on blogs since I’ve joined Tesla, but whenever I do get a chance to post something I have recently been defaulting to…

A Survival Guide to a PhD

http://karpathy.github.io/2016/09/07/phd/

Published: September 7, 2016 11:00

This guide is patterned after my “Doing well in your courses”, a post I wrote a long time ago on some of the tips/tricks I’ve developed during my undergrad. I’ve received nice comments about that guide, so in the same spirit, now that my PhD has come to an…

Deep Reinforcement Learning: Pong from Pixels

http://karpathy.github.io/2016/05/31/rl/

Published: May 31, 2016 11:00

This is a long overdue blog post on Reinforcement Learning (RL). RL is hot! You may have noticed that computers can now automatically learn to play ATARI games (from raw game pixels!), they are beating world champions at Go, simulated quadrupeds are…

Short Story on AI: A Cognitive Discontinuity.

http://karpathy.github.io/2015/11/14/ai/

Published: November 14, 2015 11:00

The idea of writing a collection of short stories has been on my mind for a while. This post is my first ever half-serious attempt at a story, and what better way to kick things off than with a story on AI and what that might look like if you extrapolate…

What a Deep Neural Network thinks about your #selfie

http://karpathy.github.io/2015/10/25/selfie/

Published: October 25, 2015 11:00

Convolutional Neural Networks are great: they recognize things, places and people in your personal photos, signs, people and lights in self-driving cars, crops, forests and traffic in aerial imagery, various anomalies in medical images and all kinds of…