Dear friends,
Did you ever spend days obsessing over a technical problem? If so, I applaud you. Determined pursuit of solutions to hard problems is an important step toward building deep expertise.
I’ve been privileged to have worked with several of today’s AI leaders when they were still students. Every one of them spent days, weeks, and months relentlessly trying out different approaches to a range of problems, coming up with hypotheses and performing experiments to hone their intuition. This gave them a thorough understanding of machine learning.
It takes many judgement calls to build an effective AI system. How do you tune a particular hyperparameter? What are the tradeoffs between model size, real-time throughput, and accuracy for an application? What type of data pre-processing will yield the best results? When facing complex questions, engineers with deep expertise will come up with better answers.
Lately I’ve been thinking about how to train neural networks on small amounts of data. I try to find quiet time to brainstorm, and sometimes I end up with many pages of handwritten notes. After I’ve obsessed over a problem during the day, before I fall asleep I remind my brain that I want to make progress on it. Then, if I’m lucky, I awaken in the morning with new ideas.
The world is complex and becoming more so. We need people, in AI and other disciplines, who will take the time and effort to build deep expertise. When a worthy problem taps you on the shoulder, I encourage you to give it your attention. Give yourself the time you need to explore a solutions, and keep at it. It’s not a weird thing to do. Even if you don’t succeed — as a student, I spent countless hours trying, and failing, to prove P ≠ NP, and I don’t regret a minute of it — the journey will make you better.
Keep learning!
Andrew