By Eugene Yan
Machine learning changes fast. Take natural language processing. Word2vec, introduced in 2013, quickly replaced one-hot encoding with word embeddings. Transformers revolutionized the field in 2017 by parallelizing the previously sequential training process. Subsequent transformer models grew from millions to trillions of parameters, bringing new challenges for training and deployment.
How can you keep your skills sharp amid the deluge of new developments? Here’s what has worked for me as I continue to learn about machine learning.
Try something different. Use each new project as an opportunity to extend your grasp. If you typically use pandas for data processing, try Spark or Dask. If you usually rely on decision trees for machine learning, experiment with neural networks. Your explorations may not pay off immediately, but they’ll pave the way to solving previously unapproachable problems, such as processing data that doesn’t fit into memory. Just be sure to timebox yourself so your projects don’t get derailed.
Take on personal projects. Stretch yourself periodically with personal projects. Say, try a new framework or build an app. Pick projects that align with your interests. That makes them more fun and thus more likely you’ll complete them. Try something scary. If you’re not worried about failing, the project may not be challenging enough to deliver much learning. Personally, I’m motivated to finish projects that give me opportunities to learn something new, help others, and have fun — all at the same time.
Schedule projects regularly. Set a sustainable pace. I tend to do one project a year and complete it in three to six months. This leaves room for vacations and hectic stretches at work. Track your progress via milestones and quantifiable output; for example, lines of code or mistakes made (which are tantamount to lessons learned).
Attend events. Meetups and conferences let you interact with people who push the boundaries of research and apply machine learning in industry. I usually learn about such events from peers or Twitter. You can also search meetup.com, and don’t miss DeepLearning.AI’s own Pie & AI. Specialized conferences on tools, such as Data+AI Summit, or domains, like RecSys, can be more directly useful. I try to attend two conferences a year and one meetup — often virtual these days — each month.
Read papers. Reviewing research is a great way to widen your perspective and stay up-to-date. When you start a new project, conduct a literature review so you don’t waste time reinventing the wheel. I find the classic three-pass approach helpful: First pass (5 to 10 minutes): Is the paper relevant to my current work? Second pass (1 hour): What is the main thrust? Third pass (4 to 5 hours): How can I implement this work? Andrew Ng presents his own advice in this video.
Share your experiences. Letting others know what you’ve learned helps cement the learning in your mind and helps them learn it, too. Summarize papers you’ve read and post your summaries. Offer your teammates a 10-minute demo of your latest exploration. Wrap up projects by publishing a blog post or presenting at a meetup. After a conference, consolidate the experience by writing about it. For example, here are some great recaps of RecSys 2021.
Recruit mentors. Where do you see yourself in a few years, and what do you need to learn to get there? I’ve found it helpful to ask people who are a few steps ahead of me. If you want to switch from, say, data engineering to machine learning engineering, someone who made the change recently can provide more relevant advice than a CTO or head of machine learning. Look for mentors within your company. Search LinkedIn to find friends of friends. Once you find them, touch base with them on a schedule; quarterly works well. For a taste of the kind of help mentors can provide, here are some questions and answers from machine learning practitioners.
Adopt a beginner’s mind. The Zen teacher Shunryu Suzuki said, “In the beginner's mind, there are many possibilities, but in the expert's there are few.” Regardless of our experience and expertise, we can maintain a beginner’s mind by staying curious and trying new things.
In the spirit of Suzuki’s advice, I challenge you to learn one new thing each week. Before long, problems that were previously out of reach will seem like child’s play.
Eugene Yan is an applied scientist at Amazon where he builds systems to help customers discover and read books. He writes at eugeneyan.com and ApplyingML.com.