Good machine learning developers do this

Every year we communicate with students who work on short-term machine learning research projects. So far, we have talked with many students and found many problems in their projects, and gave them many suggestions based on the questions. We have found that many problems exist for everyone. In this article, we will focus on these issues and give solutions. This article is by no means comprehensive, but rather highlights the pitfalls we have seen over and over. For example, we won't talk about how to choose a good project. Some of our recommendations are generally applicable to machine learning, especially for deep learning or reinforcement learning research projects.

1. The main trap:

1.1 Suppose your code is error free

It is well known that machine learning, especially deep learning and reinforcement learning models, are difficult to debug. To help you understand the myriad of ways to make mistakes, check out Andrej Karpathy's Twitter theme. All of us, even more advanced researchers, will always make the same mistakes. The reason why these are so hard to detect is that even a defective model can still learn and produce meaningful output. Errors can make minor changes to your model, and most will only be displayed at runtime. With this in mind, the worst thing you can do is assume that your code doesn't contain any errors. The attitude of machine learning researchers to code is also the original intention of dividing them into productive and non-productive. If your default assumption is that your code may have problems, then you will search for the error more carefully, complete the code line by line, and double check the intermediate output. If possible, you will also visualize them. Would you consider whether the shape of the tensor is appropriate? Are they properly initialized? Is cloning still separated? Monitor the gradient during training and pay attention to NaN. Write unit tests by setting the seed of the random number generator and make your experiments reproducible and more. These measures can make your code more practical. For more tips on neural network debugging, see Goodfellow et al.

1.2 Only look at the final evaluation indicators

Although your project goal may be to improve some of the evaluation metrics, but more importantly, you should fully understand how the model works and why. Especially in the early stages of the project, the final evaluation indicator contains very little information, but it is useful for iterating and developing algorithms or models. In order to get more information, we must ask more in-depth questions and get feedback. For example, if you introduce a gating mechanism or a caution mechanism, does your model actually use it? Which model innovations do you propose actually contribute to overall performance improvement? How many training samples/periods does your model require to achieve reasonable performance, which is different from the baseline you use? Is there a system difference between test instances that are running well or very severe in your model? Can you predict important features from the hidden state of the model? Keep in mind that your research and project reports do not really inform the research community about some (marginal) improvements in the state of the art, but rather contribute to our understanding of the field.

1.3 Trying random changes has no clear expectations

Using the current deep learning library, you can easily make the model more complex by adding more components, layers, and optimization techniques. However, when you make changes to your code or model, you should at least have a general intuition about the change. Again, when you experiment, you should have a clear expectation of the results. What do you expect the results to look like, and what will they tell you? This is even more important when you find that your model is not doing as expected. With these measures, you are more likely to see the cause of the error, because expanding your model will not help you find the error, and may even make it harder to find the problem. Before making the model more complex, first understand the bottom line where it may go wrong. Also, remember that in your report, you must prove what you are doing. Because your report is the only entry point for others to help you solve the problem. If you can't make a research hypothesis and explain to yourself why you are doing this work, then no one else has a chance to know what you are doing.

1.4 Excessive complexity

We often see active students jumping out of the puzzle and trying out complex solutions. In fact, the real problems are generally difficult to analyze. Instead, ask yourself: What is the smallest thing that should work? Can the model learn to remember a small data set? What do you learn when using only a few parameters? Is the code valid in a single training instance rather than batch training? What is the simplest general form we expect to see?

1.5 Iteration is too slow

It may take a long time to conduct a training experiment. In particular, deep learning and reinforcement learning can be very time consuming when accumulating a statistically significant number of random seeds. Therefore, it is important not to fall into a slow iterative cycle too early in the short-term project. Try to debug your model with a simple environment and implement proof of concept for your ideas, so that the whole process can run on your personal computer. Sometimes, simple matrix games or grid world experiments can also provide useful idea validation. Similarly, you can use MDP's exact value function to test algorithmic ideas without having to use gradient estimation, actor comment training, and so on. Simplify the process of starting your experiment and checking the results as you move to larger experiments. Check these results before the experiment completes the process to see if the performance is stable. The infrastructure used by the investment training model can be time consuming at the outset, but it pays off at the end of the project.

2. Some suggestions:

2.1 Begin reading the background and preparing related work before the project begins.

We usually analyze the project before the official start date. One of the reasons is to understand the background and related work, then we have to implement and experiment, and finally we must carefully write a good report, and we must have a big picture before we do the project. Another reason is that we usually propose research projects that, if successful, may be published in machine learning forums or magazines. Although we all know that there will be many things happening throughout the project, most of the time we encourage you to start reading the relevant literature at least in advance. Ideally, when you start a full-time job, you should know what to do, how it relates to existing methods, and how to do it.

2.2 Correct use of software version and backup

You really should use strict version control for your project code and project reports, and make backups in a timely manner. Nothing is worse than losing all the hard work before the deadline. If you haven't already, open a GitHub account and as a student you can get a free private repository.

2.3 Use random repeat evaluation

In academia, you can't use multiple GPUs for training during a project. However, another embarrassing thing, especially in deep reinforcement learning, is not simply to draw premature conclusions from a single or several experiments. Ideally, you want to repeat the experiment multiple times, and as described above, you can understand the robustness of different starting conditions and hyperparameters.

2.4 Start writing documents as early as possible in the implementation project

If you are working on a master's program, your work will be evaluated based on your written report, not on the outstanding work you do. At this time you need to have enough time to write out what you did. My personal recommendation is to start writing as early as possible and not underestimate the value of communication research. Defining your goals, assumptions, and contributions, and letting readers follow your thinking process and read your articles carefully. You need to explain your design choices in the documentation and discuss your findings clearly. Ideally, you should write a report consistently throughout the project. In this way, you force yourself to consider the next step, and you are unlikely to forget any important information when the deadline arrives.

2.5 Actively ask for help when needed

Your superiors are busy people, but they exist to help you. Be sure to promptly contact the superior in case of problems, and do not wait until the next scheduled meeting. Proactively communicate and prepare the results you want to discuss in advance. Make the most of your supervisor! Finally, don't panic! They have all experienced this. Some problems may be an unsolvable problem in your place, but it may be a good thing in your superiors.