Why machine learning is difficult to apply

Applying machine learning is challenging.

In the field of machine learning, you have to make a lot of decisions on the question without the right answer! E.g:

• What framework?

• What data is used as input and what data is to be output?

• What algorithm is used?

• What algorithm configuration?

These problems are a serious challenge for beginners.

After reading this article, you will understand:

• How to form a clear learning problem.

• When designing a learning system for your problem, there are four decision points to consider.

• You can use three decisions to clearly address the challenges of designing a learning system in practice.

Overview

This article is divided into the following six parts:

1. Well-posed learning problems

2. Select training data

3. Select the objective function

4. Select the expression of the objective function

5. Choose the learning algorithm

6. How to design a learning system

Well-posed learning problem

We can define a general learning task in the field of applied machine learning as a program that can learn from some tasks based on specific performance measures.

Tom Mitchell made a clear statement in his 1997 "Machine Learning" book:

A computer program learns experience E from a certain type of task T and performance measurement result P. If its task in T is P, it is improved by experience E.

We use this as a general definition of the types of learning tasks we might be interested in, such as predictive modeling and other applied machine learning. Tom lists a few examples to illustrate this, as follows:

• Learn to recognize spoken language

• Learn to drive unmanned

• Learning celestial structure classification

• Learn world-class backgammon

We use the above definition to define our own predictive modeling problem. Once defined, the task becomes a design learning system to deal with.

Design a learning system, such as a machine learning application that involves four design choices:

1. Select training data

2. Select the objective function

3. Choose the form of expression

4. Choose the learning algorithm

For a given problem and providing unlimited resources, there may be the best set of choices, but we don't have unlimited time to calculate resources, as well as knowledge of the domain or the learning system.

Therefore, although we can prepare a well-defined description of the learning problem, it is still difficult to design this most likely learning system. It is best for us to use our knowledge, skills, and available resources to do our work through design choices.

Let's take a closer look at each design choice.

Select training data

You must choose the data that the learning system will use as a learning experience.

This is the data observed in the past

The type of training experience available has a major impact on the success or failure of researchers.

For learning problems, you must always collect the data you need.

This means:

• Clear files

• Query data

• Executable file

• Organize different resources

• Merged entities

You need to get all the data in one go and become a standardized form, so an observation represents an entity whose result is available.

Select the objective function

Next, you have to choose a framework for learning problems.

Machine learning is actually a question of learning the mapping function (f) from input (X) to output (y)

Y=f(x)

This function can be used to predict new data for the most likely output in the future.

The goal of the learning system is to prepare a function that provides the resources available to map the input to the output. This is a problem called function approximation. This result will be an approximation, meaning there is an error. We will try to reduce this error, but some errors will always exist in the data and interfere.

This step is about precisely selecting what data to input to this function, such as: input features or input variables and what to predict, such as: output variables.

I often refer to this as the framework for learning problems. Selecting input and output is essentially the type of objective function chosen, that is, we will seek similar functions.

Select the expression of the objective function

Next, you have to choose the expression you want to use to map the function.

Consider this as the type of final model you wish to use to make predictions. You must choose the form of this model and choose whether you like the data structure.

Now that we have detailed this ideal objective function V, we must choose an expression that the learning program will use to describe the function V that will be learned.

E.g:

• Perhaps your project needs a decision tree that is easy to understand and explain to stakeholders.

• Perhaps your stakeholders tend to have a linear model that statisticians can easily explain.

• Perhaps your stakeholders don't care about anything other than model performance, so all model expressions can be won.

The choice of representation will limit the type of learning algorithm that you can use to learn the mapping function.

Choose a learning algorithm

Finally, you must choose a learning algorithm that performs input and output data and learns the expressions you prefer.

If there are no restrictions on the choice of expressions, then this is often the case, and then you may evaluate a range of different algorithms and expressions.

If there are some strict restrictions on the choice of expressions, such as a weighted and linear model or a decision tree, then the choice of algorithm will be limited to those that can operate on a particular form of expression.

The choice of algorithm can take advantage of its own limitations, such as the preparation of specific data like data standardization.

How to design a learning system

Developing a learning system is challenging.

No one can tell you the best answer for each decision in this way; the best answer to your assigned learning question is unknown.

Mitchell described the clarification of this when designing a choice for a chess learning system.

Depiction-of-Choices-in-Designing-a-Chec

A description of the choices when designing a chess learning system.

From "Machine Learning", 1997.

Mitchell said:

In many ways, these design choices have constrained the learning task. We have defined the types of knowledge that can be used to obtain a linear evaluation function. In addition to this, we have also defined that the evaluation function relies on these six specific board features that are only provided. If this correct objective function V can be truly represented by a linear combination of these special features, then our program will have a good chance to learn it. If the opposite is true, then we hope that it will be better to learn an ideal approximation, because a program will never learn anything that it can't at least represent.

Often, you can't figure out the answers to these choices by analyzing them, such as what data to use, what algorithm to use, and what algorithm to use.

Here are the three strategies you can use in practice:

1. Copy: Look at the literature or learn from the experts the same or similar questions as yours, and copy the design of the learning system. Most likely you are not the first person to work on a given type of problem. In the worst case, the copied design gives you a starting point for your design.

2. Find: List the available options at each decision point and evaluate each one to see which one is most effective for your specific data. This is probably the most robust and practical result in applying machine learning.

3. Design: Through the above copy and find methods, after completing many projects, you will form an intuition for how to design and learn the system.

Developing a learning system is not a science but a project.

Developing new machine learning algorithms and describing how they work and why work is a science is often not necessary when developing a learning system.

Developing a learning system is very similar to developing software. You must combine the copies of the results you have designed in your past work, show the prototypes that are useful, and the design experience when developing a new system for the best results.