Classification of Machine Learning
What is learning?
Being able to solve new but relatable problems based on past experience.
What about machine learning?
Emulating the process of learning on computer would be machine learning. Since in computation the complexity or we can call it processesing efficiency of an algorithm is a major concern. So, the follow up question is are machine learning algorithms fast or slow?
The training of algorithm which involves preparation of machine learning model is slow and the execution of model is fast. Obviosly its not always true and as every CS question the answer to it is 'it depends'. It depends on what type of algorithm it is. For an algorthim declaring whether someone will like a movie or not depending upon their past preferances can be extremely fast when it comes to execution whereas a machine learning based algorithm for labelling a unlabelled data set might not be that fast.
Learning can be done in various ways for example while learning to ride a bicycle one kid can learn with the help of small side wheels and then remove those to ride without them whereas other can just try and fall until he learns to balance it. In the same way there are different ways learning can be accomplished in a computer program.
Inductive Learning
For the most part in this course we will be looking at inductive learning. In this method computer is feeded with the labelled data(solved examples). A learning algorithm is used go through the given data set. This learning algorithm with the help of these solved examples(labelled data) produces a machine learning model. This model in turn can take unlabelled data and should be able to produce labelled data with significant accuracy. Figure shown below shows a schematic of inductive learning.
Classification of machine learning based on what a model is trying to predict:
Regression
For a given dataset a regression model gives out a real number. For example a model which gives temperature as its result.
Binary Classification
For a given dataset a binary classification model gives out one of the two labels. For example a model can take a image as input and assign one of the two labels such as 'Cat' and 'Not cat'. In a binary model a negation of one label must imply that given dataset is from another label. True/false is another goof example but 'cat' and 'not dog' is not a good binary classification.
Multiclass classification
If there are more than two levels that the model can associate a given example with, this would be multiclass classification. Examples can be a model to predict weather or to find color.
Ranking
Given a preference rule, the algorithm should give a sorted order. Such examples include movie recommendation, google searches, names on facebook. Basically the model will take a unsorted dataset and sort it based on the rules of preference. Now buying recommendations on Amazon can also be made using statistics, like how many people bought one item after the other. The point here is, in case of using statistics someone will have to make a filter which analyzes all the data and gives out a preference, whereas in case of machine learing its the machine which does this job for us.
Decision Tree (A learning algorithm)
Disclaimer : Not correct, just visual representation.
Its basically a complicated if and else. So it asks certain yes or no questions based on the answers. For a case of movie recommendation Q1 can be is movie a thriller, depending on the past preferences lets say it gives a percentage value if answer is yes and another percentage value if answer was no. Then it askes whether the movie is directed by Nolan or not and again depending of the answer it gives another liking percentage. It repeats this process for indefinite amount of time and gives a final liking percentage whether a given person will like the movie or not.
A formal definition would be a decision tree is a tree whose internal nodes can be taken as tests (on input data patterns) and whose leaf nodes can be taken as categories (of these patterns). These tests are filtered down through the tree to get the right output to the input pattern.[1]
If there are too many questions the model will end up with a solved example and give the same result it was provided which is memorization, this is a case of zero generalization. If lets say based on answer to only one question it can give the final liking percentage it will be a case of too much generalization.
Inductive Bias
The bias in the answer based on algorithm used to make the model is called inductive bias. For example lets say we provide few people with pictures of birds and animals and then ask them to label them into two groups. Now some people can ask whether the oraganism can fly or not and then label based on that, someone can also ask the question whether the organism is mammal or not and can end up with different labelling. This kind of discrepancy in result is a case of inductive bias.
References
1.Overview of Use of Decision Tree algorithms in Machine Learning, doi:10.1109/icsgrc.2011.5991826
Link2.Lecture two of course CS460/CS660
3.Video Lecture
Lecture 2 Zoom recordingPasscode : 97A@kvQd