Feature selection

Sometimes called feature importance, this is the process of ranking the input variables according to their contribution to the target/output variable. Also, this process can be considered a ranking process of the input variables according to their value in the predictive ability of the model.

Some learning methods do this kind of feature ranking or importance as part of their internal procedures (such as decision trees). Mostly, these kind of methods uses entropy to filter out the less valuable variables. In some cases, deep learning practitioners use such learning methods to select the most important features and then feed them into a better learning algorithm.