• Sign up
  • Log in
ABM – Machine Learning Platform for everyoneABM – Machine Learning Platform for everyoneABM – Machine Learning Platform for everyoneABM – Machine Learning Platform for everyone
  • How it works
  • Pricing
  • Developers
  • Blog
NextPrevious

Tutorial: How to establish quality and correctness of classification models? Part 3 – Confusion Matrix

By Algolytics | Predictive models | Comments are Closed | 3 June, 2015 | 0

In the previous parts of the tutorial (part 1, part 2) we introduced quantitative indicators of classification model quality. In the next two parts we will take a closer look at a couple of graphical indicators. The first one is called the Confusion Matrix (the name “Contingency Table” is also used).

What is a Confusion Matrix?

Confusion Matrix is an N x N matrix, in which rows correspond to correct decision classes and the columns to decisions made by the classifier. The number ni,j at the intersection of i-th row and j-th column is equal to the number of cases from the i-th class which have been classified as belonging to the j-th class.

Examples:

confusion_matrix_1

Forms of Confusion Matrices

Various forms of Confusion Matrices let us more easily observe certain characteristics of the classification (i.e. the cost incurred by incorrect classifications).

  • Numerical form – contains counts of observations assigned to particular classes.

confusion_matrix_2

  • Percentage form – contains the percentages of observations assigned to particular classes calculated as the ratio of the count of observations assigned to the class to the total observation count.

confusion_matrix_3

  • Gains and losses form – contains information about gains and losses due to correct and incorrect classification decisions.

confusion_matrix_4

 

Confusion Matrix in the gains and losses form contains sums of costs due to classification decisions.

Example 1

confusion_matrix_5

 

Example 2

confusion_matrix_6

 

Cut off point and the Confusion Matrix

Cut off point is a certain threshold value which can be used to determine whether an observation belongs to a particular class.

if P(class(x)=1) >= alfa, then assign to class 1

where:

alfa – the cut off point

P (class(x)=1) – probability, that the given element belongs to the class denoted by 1

For example:

If the probability (calculated by our classification model) that a given loan applicant will not be good at repaying the loan is greater or equal to 60%, then assign this applicant to the class of bad debtors, otherwise assign him/her to the class of good debtors.

Different cut off points can be considered for the same problem (i.e. assessing creditworthiness), which will lead to different confusion matrices. By analyzing these matrices the optimal cut off point can be selected.

confusion_matrix_7

confusion_matrix_8

 

Confusion Matrix – summary

  • A simple and readable way of collecting classification results
  • Makes assessment of classification quality easier
  • Different forms of the Confusion Matrix can help in observing the required properties of the classifier
  • Can be used to determine gains and losses due to classification

In the next parts of our tutorial you will learn about other examples of graphical quality indicators: ROC curve and LIFT curve.

Want to read more news like this? Sign up for our Newsletter!

NAME

EMAIL

I agree to the processing of my personal data for the purpose of sending marketing information.

The administrator of the data given in the above form is Algolytics Technologies Sp. z o. o., ul. Przeskok 2, 00-032 Warszawa, NIP: 701-080-13-66, Regon: 369456263, District Court for the Capital City of Warsaw in Warsaw, XII Commercial Division of the National registered under KRS number 0000074723, Amount of the share capital: 321 300,00 PLN. Data is provided voluntarily and processed in order to respond to enquiries made using the form and to send marketing information. We would like to inform you about your right to be forgetten, your right to access the data and your right to correct it. Please note that your consent may be revoked at any time by sending an e-mail to gdpr@algolytics.pl from the address to which consent relates.

I accept Terms of service and Privacy policy

Read Terms of service

Read Privacy Policy

Share
Share
Tweet
0 Shares
Predictive models

Algolytics

More posts by Algolytics

Related Post

  • correlation_causation_example1

    Correlation does not imply causation

    By Algolytics | Comments are Closed

    A popular phrase tossed around when we talk about statistical data is “there is correlation between variables”. However, many people wrongly consider this to be the equivalent of “there is causation between variables”. It’s importantRead more

  • Understanding machine learning #3: Confusion matrix – not all errors are equal

    By Algolytics | Comments are Closed

    One of the most typical tasks in machine learning is classification tasks. It may seem that evaluating the effectiveness of such a model is easy. Let’s assume that we have a model which, based onRead more

  • Understanding machine learning #2: Do we need machine learning at all?

    By Algolytics | Comments are Closed

    In the previous post of our Understanding machine learning series, we presented how machines learn through multiple experiences. We also explained how, in some cases, human beings are much better at interpreting data than machines.Read more

  • Understanding Machine Learning #1 – How machines learn?

    By Algolytics | Comments are Closed

    “If (there) was one thing all people took for granted, (it) was conviction that if you feed honest figures into a computer, honest figures (will) come out. Never doubted it myself till I met aRead more

  • How to assess quality and correctness of classification models? Part 4 – ROC Curve

    By Algolytics | Comments are Closed

    In the previous parts of our tutorial we discussed: Basic notation used in assessing classification models Quantitative quality indicators Confusion Matrix In this fourth part of the tutorial we will discuss the ROC curve. WhatRead more

NextPrevious

100px white

Created with love by Algolytics

+48 691 303 305
abm_support (at) algolytics.com

Company

  • About us
  • Blog
  • Contact Us

Product

  • Documentation
  • Pricing
  • Terms of service
  • Privacy policy
  • API
Copyright © 2020 Algolytics Technologies | All Rights Reserved
  • How it works
  • Pricing
  • Developers
  • Blog
ABM – Machine Learning Platform for everyone
This site uses cookies: Find out more.