Pacific-Design.com

    
Home Index

1. Machine Learning

2. 4 ID3 J48

Machine Learning / 4 ID3 J48 /

Matlab - Learning Algorithms ID3 & J4.8

%-------------------------------------------------------------------------%
%  Learning Algorithms ID3 & J4.8
%-------------------------------------------------------------------------%

fprintf('Magazine - Watch - Sex - Class') 

class1  = { 'Yes' 'Yes' 'Female' 'Yes' };               
class2  = { 'Yes' 'Yes' 'Female' 'Yes' };                       
class3  = { 'Yes' 'No'  'Male'   'Yes' };                       
class4  = { 'Yes' 'No'  'Female' 'Yes' };                       
class5  = { 'Yes' 'Yes' 'Male'   'Yes' };                       
class6  = { 'Yes' 'No'  'Male'   'Yes' };               
class7  = { 'Yes' 'No'  'Male'   'Yes' };                       
class8  = { 'No'  'No'  'Male'   'No'  };                       
class9  = { 'No'  'No'  'Female' 'No'  };                       
class10 = { 'No'  'Yes' 'Male'   'No'  };

matrix = [class1;class2;class3;class4;class5;class6;class7;class8;class9;class10]

%EntropyClass   ClassYes=7/10     ClassNo=3/10
EntropyClass = -0.7 * log2(0.7) - 0.3 * log2(0.3) 


fprintf('----------- Split using "Magazine" Attribute: ----------------\n')

%Magazine  (MagYes+ClassYes)/TotMagYes  (MagNo+ClassNo)/TotMagYes
Entropy1Mag = -(7/7) * log2(7/7) - (0/7) * log2(0/7)

%Magazine  (MagNo+ClassYes)/TotMagNo  (MagNo+ClassNo)/TotMagNo
Entropy2Mag = -(0/3) * log2(0/3) - (3/3) * log2(3/3)

% Information Gain              MagYes/TotalMag    MagNo/TotalMag
GainMagazine = EntropyClass  - (7/10)*Entropy1 - (3/10)*Entropy2

fprintf('----------- Split using "Watch" Attribute: -------------------\n')

%Watch  (WatchYes+ClassYes)/TotWatchYes  (WatchNo+ClassNo)/TotWatchYes
Entropy1Watch  = -(3/4) * log2(3/4) -(1/4) * log2(1/4)

%Watch  (WatchNo+ClassYes)/TotWatchNo  (WatchNo+ClassNo)/TotWatchNo
Entropy2Watch  = -(4/6) * log2(4/6) -(2/6) * log2(2/6)

% Information Gain        WatchYes/TotalWatch  WatchNo/TotalWatch
GainWatch = EntropyClass - (4/10)*Entropy1 - (6/10)*Entropy2

fprintf('----------- Split using "Sex" Attribute: -------------------\n')

%SexAttr  (SexFem+ClassYes)/TotalFem)  SexFem+ClassNo)/TotalFem)
Entropy1Sex  = -(3/4) * log2(3/4) -(1/4) * log2(1/4)

%SexAttr  (SexMale+ClassYes)/TotalMale   (SexMale+ClassNo)/TotalMale
Entropy2Sex  = -(4/6) * log2(4/6) -(2/6) * log2(2/6)

%Information Gain         Female/TotalSex    Male/TotalSex
GainSex = EntropyClass - (4/10)*Entropy1 - (6/10)*Entropy2

fprintf('-------- Result of Information Gain Lowest is Greatest -----\n\n');
fprintf('The highest information gain is magazine attribute.\n')
fprintf('Magazine attribute will be use to split for ID3/J4.8 methods.\n')

Results:

Magazine - Watch - Sex - Class
matrix = 

    'Yes'    'Yes'    'Female'    'Yes'
    'Yes'    'Yes'    'Female'    'Yes'
    'Yes'    'No'     'Male'      'Yes'
    'Yes'    'No'     'Female'    'Yes'
    'Yes'    'Yes'    'Male'      'Yes'
    'Yes'    'No'     'Male'      'Yes'
    'Yes'    'No'     'Male'      'Yes'
    'No'     'No'     'Male'      'No' 
    'No'     'No'     'Female'    'No' 
    'No'     'Yes'    'Male'      'No' 


EntropyClass =

    0.8813

----------- Split using "Magazine" Attribute: ----------------

Entropy1Mag =

   NaN


Entropy2Mag =

   NaN


GainMagazine =

    0.0379

----------- Split using "Watch" Attribute: -------------------

Entropy1Watch =

    0.8113


Entropy2Watch =

    0.9183


GainWatch =

    0.0058

----------- Split using "Sex" Attribute: -------------------

Entropy1Sex =

    0.8113


Entropy2Sex =

    0.9183


GainSex =

    0.0058

-------- Result of Information Gain Lowest is Greatest -----

The highest information gain is magazine attribute.
Magazine attribute will be use to split for ID3/J4.8 methods.