Multi-Layer Perceptron: Overcoming the Local Minima Problem: Hierarchical Binary Classifiers

EasyChair Preprint 14997

Abstract

It  is  well  known  that  the  gradient  descent  rule  employed  in  training  the   Multi-Layer Perceptron (MLP) could  get  stuck  in  a  local  minima  of  the  error/loss  function ( based on mean squared  error ).  We  reason  that  by   realizing  MLP  using  a  cascade  of  binary  classifiers ( MLP  with  single  neuron  in the  output  layer ),  the  Hierarchical  classification  approach  overcomes  the  local  minima  problem  ( since  the  loss  function  of  each  binary  classifier  is  a  paraboloid ). Several  innovative  ideas  related  to  such  Artificial  Neural  Network  architecture  are  being  proposed.

Keyphrases: Cascade Architecture, Classification, Local minima Problem, Multi Layer Perceptron, binary classifiers

@booklet{EasyChair:14997,
  author    = {Rama Murthy Garimella},
  title     = {Multi-Layer Perceptron: Overcoming the Local Minima Problem: Hierarchical Binary Classifiers},
  howpublished = {EasyChair Preprint 14997},
  year      = {EasyChair, 2024}}