%\documentstyle[12pt,eclepsf]{article}
\documentstyle[]{article}
\textheight 230mm
\topmargin 5mm
\textwidth 160mm
\oddsidemargin -2mm

\pagestyle{myheadings}
\markright{(Report on Doctrate Dissertation of Mariusz Rybnik)}

\begin{document}
\thispagestyle{empty}
\title{Report on \\the Doctrate Dissertation of Mariusz Rybnik}
\author{Prof. Akira Imada, \\
%%\hspace*{1in}\\
%%\hspace*{1in}\\
Brest State Technical University\\
           Moskowskaja 267, 224017 Brest, Republic of Belarus\\
           akira@bsty.by}
\maketitle

\begin{abstract}
\noindent
This is comments on 
Mariusz Rybnik's Doctrate Dissertation titled ``{\it
The Contribution to Modelling, Designing and Implementing Multiple Neural Networks:
Auto-Adaptation of Data Processing System by Treelike-Divide-to-Simplify Approach
using Complexity Estimation Methods}'',
submitted to University Paris XII, Creteil, France. Notice that this report is 
based on one version earier than the almost final one, since I've just received
the newest version.
\end{abstract}


\section{To Start with}
Though this is a little unusual style, please allow me to start with the last chapter
of the dissertation, since this might be the most important section
to show what has the author done,
and what I want to comment concentrates on this chapter. 

\section{Applications of T-DTS System}
This chapter is to verify the proposed T-DTS System by applying it to examples.
\subsection{Model identification -- linear academic problem.}
\subsubsection*{Summary of the section}
The description of the first example begins with the section 5.1.1.
The problem might be sumarized as follows.\\

\noindent
{\bf Method:}
The data created from equation (5.1.2) are firstly given to the left-most 
Decomposition Unit (DU). 
DU is for the purpose of dividing the database into several sub-databases.
The unit is constructed using Kohonen's Self-Organizing Maps.
The output of the 1st DU are devided into
two sub-data, one of which is further given to the 2nd DU while
the other is the first teminal sub-data. In this way, the tree sturcture of
three DU's is constructed, and thus the original data is decomposed to four sub-data. 
Each of these four sub-data, is given, one by one, to the corresponding
processing unit (PU) which terminate the tree as the leaf. 
That is, the tree structure is made up of 4 DU's and 4 PU's. \\
Processing Unit models used in this work are: LVQ, MLP, LN,
Perceptrons, GRNN, and Probabilistic Networks.\\

\noindent
DU, in this 1st example is a "Competitive Network of 2 neurons, 
and PU is Multi Layer Perceptron (MLP) with one hidden layer with 4 neurons. \\

\noindent
Then the 4 models are created and each of them are trained 
using "Levenberg-Marquadt (LM) learning algorithm" with the sub-data
given as a training set. \\

\noindent
{\bf Results:}
Then the author shows the results by 
(1) Aggregation of patterns created by decomposition;
(2) evolution of learning error for each of the 4 PU's
    after 8 epochs;
(3) comparison between learning signals and generalization signals.\\

\noindent
{\bf Conclusion:}
Author concludes that the third results above 
{\it assures us that a faithful model was built,
proving the efficiancy of T-DTS in system identification task.}\\

\subsubsection*{Comments}
\noindent
We now understand how the proposed system works.
Let me begin with three tiny unclear points.\\

\noindent
(i) What is competitive network of 2 neurons? Is it possible
for it to decompose the imput data, if such a neural network exist?\\

\noindent
(ii) What is Levenberg-Marquadt learning? I tried to find it but
all I found is just a description of "Multi-Layer Perceptron with 
as one of six possible choises in Appendix A.2 --- No further
detailed explanation in the dissertaion.\\

\noindent
(iii) As summarized above, the author describes ``{\it 
That ends decomposition tree building phase. 
Then the models 1, 2, 3 and 4 are created and trained using the learning sub-databases 
that have arrived to their
location in the tree. That ends T-DTS learning phase.}''
However, how on earth, the 4 models are created? --- no such description, either.
In addition, the author describes all or those four PU are MLP by LM learning,
instead of showing why so, and not the other ones? 
The author listed in the previous chapter, 
LVQ, MLP, LN, Perceptrons, GRNN or Probabilistic Networks as candidates of the model.\\

\noindent
(iV) The selection of which is by "assignment rules". The author introduces
here a couple of different rules, but which one is used is remain unclear.\\

\noindent
Then the more inportant two quesions are:
First, in my humble opinion, dissertaion should be self-contained.
In this 1st example, we can create data for learning from (5.1.2), 
and we can prepare data to validate the learning from (5.1.3). However
We should learn LM learning by ourselves. Even reference is not given
for LM learning. Second, what we want to know the most is 
what happens if the original data is given to one LM Model without being decomposed. 
The proposed system out performs this case? --- No such comparison, here.\\

\noindent
Second, what we want to know the most in this section is 
``What if the original data is givne to the one, instead of four,
Neural Network with LM learning without the original data being decomposed?''
--- No such description, either.

Now back to the earlier chapters.

\section{Introduction (Chapter 1)}
\noindent
Though the section Introduction has been improved a lot,
still the draft of the newest version resists to be
easily understood.

\section{State of the Art (Chapter 2)}
As the Chapter 1. Artificial Neural Network (ANN) is a general
explanation of what is ANN, and most part of the description
has no direct relation of the theme of this dissertation 

\section{Devide and Conquere Approach (Chapter 3)}
\subsubsection*{Summary \& Comments of the Section}
\noindent
This section starts with:
\begin{quotation}
\noindent
{\it
     Apart from explicit solution of problem by specialized 
     "one-piece" algorithm, there exist a number of solutions, 
     which have modular structure. In modular structure modules
     could have some defined and regularized structure or be
     more or less randomly connected, ending up at completely
     independent and individual modules.}
\end{quotation}

\noindent
Let me try to paraphrase. Or, I might call it a  possible 
interpretation:

\begin{quotation}
\noindent
     {\it Besides an explicit solution usually obtained when we solve 
     a problem by a so-called one-piece-algorithm, sometimes 
     there exists a number of implicit solutions which have 
     a modular structure in which each of the modules can be some
     of those already defiened and usually completely independent 
     with each other, is connected to some of the other modules 
     at random, more or less, and constructs a whole regularized 
     structure.}
\end{quotation}

\noindent
Though still not a beautiful sentence though. 
Anyway an interpretation of the original English version is not easy
task.
However, I assume this english version of the dissertation is
a complementary one which compliments the original French version.
So let us now forget the issue of English representation, for 
the time being. We are running out of time, are we not? \\
Well, the author ascerts 

\begin{quotation}
\noindent
{\it
Devide and Conquer approach is, 
essencially made up of the following three procedures.
1) Divide the target problem into some sub-problems of similar structure;
2) Solve each of the sub-problems;
3) Combine the results to create the solution to the original problem."}
\end{quotation}

\noindent
Though author descirbe the a couple of previous devide and conquer
approach in the research field of ANN. Those are 

\begin{quotation}
\noindent
{\it Mixture of Expert (no reference); Multi-modelling; 
Hybridization as a general approach; Intelligent Hybrids Systems;
Neural Network Ensemble Concept; dels of experts Mixture; 
Dynamic Cell Structure Architecture; and Active learning.}
\end{quotation}

\noindent
as the one particularly based on ANN.
Although the purpose is described like it is to reduce 
the complexity and run-time of the algorithm, description of 
each method is not given and remain totally unclear. 
As it is difficult to guess what does each of these methods 
look like from the name of the method, summary, and hopefully
example of applying it to a problemm, from one method to the
next is more inportant than a very general detailed expalanation
of ANN in the chapter 1.

%\noindent
%Then author gives a formula to estimate time-complexity

\noindent
This section is followed by two alternatives of the divide and conquer
approach, i.e., Committee Machines and Multi Agent System (MAS).
What I cannnot understand is that author says those two alternatives
are less important than the Devide \& Conquer and despite of that
each of these two less important are given much more detailed 
explanation, including pseud-code for the algorithm.
Then the topic turns to "Clustering" which has not even been explained 
so far in this chapter. 

\begin{quotation}
\noindent
{\it
     Decomposition modules could be based on any efficient clustering 
     algorithm. We have chosen to use for decomposition ANN structures 
     similar in activity to well-known k-means clustering algorithm,\\

\noindent
     Clustering is described in details in the section 2.4 of this work
     in connection with modular algorithms.}
\end{quotation}

\noindent
And in fact, there is a description of "Clustering" in the section 2.4
but the method --- How an ANN could be used to "decompose" the problem.


\section{Complexity Estimation (Chapter 3)}
\subsubsection*{Summary of the chapter.}
\noindent
%{\bf Summary of the chapter.}
Chapter 3 is for "Complexity Estimation" As author writes at the 
beginning of the chapter

\begin{quotation}
\noindent
     {\it Complexity estimation is a supportive tool for classification},
\end{quotation}

\noindent
the task of decomposision of a problem starts with estimating
the complesity of the problem.\\

\noindent
The subsection 3.5 ``Presentation of Selected Methods'' introduces
a variety of mesures.
In 3.5.1 seven different measures of ``Indirect Bayes Error Estimation'', are givenm, i.e.,
1) Normalized Mean Distance; 2) Chernoff bound;
3) Bhattacharyya bound; 4) Divergence; 5)Mahalanobis Distance, Jeffries-Matusita Distance
and 7) Shannon's Entropy measures.\\

\noindent
3.5.2 gives four mesures concerning `` Non-Parametric Bayes Error
Estimation and Bounds'', i.e.,
1) Error of the Classifier Itself; 2) k-Nearest Neighbors; 3) Parzen Estimation
4) Boundary Method. \\

\noindent
In 3.5.3 as ``Measures related to space partitioning''
1) Class Discriminability Measures; 2) Purity measure;
3) Neighborhood Separability; 4) Collective entropy are given.\\

\noindent
Then in 3.5.4, as ``Other Measures''
1) Correlation-based approach; 2) Fisher discriminant ratio
3) Interclass distance measures (scatter matrices);
4) Volume of the overlap region;
5) Feature efficiency;
6) Minimum Spanning Tree (MST);
7) Inter-intra cluster distance;
8) Space covered by epsilon neighborhoods.

\subsubsection*{Comments.}
\noindent
%{\bf Comments.}
It's good to read the result of almost exhaustive survey like above, though
when ''Complexity'' is mentioned, this will remind me of Kolmogolov's, though.
The bellow are what I felt when I read this sub-section.\\

\noindent
First, is Shannon's entropy a measure of complexity? Yes, it could be. 
But at least a little explanation is needed, since it's not a direct mesure
of complexity.\\

\noindent
Second, in the last subsection gives us
the formula of "overlap-sum". But again it's not the explist
measure of complexity.\\

\noindent
After a series of presenting mesures, in 3.5.5 ``Ensemble of estimators'', the author writes

\begin{quotation}
\noindent
      {\it The computation of several methods at once is potentially 
     more difficult than one, but using several simple methods 
     could be faster than one complex method.}
\end{quotation}

\noindent
%%According to this each of them above, a method of complex estimation and 
the author seems to propose 

\begin{quotation}
\noindent
"{\it pick up som simple methods and combine them somehow.}" 
\end{quotation}

\noindent
However, in my humble opinion none of the 7 methods is not
a method for estimating complexity.

\subsection {Complexity Estimation Conclusion (3.6)}
\subsubsection*{Summary \& Comments of the section}
This section start with

\begin{quotation}
\noindent
     {\it Classification complexity estimation methods present great variability.}
\end{quotation}

\noindent
Hence, we now know what has so far called Complexity Estimation is 
classification complexity.  No wonder 
Kolmogorov --- probably one of the most popular
complexity mesures --- was not referred to.\\

\noindent
Then this section was concluded as follows.

\begin{quotation}
\noindent
{\it Not every method 
is suitable to estimate a complexity of multi-class classification problem - some are
designed only to two-class problems, and as such they need special procedures to
accommodate them to multi-class problem (like counting the average of complexi-ties
of all two-class combinations) what usually result in combinatorial increase in
computational complexity.}
\end{quotation}

\noindent
That's it. My guess seems to be correct.\\

\noindent
This is the end of Part I and we expected 
how the author would estimate classification complexity in the Part II.
It is Chapter 5 that follows the Part I and we expected hereafter
a concrete implementation of T-DTS. 
The result of this expectation was already described at the beginning of this article.

\section{To Conclude}
I have reviewed the Mariusz
Mariusz Rybnik's Doctrate Dissertation titled ``{\it
The Contribution to Modelling, Designing and Implementing Multiple Neural Networks:
Auto-Adaptation of Data Processing System by Treelike-Divide-to-Simplify Approach
using Complexity Estimation Methods.}'' 
Although those comments described are mostly a little negative ones.
The topic is exciting enough to stimulate and attract reader's interests 
including me.
While reading this dissertation,  an idea of exploring this topic ---
complexity reduction of a huge database dances in front of me.
It's been a always fun to read, interpreting the lines in my way,
and I thank the author and his supervisor for that.




\end{document}






