\documentstyle[eclepsf]{llncs}

\newtheorem{excersize}{Problem}

\title{When a family of  iris flower is normal then are others abnormal?}
\author{Akira Imada}
\institute{Brest State Technical University\\
           Moskowskaja 267, 224017 Brest, Republic of Belarus\\
           akira@bsty.by}

\begin{document}
\maketitle
\pagestyle{empty}
\thispagestyle{empty}

\begin{abstract}
This article is not a report of success but rather
a challenge to those who claim to have designed 
a network intrusion detection system successfully by means
of a machine learning technique using artificial dataset
to train and to test the system.
\end{abstract}

\section{Introduction}
\begin{quotation}
\begin{small}
\noindent
{\it Those highly qualified hackers who provide security services to companies during 
the daytime and then go home at night to conduct totally illegal hacking are the ones 
who are the most dangerous. -- by Enis Senerdem from Turkish Daily News on 29 March 2006}.
\end{small}
\end{quotation}


\noindent
Designing a  network intrusion detection system
is one of the hottest issues in our computer network society.
Designing such a system with so-called a soft-computing 
now seems to became a sort of fashion these days.
To try one of these approaches,
we usually need a dataset to train the system and to test the system afterwards.
Sometimes an artificial dataset is employed for the purpose.
In fact we could find a lot of such databases in public domain. 
Spearman's iris database is one of those examples.
%% and probably the most frequently used one.
We doubt, however, these dataset cannot necessarily reflect transactions in a real world. 
We usually don't know what does a coming intrusion looks like 
until it has completed the illegal intrusion when it is too late.
In this paper, we give it a consideration to this issue.\\

\section{Iris Database}
\noindent
Iris database\footnote{University of California Urvine Machine Learning Repository. 
ics.uci.edu: pub/machine-learning-databases.} is made up of 150 samples 
consists of three species of iris flower, that is, {\it setosa, versicolor} and {\it virginica}.
Each of these three families includes 50 samples. 
Each sample is a four-dimensional vector representing four attributes of the iris flower, 
that is, {\it sepal length, sepal width, petal length}, and {\it petal width}.\\

\noindent
As is often mentioned, this iris flower database
is perhaps one of the most often used datasets 
in pattern recognition/classification, machine learning, data mining, etc.
As a matter of fact, there have been fair amount of studies in which this iris flower 
database is employed as a dataset to train and to test the system.
Quite naturally, all of these papers report their success in designing a system.\\

\noindent
Let us take an example.
Castellano et al.~\cite{cast} assumed one family of this iris flower
to be normal whilst the other two to be abnormal.
The whole dataset was divided into 10 parts each of which has  
15 samples and are uniformly drawn from the three classes.
The system is trained by the remaining 135 samples.
The originally picked up 15 samples are used to test the results. 
After this 10-fold cross validation, 
the authors concluded that the system shows
the invasion detection rate is 96\% while the false alarm rate is 0.6\%.\\

\noindent
In reality, however, it is not so simple.
Just imagine that a hacker always tries to explore an unlearned region
to invade the network.\\

\section{Challenges}
First of all, let us formalized the standard way of using iris database 
for designing a network intrusion detection.

\bigskip
\begin{excersize}[One is normal while others abnormal]
Assuming one out of three families of iris flower to represent 
illegal transactions while the remaining two families represent
legal ones. Is it possible then to simulate a system for network intrusion detection
by using part of the dataset to train and remaining data to test the system?
\end{excersize}

\noindent
Let us now take a look at how those iris data are distributed in the whole search space.
We tried a Sammon mapping 
to see those data in 2-dimensional space.\\

\noindent
Sammon Mapping is a mapping a set of points a in high-dimensional space to 
the 2-dimensional space with the distance relation being preserved as much as possible,
or equivalently, the distances in the $n$-dimensional space are approximated 
by distances in the 2-dimensional distance with a minimal error.\\

\noindent
One of the results is shown in Figure 1.
Just a brief look at the figure reveals us that there remains an enormously
big region of unlearned.



\begin{figure}[h]
\begin{center}
\epsfile{file=final-2d.eps,width=10cm,height=6cm}
\caption{A 2-D visualization of iris flower data by Sammon Mapping}
\end{center}
\end{figure}

\subsection{Evaluation}
One way to evaluate the system we are designing is as follows.
The whole dataset was divided into 10 parts each of which has  
15 samples uniformly drawn from the three classes.
The system is trained by the remaining 135 samples.
The originally picked up 15 samples are used to test the results. 
This is called {\it 10-fold cross validation.}
Or more simply, we may evaluate the system with 2-fold cross validation.

\subsection{Mutation}
We now introduce a mutation which is applied to the data.
in order to avoid for the data to be deterministic. 
Real data are never expected to be deterministic.
Each of the iris database is given as the form of
\begin{displaymath}
(x_1, x_2, x_3, x_4)
\end{displaymath}
With a small probability called mutation-rate we modify record by
\begin{displaymath}
x_i^{\mbox {\tiny new}} = x_i^{\mbox {\tiny old}} + r \sigma_i
\end{displaymath}
where $\sigma_i$ is standard deviation of 150 values of $x_i$,
and $r$ is a small random number anewed each time when mutation occurs.
Then our problem turns into as the following.

\bigskip
\begin{excersize}[Mutants of normal and attack]
All points of one family of iris flower are given mutation and assumed to be normal
while
the remaining two families are also mutated and are assumed to be abnormal.
Then is it possible to use these all mutated samples to train and to test the system?
\end{excersize}

\noindent
Still it would be not so difficult.
Furthermore even if for a large mutation rate and/or large range of value $r$
we still have large region of unleared samples. 
The  more realistic and challenging one would be

\bigskip
\begin{excersize}[random discretely located normal and attack samples]
A set of normal and a set of abnormal samples are specified at random one by one.
Then can the system trained by those samples recognize a mutant of abnormal sample?
\end{excersize}

\noindent
In other words, ``Is the system immune to mutants of previous invasion?''\\

\noindent
Let us now be more specific. 
We assume the whole universe is the 4-dimensional Euclidean space all of whose coordinate 
lie between 0 and 1. Then we create 
50 normal samples and 100 abnormal samples
all at random. We train the system using those 150 samples.
Then the question is, ''A test
with 100 samples, 50 of which are mutants from normal samples 
and yet another set of 50 mutants from abnormal samples, will satisfy us?''\\

\noindent
This is reminiscent of the experiment by 
Ayara et al.~\cite{ayar} who created 8-bit binary random strings,
as a set of training samples of normal patterns assuming other
comparable amount of 8-bit binary patterns as abnormal with their 
pattern-recognition results
being very successful. Still this is not difficult, however, 
as the data used were fixed ones.
How about, then, our case of mutants of abnormal sample?
Does the system which is trained by an random abnormal sample
still immune to those mutants? It would be really hard to believe.


\subsection{Don't We Expect the Result a priori?}
We have to be careful, because we sometimes tend to
{\it unconsciously} pick up only a set of data that will be 
suitable to draw our {\it a priori} expected conclusion,
if not {\it intentionally} at all.\\

\noindent
In the way that just a  {\it powder-from-sugar} sometimes 
has a same effect as, or more efficient than, a medicine under developing
enough to cure a disease for a group of innocent volunteers.
Let's conclude the discussion with the following final question.\\

\bigskip
\begin{excersize}[A Placebo Experiment]
(1) Create a simple device which randomly returns 
either one of Normal or Attack.
(2) Prepare a test dataset including enough amount of 
records uniformly from Normal and Attack.
(3) Compare the performances of the detector you designed
with the random-reply-machine created in step 1,
feeding the same dataset prepared in step 2.
\end{excersize}


\subsection{Can a Sommelier be Trained without Bootlegs?} 
Though we have not remarked so far, there remains further difficult issue, that is,
``How the system can learn only from normal data to detect abnormal?'' 
We usually have enormous amount of normal data but we have no information
about coming attacks untill it's too late.\\ 

\noindent
Gomez et al.~\cite{gome} claimed, {\it ``A new technique for generating
a set of fuzzy rules that can characterize the non-self (abnormal) space 
using only self (normal) samples.''}
It would be terrific if the report was really successful, 
but we are fishy more or less.\\

\noindent
This issue is something like we require a wine-taster 
to recognize bootleg champagne by only providing him/her 
plenty of real champagne to learn.\footnote{Or, in an opposite way.
I usually enjoy Georgian sparkling wine like once a week, but still a real champagne
would be able to pretend to be a Georgian one to me.''}\\

\noindent
Though this {\it training-only-with-normal}
is our ultimate goal, but not so simple to be realized.
To study how this is difficult, why not try the following?\\

\bigskip
\begin{excersize}[Dummy Attacks]
(1) Assume one family of iris as Normal while the other two Abnormal.
(2) Furthermore, randomly create an attack dataset -- dummy attacks.
(3) Train your intrusion detection system only with Normal.
(4) Then, try two tests, one with only Abnormal, and 
the other with only Dummy, avoiding any a~priori prediction.
\end{excersize}


\section{Experiment}
Each of the above 5 different challanges is now being tried by 
a {\it decision tree algorthm} --- C4.5 here.
However, our goal will not be to show the success results but rather be opposit.\\

\noindent
Also we will try to specifies the samples so that 
successful detection rate become as low as possible,
and false alarm rate becomes as high as possible.
Not performed yet either,  but we also plan to create abnormal samples
by using a co-evolution of pledetor-and-prey type, for the purpose.\\

\noindent
Results of the both experiments are not shown in this submission paper but sure to be appeared 
in the final version if this submission is to be accepted.\\

\noindent
Also by using this iris flower database, we will explore
 an issue of learning where we doubt that training using 
previous abnormal data is really practical. 
The question is like, ``If a wine taster who trained only with
real champagne can recognize bootleg or other sparkling wine, or not.''

\section{Concluding Remarks}
We feel sorry that we have described the above not so optimistically.
However, we should be careful to draw a conclusion from our experiment or
simulation.
We sometimes tend to overestimate our results so that we like it.
This article is not to deny the possibility, but instead we hope to be a challenge
for a new innovative approach to be emerged.


\begin{thebibliography}{99}
\bibitem{cast}
G. Castellano and A. M. Fanelli(2000)
"Fuzzy Inference and Rule Extraction using a Neural Network." 
Neural Network World Journal Vol. 3, pp. 361--371.


\bibitem{ayar} M.~Ayara, J.~Timmis, R.~D.~Lemos, L.~N.~D.~Castro, and R.~Duncan (2002) 
"Negative Selection: How to Generate Detectors." 
Proceedings of 1st International Conference on Artificial Immune Systems
pp.~89--98.

\bibitem{gome}
J.~Gomez, F.~Gonzalez, and  D.~Dasgupta (2003)
{\em ``An Immuno-Fuzzy Approach to Anomaly Detection.''}
Proceedings of IEEE International Conference on Fuzzy Systems, 
Vol.~2, pp.~1219--1224.

\end{thebibliography}
\end{document}

