\documentstyle[12pt,eclepsf]{article}

\textheight 260mm
\topmargin -10mm
\textwidth 180mm
\oddsidemargin -7mm

\pagestyle{myheadings}
\markright{(Software Engineering)}

\begin{document}
\begin{LARGE}

\begin{center}
\begin{Huge}
\noindent
{\bf Software Engineering}\\
\end{Huge}

\vspace{1in}
Dokuz Eyl\"{u}l University \\
Computer Engineering Department \\
19 February 2001 -- 8 June 2001\\
\vspace{1in}
Akira Imada\\
imada@cs.deu.edu.tr

\end{center}

\clearpage
%---------------------------------------------------------------------
\noindent
$\Box$ {\bf General Remarks:}\\

\begin{itemize}
\item[1.] {\bf Send your e-mail address to imada@cs.deu.edu.tr}
   \begin{itemize}
   \item[] with the subject being only this course name. 
           and with the text being three lines giving
           (1) your e-mail address (your most convenient one
               but preferably the one at this department), 
           (2) your name, and (3) your student-ID.
   \item[]as follows:
       \begin{itemize}
       {\it
       \item[] To: imada@cs.deu.edu.tr
       \item[] Subject: CSE 444
       \item[] ------------------
       \item[] ayhan@hamsi.deu.edu.tr
       \item[] Ayhan Akkun Ozturk
       \item[] 1999510026
       }
      \end{itemize}
   \end{itemize}
\item[$\star$] This should be done immediately after the first course.
\item[$\star$] This is to communicate you all about the course.
\item[$\star$] Important Note: {\it Do not use any Turkish characters!}\\
\item [ 2.] {\bf You should basically attend the class every time.}
   \begin{itemize}
   \item[] Don't be a {\it never-shown-up-student}
           when the final exam.
   \end{itemize}
\item[$\star$] Although I won't check your presence/absence each time,
               this does not mean you may skip the class.  
\item[$\star$] Instead of home works, I give you some tasks during a course 
               and this is to be submitted until the end of the course, 
               not after.
\item[$\star$] Since the course materials given in my home-page is 
               not a perfect version like a textbook.
               but just to compensate my talk in classes, 
               you should take your own note in every classes.
\item[$\star$] Though it might be a good strategy to ask me questions 
               before the final exam about the course 
               without attending most of my classes, 
               it will not be accepted.\\
\item [ 3.] {\bf Final examination.}
\item[$\star$] Examination is given only once at the end of the semester 
               where
   \begin{itemize}
   \item You may open your materials during the exam.
   \item Questions are numerical and all fundamental ones 
         that are well explained in the course.
   \end{itemize}
\item[$\star$] Under these conditions, 
      \begin{itemize}
      \item do not expect more careful treatment in evaluating your exam. 
            That is, even if
          \begin{itemize}
          \item the equations you employed are appropriate ones, or
          \item your mistakes are just careless
          \end{itemize}
       those will not be evaluated.
      \item If you fail to obtain the least point to pass the course, 
            then it means you fail. 
      \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf This course is based on the following three books:}\\
\begin{itemize}
\item S. H. Kan:
   \begin{itemize}
   \item {\it Metrics and Models in Software Quality Engineering.}
   \item  Addison-Wesley 1995.
   \end{itemize}
\item S. L. Pfleeger:
   \begin{itemize}
   \item {\it Software Engineering: Theory and practice.}
   \item  Prentice Hall, 1999.
   \end{itemize}
\item A. von Mayrhauser:
   \begin{itemize}
   \item {\it Software Engineering: Methods and Management.}
   \item  Academic Press, 1990.
   \end{itemize}
\end{itemize}

\vspace{1in}
\noindent
Available from, e.g.,

\begin{itemize}
\item {\it http://www.amazon.com}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Contents:}\\

\begin{itemize}
\item [1.] Introduction
   \begin{itemize}
   \item[$\star$] What is Software Engineering?
   \item[$\star$] Why Software Engineering?
   \end{itemize}
\item [2.] Measurement.
\item [3.] Modeling. 
\end{itemize}

%========== New Chapter ==========
\clearpage
\begin{Huge}
\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
{\bf
Chapter 1 \\
\hspace*{1in}\\
\hspace*{1in}Introduction
}
\end{Huge}

\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
Contents:
\begin{itemize}
\item[(1)] What is Software Engineering?
\item[(2)] Why Software Engineering?
\item[(3)] Examples of badly designed/developed system
\item[(4)] Two criteria in Software Engineering
   \begin{itemize}
   \item[$\star$] Safety critical system
   \item[$\star$] Six sigma criterion
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf What is Software Engineering?}\\

\begin{itemize}
\item An engineering that helps us 
   \begin{itemize}
   \item[] to design \& develop ``{\it high quality software.}''\\
   \end{itemize}
\item What then is a ``{\it high quality software}?''
   \begin{itemize}
   \item more efficient;
   \item more precise;
   \item easier to modify;
   \item easier to use;
   \item easier to understand.\\
   \end{itemize}
\item How about the quality of our daily life?
   \begin{itemize}
   \item Where is the button that should be at the sleeve?
   \item Why streets are like a river once it rains?
   \item Is there any room to improve water drains in our houses?
   \item Why should I fixed the washing machine once a year?\\
   \end{itemize}
\item Software Engineering is also a technique to complete
   \begin{itemize}
   \item {\it A mission Impossible}.\\
   \end{itemize}
\item The term ``{\it Software Engineering}'' was first used 
      in 1968 at a NATO conference.       
\end{itemize}

%%\vfill
%%{\bf Keywords:
%%\begin{itemize}
%%\item neuron,synapse
%%\end{itemize}}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Why Software Engineering?}\\

\begin{itemize}
\item Badly developed system
   \begin{itemize}
   \item[]{\it would cost more} than properly developed.
   \end{itemize}
\item Improperly designed or programmed software
   \begin{itemize}
   \item[]{\it would threaten our lives.}
   \end{itemize}
\item Sometimes it is
   \begin{itemize}
   \item[]{\it impossible to test.}\\
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Examples of badly designed/developed system}\\
\begin{itemize}
\item A \$4-billion fiasco due to inadequate planning.
   \begin{itemize}
   \item US Internal Revenue Service's project to build an automated 
         federal income tax form processing system proved inadequate to 
         the workload and must be replaced and additionally costed:
      \begin{itemize}
      \item replacement cost;
      \item interest for delayed refunds to taxpayers;
      \item employees' overtime wages for that job. (1985)
      \end{itemize}
   \end{itemize}
\item THERAC-25 killed patients!
   \begin{itemize}
   \item A radiation therapy machine malfunctioned 
         and killed several patients 
         by not anticipated the nonstandard usage of arrow keys. (1993)
   \end{itemize}
\item A jet fighter targeted a traffic radar gun!
   \begin{itemize}
   \item One day in Scotland, a radar gun to identify speeding motorists 
         locked up indicating over 300 miles per hour. Second later a low
         flying Harrier jet flew by whose target seeker had recognized 
         the radar and thought it to belong to an enemy (1996). 
   \end{itemize}
\item Pilot's computer error caused his plane crash.
   \begin{itemize}
   \item The code for Cali was the same as the one for Bogoda
         and this causes the American Airline's jet to hit a mountain
         in Columbia. 159 people died. (1996)
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Two Criteria in Software Engineering:}
\begin{itemize}
\item Safety Critical System
   \begin{itemize}
   \item A system whose failure\footnote{A {\it fault} occurs when a human 
      makes a mistake called an {\it error} while a failure is a 
      departure from the system's required behavior.}
      poses a threat to life or health
      should have reliability of at least $10^9$, that is,
      the system can fail no more often than once in $10^9$
      hours (= 114,000 years) of operation.
   \end{itemize}
\item Motorola's Six Sigma Strategy\footnote{See the next page for more 
                                             in detail}
   \begin{itemize}
   \item less than 3.4 defective parts per million $(10^6)$ parts.\\
   \end{itemize}
\end{itemize}

\hspace*{1in}\\
\begin{center}
%%\underline{\hspace*{7in}} \\ 
How can we estimate that a criterion is fulfilled?\\
$\Downarrow$ \\
By creating a {\it model}.\\
$\Downarrow$ \\
Measure and plot the data to ascertain the model.\\
$\Downarrow$ \\
Measure what/how?\\
$\Downarrow$ \\
Metrics should be proposed to establish a model.\\
$\Downarrow$ \\
So, the next topics are:\\
$\Downarrow$ \\
What metrics can be used and how they should be measured?
\end{center}

%---------------------------------------------------------------------
\clearpage
\noindent
{\large (But before proceeding)}\\
$\Box$ {\bf What does it mean by 3.4 ppm?}

\begin{itemize}
\item If we assume the Gaussian distribution:
   \begin{itemize}
   \item The probability of defect free parts being distributed within
      \begin{itemize}
      \item $[ -\sigma, \sigma]$ is 68.26 \%;
      \item $[ -2\sigma, 2\sigma]$ is 95.44 \%;
      \item $[ -3\sigma, 3\sigma]$ is 99.73 \%;
      \item $[ -4\sigma, 4\sigma]$ is 99.9937 \%;
      \item $[ -5\sigma, 5\sigma]$ is 99.999943 \%;
      \item $[ -6\sigma, 6\sigma]$ is 99.9999998 \%.\\
      \end{itemize}
   \item Hence the proportion of the outside $[ -6\sigma, 6\sigma]$
         is 0.2 ppm and more precisely speaking 0.1 ppm for the right hand 
         side of the region and 0.1 ppm for the left hand side.\\
   \item But sometimes the distribution shifts due to variations 
         in process execution, and the maximum process shifts by research 
         is $1.5 \sigma$. \\
   \item So the interval of $[ -6\sigma, 6\sigma]$ sifts between
         $[ -7.5\sigma, 4.5\sigma]$ and $[ -4.5\sigma, 7.5\sigma]$ \\
   \item It results in 0.34 ppm for the right hand side of the region and 
         practically 0 ppm for the left hand side. \\
   \end{itemize}
\end{itemize}

%========== New Chapter ==========
\clearpage
\begin{Huge}
\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
{\bf Chapter 2 \\
\hspace*{1in}\\
\hspace*{1in}Measurement}
\end{Huge}

\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
Contents:
\begin{itemize}
\item[(1)] Terminology in measuring
\item[(2)] Measurement Errors
\item[(3)] Assessing Reliability
\item[(4)] Correction for Correlation
\item[(5)] Be careful with Correlation
\item[(6)] Examples of variables in Software Engineering
\item[(7)] Four Levels of Measurement
\item[(8)] Examples of Model
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Terminology in measuring I:}\\
%%\hspace*{.3in}
%%Ratio, Proportion, Fraction, Percentage, and Rate.\\

\begin{itemize}
\item Ratio:
   \begin{itemize}
   \item One quantity divided by another;
   \item Both quantities are from two distinct populations, and
   \item should be {\it mutually exclusive}
   \item Example:
      \begin{equation}
      \frac{\mbox{\{number of males\}}}{\mbox{\{number of males\}}}
      \end{equation}
   \end{itemize}
\item Proportion:
   \begin{itemize}
   \item Numerator should be part of the denominator;
   \item Both numerator and denominator should be integer.
   \item Example:
      \begin{equation}
      \frac{a}{a+b+c+d+e}
      \end{equation}
   \end{itemize}
\item Fraction
   \begin{itemize}
   \item The same as proportion but when the measurement unit is not integer.
   \end{itemize}
\item Percentage
   \begin{itemize}
   \item Proportion/Fraction $\times$ 100.
   \end{itemize}
\clearpage
\item Rate
   \begin{itemize}
   \item A measure of change in one quantity $(y)$ per unit of another 
         quantity $(x)$ on which the former $(y)$ depends;
   \item Both $x$ and $y$ are a function of time, i.e.,
   \item rate is {\it dynamic} while Proportion, Fraction, Percentage are
         {\it static}.
   \item Example:
      \begin{equation}
      \mbox{\{birth rate\}} = 
        \frac{\mbox{\{total number of birth in a given year\}}}
             {\mbox{\{mid-year population\}}}
      \end{equation}
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Terminology in measuring II:}\\
\hspace*{.3in}
Reliability and Validity \\
\hspace*{1in}$\cdots$ Criteria of measurement quality.\\

\begin{itemize}
\item Reliability:
   \begin{itemize}
   \item Consistency of a number of measurements using the same method 
         on the same subject.
   \item A degree of {\it preciseness}.
   \item It can be expressed as:
      \begin{equation}
      \mbox{\{Index of Variation\}} = \frac{\sigma}{\mu}
      \end{equation}   
   \item The smaller the index the more reliable.\\
   \end{itemize}
\item Validity:
   \begin{itemize}
   \item Whether the measurement really measures what we intend it 
         to measure.
   \item A degree of {\it accuracy}.\\
   \end{itemize}
\item[] Note that:
   \begin{itemize}
   \item Reliable measurements may not necessarily be valid.
   \end{itemize}
\end{itemize}

\begin{figure}[h]
 \begin{center}
   \epsfile{file=reliability.eps,width=14cm,height=3.5cm}
 \end{center}
\end{figure}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Measurement Errors}\\

\begin{equation}
M = T + s + e.
\end{equation}

\begin{itemize}
\item[]where
   \begin{itemize}
   \item[] $M \cdots$ observed (measured) score;
   \item[] $T \cdots$ true score;
   \item[] $s \cdots$ systematic error (associated with validity);
   \item[] $e \cdots$ random error (associated with reliability).\\
   \end{itemize}
\item[] But under a valid measurement:
\begin{equation}
M = T + e.\\
\end{equation}
So,
\begin{equation}
var(M) = var(T) + var(e).\\
\end{equation}
where
\begin{center} $var(\cdot)$ represents variance \end{center}
and \\
\begin{equation}
\mbox{Reliability} = \frac{var(T)}{var(M)} = 1 - \frac{var(e)}{var(M)}
\end{equation}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Assessing Reliability}\\

\begin{itemize}
\item Retest Method\footnote{Carmines. E.\ G.\  et al.\ proposed
the following method to assess the reliability of empirical measurements: \\
\indent
\hspace*{.5in}
(1) Retest, (2) Alternative-form, (3) Split-halves, and 
(4) Internal consistency\\
in their book ``Reliability and Validity Assessment.'' Beverly Hills. California: Sage Publications. (1979)
}
   \begin{itemize}
   \item[(1)] Measure the subjects twice.
      \begin{itemize}
      \item[]If the measurement is reliable the 1st and 2nd measurement 
             are highly correlated. So,
      \end{itemize}
   \item[(2)] Compute the correlation\footnote{Correlation between
   $x_i$ and $y_i$ $(i=1,\cdots, N)$ is defined as
   \begin{equation}
   \frac{\displaystyle \sum_{i=1}^N (x-\bar{x})(y-\bar{y})}
        {\sigma_x \sigma_y}
   \end{equation}} between the first and the second measurements.
   \end{itemize}
\item[] Then
   \begin{itemize}
   \item[] the correlation coefficient is the reliability.
   \end{itemize}
\end{itemize}

\hspace*{1in}\\

\noindent
$\Box$ {\bf Correction for Correlation}\\

\hspace*{.4in}
\{estimated true correlation between $x$ and $y$\}
\begin{equation}
= \frac{\mbox{ \{observed correlation between $x$ and $y$\}}}
{\sqrt{\mbox{ \{reliability of} \hspace{.1in}x\} 
       \mbox{ \{reliability of} \hspace{.1in}y\}}}
\end{equation}

\hspace*{1in}\\

\noindent
$\Box$ {\bf Be careful with Correlation}\\

\begin{itemize}
\item Plot the data before calculating correlation coefficient!
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Examples of variables in Software Engineering}\\
\begin{itemize}
\item Which development process was employed?
   \begin{itemize}
   \item[$\cdot$]waterfall process
   \item[$\cdot$] spiral process
   \item[$\cdot$] iterative process
   \item[$\cdot$] object-oriented programming process
   \end{itemize}
\item The quality of software inspection.
   \begin{itemize}
   \item[$\cdot$] very effective
   \item[$\cdot$] effective
   \item[$\cdot$] somewhat effective
   \item[$\cdot$] not effective
   \item[$\cdot$] poor
   \end{itemize}
\item The inspection coverage
   \begin{itemize}
   \item[$\cdot$] argument is percentage
   \end{itemize}
\item The total number of defects customers will encounter for a software 
      product over a specified time length
   \begin{itemize}
   \item[$\cdot$] integer argument
   \end{itemize}
\item The process rigor scale.
   \begin{itemize}
   \item[$\cdot$] totally adheres to process
   \item[$\cdot$] somehow adheres to process
   \item[$\cdot$] does not adhere to process
   \end{itemize}
\clearpage
\item Customer satisfaction.
   \begin{itemize}
   \item[$\cdot$] completely satisfied
   \item[$\cdot$] somewhat satisfied
   \item[$\cdot$] neutral
   \item[$\cdot$] dissatisfied
   \item[$\cdot$] completely dissatisfied
   \end{itemize}

\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Four Levels of Measurement}\\
\begin{itemize}
\item Nominal scale:
   \begin{itemize}
   \item A classification into 
      \begin{itemize}
      \item {\it Mutually exclusive}\footnote{A subject can be classified 
               into one and only one category:} and 
            {\it jointly exhaustive}\footnote{All categories together 
               should cover all possible cases.}
            classes.
      \end{itemize}
   \end{itemize}
\item Ordinal scale:
   \begin{itemize}
   \item When the classified subjects can be compared in order.
   \item No information of magnitude of difference:
      \begin{itemize}
      \item Not any of {\it addition}, {\it subtraction}, 
            {\it multiplication}, and {\it division} are possible.
      \item Only {\it greater than} and {\it less than} are possible.
      \item Transitive in that if $A>B$ and $B>C$ then $A>C$.
      \end{itemize}
   \end{itemize}
\item Interval scale:
   \begin{itemize}
   \item When we have exact differences between measurements.
   \item {\it addition} and {\it subtraction} are possible.\footnote{We can
 say for example that ``quantity A is 1.5 higher than B,'' or ``The 
difference between A and B is the same as the one between C and D.''}  
   \end{itemize}
\item Ratio scale:
   \begin{itemize}
   \item When an absolute or non arbitrary zero point\footnote{We cannot
say an average temperature of $40^{\circ}$C in summer is 8 times higher
than an average of $5^{\circ}$C in winter.} exists.
   \item {\it addition}, {\it subtraction}, {\it multiplication}, and
         {\it division} are possible.\footnote{We can say for example that
          A is twice as much as B.}\\
   \end{itemize}
\end{itemize}
\noindent
These four scales are hierarchical in that higher level scale has 
all properties of the lower ones.

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Examples of Model (Propositions)}
\begin{itemize}
\item    The more rigorously the front end of the software development 
         process is executed, the better the quality at the back end.
\item[]
\item    The higher the percentage of the designs and code that are 
         inspected,
\item[]  The more effective the design reviews and the code inspections 
         as scored by the inspection team,
\item[]  The more thorough the development test (in terms of test coverage)
         done before integration,
\item[] \hspace*{1in}$\Downarrow$
\item[]  the lower the defect rate will also encountered 
         at the formal machine testing phase.
\end{itemize}

\hspace*{1in}\\
\hspace*{1in}\\
\noindent
$\Box$ {\bf Then questions arise are:}
\begin{itemize}
\item What is the {\it software development process?}
\item What are the {\it front end} and the {\it back end.}
\item What is quality of the back end?
\item[]\hspace*{1in}$\Downarrow$ 
\item[]All should be well-defined, for example:\\
\end{itemize}
\hspace*{1in}\hfill (to continue)\\ 
\clearpage
\begin{itemize}
\item Software Development Process is:
   \begin{itemize}
   \item[$\star$] Requirement gathering $\Rightarrow$
                  Design $\Rightarrow$ 
                  Design reviews \& inspection $\Rightarrow$ 
                  Code $\Rightarrow$ 
                  Code inspection $\Rightarrow$ 
                  Debug \& development test $\Rightarrow$ 
                  Integration $\Rightarrow$ 
                  Formal machine testing $\Rightarrow$ 
                  early customer program\\
   \item[$\star$] front end $\cdots$ before the integration process
   \item[$\star$] back end  $\cdots$ after the integration process\\
   \end{itemize}
\end{itemize}

\indent
Or, which measurement indicators are to be used?\\
\indent
For example: Indicators for\\

\begin{itemize}
\item Reviews \& inspection
   \begin{itemize}
   \item[(1)]Inspection coverage expressed in terms of the percentage 
             of the estimated lines of code that are actually inspected.
   \item[(2)]Scoring of each inspection by the inspectors 
             at the end of inspection.
   \end{itemize}
\item Rigorous execution of the development test:
   \begin{itemize}
   \item[(1)]The percent coverage in terms of instructions executed.
   \item[(2)]The defect rate expressed in terms of number of defects 
             removed per KLOC.
   \end{itemize}
\end{itemize}

%========== New Chapter ==========
\clearpage
\begin{Huge}
\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
{\bf Chapter 3 \\
\hspace*{1in}\\
\hspace*{1in}Software metrics}
\end{Huge}

\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
Contents:
\begin{itemize}
\item[3.1.] Three categories of software metrics
\item[3.2.] Software quality metrics
   \begin{itemize}
   \item[3.2.1.] Product quality metrics
      \begin{itemize}
      \item[(1)] Defect density revisit
         \begin{itemize}
         \item[i)]   Defect density for the updated version
         \item[ii)]  A method of counting updated LOC
         \item[iii)] An example of estimating updated release
         \item[iv)]  Customer's perspective of defect rate
  \end{itemize}
      \end{itemize}
   \end{itemize}
\item[3.3.] End-product quality metrics
   \begin{itemize}
   \item[3.3.1.] Customer problems metric
   \item[3.3.2.] Function Point (FC) instead of KLOC
   \end{itemize}
\item[3.4.] In-process quality metrics
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Three categories of Software metrics.}\\

\begin{itemize}
\item Product metrics
   \begin{itemize}
   \item describe the characteristics of the product, such as:
      \begin{itemize}
      \item size
      \item complexity
      \item design features
      \item performance
      \item quality level
      \end{itemize}
   \end{itemize}
\item Process metrics
   \begin{itemize}
   \item can be used for improving the software development and 
         maintenance process, e.g.,
      \begin{itemize}
      \item effectiveness of defect removal during development
      \item pattern of testing defect arrival
      \item response time of the fix process
      \end{itemize}
   \end{itemize}
\item Project metrics
   \begin{itemize}
   \item describe the project characteristics and execution, e.g.,
      \begin{itemize}
      \item number of software developers
      \item developers' skill level
      \item staffing pattern over the life cycle of the software
      \item cost
      \item schedule
      \item productivity
      \end{itemize}
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Software quality metrics.}
\begin{itemize}
\item[]A subset of software metrics that relates with the quality
       of the product, that is:
   \begin{itemize}
   \item[$\star$] Product quality metrics
   \item[$\star$] Process quality metrics
   \item[$\star$] Project quality metrics\\
   \end{itemize}
\end{itemize}

\hspace*{1in} $\Downarrow$
Further divided into two categories:\\

\begin{itemize}
\item end-product quality metrics
\item in-process quality metrics\\
\end{itemize}

\hspace*{1in} $\Downarrow$
What is Software Engineering (revisit)\\

\newtheorem{se}{Definition}%%[section]
\begin{se}[Software Engineering]
Software Engineering is  to investigate the relationships 
between in-process metrics and end-product quality.
\end{se}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Product Quality Metrics.}\\

\begin{itemize}
\item Intrinsic product quality
   \begin{itemize}
   \item[(1)] Mean time to failure (MTTF)
      \begin{itemize}
      \item[e.g.] $\cdots$ $10^{-9}$ criteria in safety critical system.
      \end{itemize}
   \item[(2)]  Defect density
      \begin{itemize}
      \item[e.g.] $\cdots$ 3.4 defects per million lines in 
                  six-sigma strategy.
      \end{itemize}
   \end{itemize}
\item Customer satisfaction
   \begin{itemize}
   \item[(3)]  Customer problems
   \item[(4)]  Customer satisfaction
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Defect density revisit}\\

\begin{itemize}
\item How do we count lines of code (LOC)?
   \begin{itemize}
   \item only executable lines?
   \item plus data definition?
   \item further plus comments?
   \item physical lines on an input screen?
   \item as the number of logical delimiters?\\
   \end{itemize}
\end{itemize}

\hspace*{1in} $\Downarrow$
A definition of LOC (Conte et al,1986)\\

\noindent
{\it Any line of program text that is not a comment or blank line,
regardless of the number of statements or fragments of statements
on the line including all lines containing headers, declarations, 
executable, and non executable statements.}


%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Defect density for the updated version.}\\

\begin{itemize}
\item For the first version, it is easy to state, for example,
   \begin{itemize}
   \item[]''This product has a total of 50 KLOC and the latent defect
            rate during the next four years is 2 defects/KLOC.''
   \end{itemize}
\item[]\hspace*{1in}$\Downarrow$
\item What about subsequent release of the product?
\item[]\hspace*{1in}$\Downarrow$
\item[]We need to measure the quality of
   \begin{itemize}
   \item the entire product for the new release; and
   \item the portion of the product that is new.
      \begin{itemize}
      \item[] (true development quality, i.e., 
      \item[] \hspace*{.2in}$\Rightarrow$ 
              the defect rate of the new and changed code)
      \end{itemize}
   \end{itemize}
\item[]\hspace*{1in}$\Downarrow$
\item Methods of evaluating defect of the {\it new and changed code.}
   \begin{itemize}
   \item Change-flagging method
   \item Delta-library method
   \item etc
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf A method of counting updated LOC.}\\

\begin{itemize}
\item change-flagging method
   \begin{itemize}
   \item When a new function is added or an enhancement is made to an 
         existing function:
      \begin{itemize}
      \item the new and changed lines of code are flagged with a specific 
            ID number (change-flagging ID) as comments; and
      \item the ID is linked with the requirements number (requirements ID)
            which is usually in the module header.
      \end{itemize}
   \item All of these IDs are further linked to the release number
   \item[] \hspace*{1in}$\Downarrow$
   \item When a defect is reported, the developer can determine
      \begin{itemize}
      \item in which function or enhancement pertaining to 
      \item what requirements, 
      \item at what release,
      \end{itemize}
   \item[]the defect was injected.
   \end{itemize}
%%\clearpage
%%\item delta-library method:
%%   \begin{itemize}
%%   \item By comparing program modules in the original library 
%%         with the new versions in the current release library.
%%   \item Tools exist to determine the amount of new and changed code 
%%         for the new release.
%%   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf An example of estimating updated release.}\\

\begin{itemize}
\item SSI \& CSI (IBM Rochester)\\
   \begin{itemize}
   \item[$\cdot$] SSI (Shipped Source Instructions)
   \item[$\cdot$] CSI (new and Changed Source Instructions)\\
   \end{itemize}
%\item[] $\star$ SSI of current release
%\begin{eqnarray}
%\nonumber
%\hspace*{.2in} &=& \mbox{SSI of previous release} \\
%\nonumber
%               &+& \mbox{CSI of current release} \\
%\nonumber
%               &-& \mbox{number of deleted code} \\
%\nonumber
%               &-& \mbox{number of changed code} \\
%\nonumber
%               & & \mbox{\Large(to avoid double count in both SSI and CSI)}
%\end{eqnarray}
\item[] $\star$ SSI of current release = SSI of previous release \\
\hspace*{.9in}+ CSI of current release -- number of deleted code \\
\hspace*{.9in}-- number of changed code \\
\hspace*{1in}{\Large(to avoid double count in both SSI and CSI)}
\clearpage
\item TVUA \& TFVUA \\ 
   \begin{itemize}
   \item[$\cdot$] APAR (Authorized Program Analysis Reports)
      \begin{itemize}
      \item[]\hspace*{.5in}$\cdots$ defects identified after the release
      \end{itemize}
     \begin{itemize}
      \item[$\cdot$] field APAR $\cdots$ opened due to customer problems
      \item[$\cdot$] internal APAR $\cdots$ opened internally
      \end{itemize} 
   \item[$\cdot$] TVUA (Total Valid Unique APAR)
   \item[$\cdot$] TFVUA (Total Field Valid Unique APAR)
   \item[]
   \item[] $\star$ $\displaystyle \frac{\mbox{TVUA}}{\mbox{KSSI}}$
           $\cdots$ total release code quality\\
   \item[] $\star$ $\displaystyle \frac{\mbox{TVUA}}{\mbox{KCSI}}$
           $\cdots$ quality of the new and changed code\\
   \item[]\hspace*{.2in}(Both are the same for the 1st release)\\
   \item[] These two are {\it process measure}, while\\
   \item[] $\star$ $\displaystyle \frac{\mbox{TFVUA}}
                                       {\mbox{KSSI or KCSI}}$\\
   \item[] is {\it customer's perspective}.
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Customer's perspective of defect rate.}\\

\begin{itemize}
\item From customer's perspective, {\it total number} instead of
      {\it rate} of defects should decrease from release to release,
      regardless of {\it size}, while from developer's point of view
      {\it rate} is more important.\\
\item If the size of a new release becomes much larger than the previous
      release, then defect rate goal should be significantly better than 
      the previous one in order to reduce the total number of defects.\\
\clearpage
\item For example:
   \begin{itemize}
   \item The initial release:
      \begin{itemize}
      \item KCSI = KSSI = 50 KLOC
      \item Defects / KCSI = 2.0
      \item Total defects = ?
      \end{itemize}
   \item The second release:
      \begin{itemize}
      \item KCSI = 20
      \item KSSI = 50 + 20 -- 4 = 66
      \item Defects / KCSI = 1.8 (10 \% improvement)
      \item Total defects = 36
      \end{itemize}
   \item The third release:
      \begin{itemize}
      \item KCSI = 40
      \item KSSI = 66 + 40 -- 10 = 96
      \end{itemize}
   \end{itemize}
\clearpage
\item[] From developer's point (they care defect rate):
   \begin{itemize}
   \item defects/KSSI in the 3rd release should be less than 
         1.62 in order to maintain the improve ration of 10 \% in 
         the 2nd release.\\
   \end{itemize}
\item[] From customer's point (they care total number of defect):
   \begin{itemize}
   \item The number of defects in the 3rd release should be 
         {\it at least} the same as the one in the 2nd release, i.e., 36.
         So, defect/KCSI in the 3rd release should be less than
         0.9 \% (36/40) which implies 50 \% improvement of the 2nd one. 
   \item And hopefully the number of defect in the 3rd release should
         be less than 13 ($36 \times 36/100 \approx 13$)
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Customer Problems Metric}\footnote{Recall we are studying
\begin{itemize}
\item Product Quality Metrics
   \begin{itemize}
   \item MTTF
   \item Defect Density
   \item Customer Problems
   \item Customer Satisfaction
   \end{itemize}
\end{itemize}
}
\begin{itemize}
\item[---] metric which denotes the problems customers encounter 
           when using the product.\\
\end{itemize}

\noindent
$\bullet$ It is made up of (i) non-defect-oriented problems such as:
\begin{itemize}
\item[$\cdot$] usability problems;
\item[$\cdot$] unclear documentation;
\item[$\cdot$] unclear information;
\item[$\cdot$] defects that were reported by other customers 
               but the current customers were not aware of;
\item[$\cdot$] user errors;
\end{itemize}

\indent
as well as (ii) defects.\\

\hspace*{1in}\\
\noindent
$\bullet$ It is measured by {\it PUM} (problems per user month) as:

\begin{large}
\begin{equation}
PUM = \frac{\mbox{\{Total defect and no-defect problem reported 
                    for a period\}}}
{\mbox{\{Total number of license-months 
         of the software during the period\}}}
\end{equation}
\end{large}

\noindent
where\\
\begin{large}
\hspace*{.5in} \{number of license-month\} \\
\hspace*{1in} = \{number of licenses installed\} $\times$
\{number of month in the period\}.
\end{large}

\clearpage
\noindent
$\bullet$ The other possibilities to represent Customer Problem?\\
\begin{large}
\begin{itemize}
\item[$\star$] $\displaystyle
               \frac{\mbox{\{defects\}}}{\mbox{\{source instructions\}}}$
\item[$\star$] $\displaystyle
               \frac{\mbox{\{problems\}}}
                    {\mbox{\{usage of the product\}}}$\\
\end{itemize}
\end{large}

\hspace*{1in}\\
\indent 
these two metrics works as product quality, but
\hspace*{1in}\\
\begin{large}
\begin{itemize}
\item[$\star$] $\displaystyle
               \frac{\mbox{\{total customer problems\}}}
                    {\mbox{\{size of the product\}}}$ 
\end{itemize}
\end{large}

\hspace*{1in}\\
\indent
doesn't work. Why?\\

\hspace*{1in}\\
\noindent
$\bullet$ {\it ``Customer Problem Metric''} can be regarded as 
\begin{itemize}
\item[--] an intermediate measure between 
      {\it ``defect measurement''} and {\it ``customer satisfaction''}.\\
\end{itemize}

\begin{figure}[h]
 \begin{center}
   \epsfile{file=d-cp-cs.eps,width=16cm,height=8cm}
 \end{center}
\end{figure}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Alternative measure for LOC}\\

\hspace*{.5in}--- Function Point Metric\\

\begin{itemize}
\item For example:
   \begin{large}
   \begin{itemize}
   \item[$\star$] $\{productivity\} = \displaystyle 
          \frac{\mbox{\{function points\}}}{\mbox{\{person-year\}}}$\\
   \item[$\star$] $\{quality\} = \displaystyle 
          \frac{\mbox{\{defects\}}}{\mbox{\{function points\}}}$\\
   \end{itemize}
   \end{large}
\item[]
\item A function can be defined as 
   \begin{itemize}
   \item[-] a collection of executable statements that performs 
              a certain task 
   \end{itemize}
together with 
   \begin{itemize}
   \item[-] declarations of formal parameters and local variables.
   \end{itemize}

\item[]
\item If defects per unit of functions is low, the quality is better 
      even though the software defects per KLOC are high
      (when the functions were implemented by fewer lines of code.)
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf How can we obtain Function Point?}\\

\begin{itemize}
\item[I)] Divide all the functions in the software we want to asses
          into 5 categories bellow. Then the complexity of each function
          of each category is estimated such as 
          low, medium or high complexity.
          That is to say, we categorize the functions into 15 groups.
          The total number of the functions in each category is denoted as
           $x_{ij}$, and its complexity weighting factor is
           $w_{ij}$\footnote{The complexity classification
           of each component is based on a set of standards that define 
           complexity in terms of objective guidelines.}
\end{itemize}



\begin{itemize}
\item[II)] Then Function Count $FC$ is:
\begin{equation}
FC = \sum_{i=1}^5 \sum_{j=1}^3 w_{ij} \times x_{ij}
\end{equation}
\clearpage 
\item[III)] Evaluate the following 14 system characteristics $c_i$,
            in terms of their likely effect for the application:
   \begin{itemize}
   \item[$\cdot$] Data Communications
   \item[$\cdot$] Distributed functions
   \item[$\cdot$] Performance
   \item[$\cdot$] Heavily used configuration
   \item[$\cdot$] Transaction rate
   \item[$\cdot$] On-line data entry
   \item[$\cdot$] End-user efficiency
   \item[$\cdot$] On-line update
   \item[$\cdot$] Complex processing
   \item[$\cdot$] Reusability
   \item[$\cdot$] Installation ease
   \item[$\cdot$] Operational ease
   \item[$\cdot$] Multiple sites
   \item[$\cdot$] Function of change\\
   \end{itemize}
\item[IV)] Then Value Adjustment Factor $VAF$ is defined as
   \begin{equation}
   VAF = 0.65 + 0.01 \cdot \sum_{i=1}^{14} c_i.
   \end{equation}
\item[]
\item [V)] Finally we obtain Function Point $FP$ as:
   \begin{equation}
   FP = FC \times VAF.
   \end{equation}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf In-process Quality Metrics}\\

\begin{itemize}
\item Defect Density during Machine Testing\footnote{Recall we have the
following developing phases:
\begin{itemize}
\item[$\star$] Requirement collection $\rightarrow$ Design $\rightarrow$ 
\end{itemize}}(DDMT)
   \begin{itemize}
   \item measured with 
         {\large $\displaystyle \{\frac{\mbox{defects}}{\mbox{KLOC}}\}$}\\
   \item[$\star$]The more DDMT, the more defects will be found later?\\
   \item If {\it the current} $DDMT \le$ {\it the previous release}
      \begin{itemize}
      \item Ask if the test for current release deteriorate.
         \begin{itemize}
         \item[-] No $\Rightarrow$ Quality improved.
         \item[-] Yes $\Rightarrow$ Do extra test such as:\\
         \hspace{.3in}$\cdot$ test with more cases to increase coverage;\\
         \hspace{.3in}$\cdot$ blitz test; \\
         \hspace{.3in}$\cdot$ stress test; \\
         \hspace{.3in}$\cdot$ test by customer.
         \end{itemize}
      \end{itemize}
   \item If {\it the current} $DDMT >$ {\it the previous release}
      \begin{itemize}
      \item Ask if we improved testing effectiveness.
         \begin{itemize}
         \item[-] Yes $\Rightarrow$ The quality remains same or improved.
         \item[-] No  $\Rightarrow$ Quality perspective negative.
         \end{itemize}
      \end{itemize}
   \end{itemize}
\clearpage
\item Defect Arrival Pattern during Machine Testing
   \begin{itemize}
   \item The pattern of defect arrivals (reported), 
         or equivalently times between failures, gives more information 
         than the overall defect density.
   \item Even with the same overall defect rate, 
         different patterns indicate different quality levels.
   \end{itemize}

\begin{figure}[h]
 \begin{center}
   \epsfile{file=dap-1-1.eps,width=8.9cm,height=7cm}
   \epsfile{file=dap-1-2.eps,width=8.9cm,height=7cm}
 \end{center}
\caption{An hypothetical data for DAP and its Cumulative result.}
\end{figure}
\begin{figure}[h]
 \begin{center}
   \epsfile{file=dap-2-1.eps,width=8.9cm,height=7cm}
   \epsfile{file=dap-2-2.eps,width=8.9cm,height=7cm}
 \end{center}
\caption{Another hypothetical data for DAP and its Cumulative result.}
\end{figure}
\clearpage
   \begin{itemize}
   \item The top Figure will give a better quality 
         than the bottom Figure where:
      \begin{itemize}
      \item Test started late;
      \item Test suite was not sufficient; or
      \item Test ended prematurely.\\
      \end{itemize}
   \item The goal is to make sure that:
      \begin{itemize}
      \item the defect arrivals stabilizes at a very low level; and
      \item the times between the failures becomes long.\\
      \end{itemize}
   \end{itemize}
\clearpage
\item[]
\item Phase-based Defect Removal Pattern
   \begin{itemize}
   \item In which phase\footnote{Here we have the following six phases:
               \begin{itemize}
               \item[$\cdot$] I0: high-level design review
               \item[$\cdot$] I1: low-level design review
               \item[$\cdot$] I2: code inspection
               \item[$\cdot$] UT: unit test
               \item[$\cdot$] CT: component test
               \item[$\cdot$] ST: system test
               \end{itemize}
         } were defects removed?
   \end{itemize}
\begin{figure}[h]
 \begin{center}
   \epsfile{file=pdrp-1.eps,width=8.9cm,height=7cm}
   \epsfile{file=pdrp-2.eps,width=8.9cm,height=7cm}
 \end{center}
\caption{An hypothetical data for PDRP.}
\end{figure}
\clearpage
\item Defect Removal Effectiveness (DRE)\\
\hspace*{1in}\\
\begin{large}\begin{equation}
DRE = \frac{\mbox{\{defects removed during a development phase\}}}
           {\mbox{\{defects latent\}}}\\
\end{equation}\end{large}
\hspace*{1in}\\
\item[-] where denominator is usually unknown and evaluated as
\hspace*{1in}\\
\begin{large}\begin{equation}
\mbox{\{defects removed during the phase\}}\hspace{.1in} + \hspace{.1in}
\mbox{\{defects found later\}}
\end{equation}\end{large}
\hspace*{1in}\\
\item[-] We call it
   \begin{itemize} 
   \item[$\star$]{\it early defect removal} 
   \end{itemize}
for the phases in the front end; and
   \begin{itemize} 
   \item[$\star$]{\it phase effectiveness} 
   \end{itemize}
for each phase.
\begin{figure}[h]
 \begin{center}
   \epsfile{file=pbdr.eps,width=8.9cm,height=7cm}
 \end{center}
\caption{An hypothetical data for DAP and its Cumulative result.}
\end{figure}
\item[-] The higher the value, the more effective the development process; 
and
\item[-] the fewere the defects that escaped 
        to the next phase of to the field.
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Metrics for Software Maintenance}\footnote{Main metrics
during maintenance are still
\begin{itemize}
\item[$\cdot$]defect arrivals by time interval
\item[$\cdot$]customer problem calls by time interval
\end{itemize}
but they do not reflect the quality of mintenace.
}\\

\begin{itemize}
\item[---] Important to improve customer satisfaction 
           (not to improve the defect rate though) such as
   \begin{itemize}
   \item[$\cdot$] fix the defects claimed as soo as possible
   \item[$\cdot$] with excellent fix quality \\
   \end{itemize}
\item Fix Backlog (FB)
\end{itemize}
\begin{large}
\hspace*{1in}{\it FB} = \{number of reported problems that remain opened\\ 
\hspace*{3in}at the end of each period (month/week)\}\\
\end{large}
\begin{itemize}
\item Backlog Management Index (BMI)
\begin{large}\begin{equation}
BMI = \frac{\mbox{\{number of problems closed during a period\}}}
           {\mbox{\{number of problems arrived during the period\}}}
      \times 100 \% \\
\end{equation}\end{large}
\item[]
\item[]
   \begin{itemize}
   \item[-] If {\it BMI} $>$ 100 $\Rightarrow$ backlog is reduced;
   \item[-] If {\it BMI} $<$ 100 $\Rightarrow$ backlog is increases.
   \end{itemize}
\end{itemize}

\clearpage
\begin{figure}[h]
 \begin{center}
   \epsfile{file=bmi-1.eps,width=12cm,height=6cm}
   \epsfile{file=bmi-2.eps,width=12cm,height=6cm}
 \end{center}
\caption{An hypothetical data for opened/closed problems and BMI.}
\end{figure}

\begin{itemize}
\item[-] The low {\it BMI} of the 1-{\it st} month is due to the
         new release.
\item[-] The low {\it BMI} of July 1999 is due to the vacations
         around Independence day.
\end{itemize}

\hspace*{1in}\\

\clearpage
\begin{itemize}
\item[]
\item Fix Response Time (FRT)
   \begin{itemize}
   \item[-] Usually we have guidelines on the time limit within which 
            the fix should be available for the reported defects
            depending on severity
            from critical cases (where work should be around the clock
            to fix it) to less severe defects for which circumventions 
            are available (where works are somehow relax).
   \item[-] Defined as\footnote{if there exist 
extreme values, use median.} : \\
   \end{itemize}
\begin{large}\begin{equation}
FRT = \{ \mbox{meantime of all problems from open to closed}\}
\end{equation}\end{large}
\end{itemize}

\clearpage
\hspace*{1in}\\
\noindent
$\bullet$ Percent Delinquent Fixes (PDF)\\
\begin{itemize}
   \item[-] Defined as:
   \item[]
   \begin{small}\begin{equation}
   PDF=\frac{\mbox{number of fixes that exceeded the fix response time
                   criteria by severity level}}
            {\mbox{total number of fixes delivered in a specified time}} \\
   \end{equation}\end{small}
\item[]
\item[-] This is not a real time metric, 
         because only for closed problems 
\item[-] Still open problems must be considered to be a real time one\\
\item[]  \hspace*{1in}$\Downarrow$ \\
\item[-] A new proposal:
   \begin{itemize}
   \item[$\star$] Percent delinquent of all problems in the active backlog
      \begin{itemize}
      \item[$\cdot$] where {\it active backlog} is all open problems 
                    for a period, that is,\\
      \item[]{\large $\mbox{
                \{existing backlog at the beginning of the period \}}$} 
      \item[]\hspace{2in}
             {\large 
                  + $\mbox{
                \{new problem arrivals during the period \}}$}\\ 
      \item[$\cdot$] which includes the information about the total 
                     number of problems to be processed for the period; and
      \item[$\cdot$] the number of delinquent poblems is cheked 
                     at the end of the week.
      \end{itemize} 
   \end{itemize}
\end{itemize}

\clearpage
\noindent
$\bullet$ Percent Defective Fixes (DF)\\
\begin{itemize}
   \item[-] Defined as:
   \item[]
   \begin{small}\begin{equation}
   DF = \frac{\mbox{fixes that are defective}}
             {\mbox{all fixes in a time interval}}\\
   \end{equation}\end{small}
\item[]
\item[-] {\it Fix Quality} is estimated by this metric;
\item[]  and
\item[-] A fix is defective if 
   \begin{itemize}
   \item[$\cdot$]it did not fix the problem that was reported; or
   \item[$\cdot$]it fixed the original problem but injected a new defect.
   \end{itemize}
\item[]
\item[-] An argument against using percentage, where
   \begin{itemize}
   \item[$\cdot$] the absolute value of the nuber of defective fixes 
                  is to be used and goal shoul be zero defective fixes 
                  without delinquency.
   \end{itemize}
\item[]
\item[-] Two ways in recordin a defective fix:
   \begin{itemize}
   \item[$\cdot$] Record it when it was discovered (customer's measure);
   \item[$\cdot$] Record it when the fix was delivered (process measure).
   \end{itemize}
\item[]\hspace*{1in}$\Downarrow$
\item[]\begin{center}
            {\large Time diference between two dates = 
                    latent period of the defective fix.}
       \end{center}
\item[] The longer the latency, the more customers are affected.
\end{itemize}



%---------------------------------------------------------------------
\clearpage
\noindent
\begin{large}
$\Box$ {\bf Other Metrics Used in the Real World}\\

Motorola

\begin{itemize}
\item To improve Project Planning:
   \begin{itemize}
   \item[-] What was the accuracy of estimating the actual value or 
            project schedule?
      \begin{itemize}
      \item[$\star$] Schedule Estimation Accuracy (SEA)
      \begin{small}\begin{equation}
      SEA = \frac{\mbox{actual project duration}}
                 {\mbox{estimated project duration}}
      \end{equation}\end{small}
      \end{itemize}
      \item[-] What was the accuracy of estimating the actual value of 
               project effort?
      \begin{itemize}
      \item[$\star$] Effort Estimation Accuracy (EEA)
      \begin{small}\begin{equation}
      EEA = \frac{\mbox{actual project effort}}
                 {\mbox{estimated project effort}}
      \end{equation}\end{small}
      \end{itemize}
   \end{itemize}
\item To increase defect containment:
   \begin{itemize}
   \item[-] What is the currently known effectiveness of the defect 
            detection process prior to release?
      \begin{itemize}
      \item[$\star$] Total Defect Containment Effectiveness (TDCE)
      \begin{small}\begin{equation}
      TDCE = \frac{\mbox{number of prerelease defects}}
                  {\mbox{number of prerelease defects}+
                   \mbox{number of postrelease defects}}
      \end{equation}\end{small}
      \end{itemize}
   \item[-] What is the currently known containment effectiveness of 
            faults introduced during each constructive phase of software 
            development for a particular software product?
      \begin{itemize}
      \item[$\star$] Phase Containment Effectiveness for phase (PCEi)
      \begin{small}\begin{equation}
      PCEi = \frac{\mbox{number of phase}\hspace{.1in}i\hspace{.1in} 
                                                           \mbox{errors}}
                  {\mbox{number of phase}\hspace{.1in}i\hspace{.1in}
                                                           \mbox{errors} + 
                   \mbox{number of phase}\hspace{.1in}i\hspace{.1in} 
                                                           \mbox{defects}}
      \end{equation}\end{small}
      \end{itemize}
   \end{itemize}
\item To increase Software Reliability
   \begin{itemize}
   \item[-] What is the rate of software failures, and how does it 
            change over time?
      \begin{itemize}
      \item[$\star$] Failure Rate (FR)
      \begin{small}\begin{equation}
      FR = \frac{\mbox{number of failures}}{\mbox{execution time}}
      \end{equation}\end{small}
      \end{itemize}
   \end{itemize}
\item To decrease Software Defect Density
   \begin{itemize}
   \item[-] What is the normalized number of in-process faults, and how 
            does it compare with the number of in-process defects?
      \begin{itemize}
      \item[$\star$] In-process Faults (IPF)
      \begin{small}\begin{equation}
      IPF = \frac{\mbox{in-process faults caused by incremental software 
                        development}}
             {\mbox{assembly-equivalent delta source size}}
      \end{equation}\end{small}
      \item[$\star$] In-process Defects (IPD)
      \begin{small}\begin{equation}
      IPD = \frac{\mbox{in-process defects caused by incremental software 
                        development}}
             {\mbox{assembly-equivalent delta source size}}
      \end{equation}\end{small}
      \end{itemize}
   \item[-] What is the currently known defect content of software 
            delivered to customers, normalized by assembly-equivalent size?
      \begin{itemize}
      \item[$\star$] Total Released Defects Total (TRDT)
      \begin{small}\begin{equation}
      TRDT = \frac{\mbox{number of released defects}}
             {\mbox{assembly-equivalent total source size}}
      \end{equation}\end{small}
      \item[$\star$] Total Released Defects Delta (TRDD)
      \begin{small}\begin{equation}
      TRDD = \frac{\mbox{number of released defects caused by incremental 
                    software development}}
             {\mbox{assembly-equivalent total source size}}
      \end{equation}\end{small}
      \end{itemize}
   \item[-] What is the currently known customer-found defect content of 
            software delivered to customers, normalized by 
            assembly-equivalent source size?
      \begin{itemize}
      \item[$\star$] Customer-Found Defects Total (CFDT)
      \begin{small}\begin{equation}
      CFDT = \frac{\mbox{number of customer-found defects}}
                  {\mbox{assembly-equivalent total source size}}
      \end{equation}\end{small}
      \item[$\star$] Customer-Found Defects Delta (CFDD)
      \begin{small}\begin{equation}
      CFDT = \frac{\mbox{number of customer-found defects caused by
                         incremental software development}}
                  {\mbox{assembly-equivalent total source size}}
      \end{equation}\end{small}
      \end{itemize}
   \end{itemize}
\item To improve Customer Service
   \begin{itemize}
   \item[-] What is the number of new problems that were opened during 
            the month?
      \begin{itemize}
      \item[$\star$] New Open Problems (NOP)
      \begin{small}\begin{equation}
      NOP = \mbox{total new post-release problems opened during the month}
      \end{equation}\end{small}
      \end{itemize}
   \item[-] What is the total number of open problems at the end of 
            the month?
      \begin{itemize}
      \item[$\star$] Total Open Problems (TOP)
      \begin{small}\begin{equation}
      TOP = \mbox{total new post-release problems that remain open 
                  at the end of the month}
      \end{equation}\end{small}
      \end{itemize}
   \item[-] What is the mean age of open problems at the end of the month?
      \begin{itemize}
      \item[$\star$] Mean Age of Open Problems (AOP)
      \begin{small}\begin{equation}
      AOP = \frac{\mbox{total open time of post-release problems remaining 
                    open at the end of the month}}
             {\mbox{number of open post-release problems remaining 
                    open at the end of the month}}
      \end{equation}\end{small}
      \end{itemize}
   \item[-] What is the mean age of the problems that were closed 
            during the month?
      \begin{itemize}
      \item[$\star$] Mean Age of Closed Problems (ACP)
      \begin{small}\begin{equation}
      ACP = \frac{\mbox{total time post-release problems closed 
                    within the month were open}}
             {\mbox{number of open post-release problems closed 
                    within the month}}
      \end{equation}\end{small}
      \end{itemize}
   \end{itemize}
\clearpage
\item[]
\item To reduce the cost of nonconformance
   \begin{itemize}
   \item[-] What was the cost to fix postrelease problems during the month?
      \begin{itemize}
      \item[$\star$] Cost of Fixing Problems (CFP)
      \begin{small}\begin{equation}
      CFP = \mbox{dollar cost associated with fixing post-release problems 
              with the month}
      \end{equation}\end{small}
      \end{itemize}
   \end{itemize}
\item To increase software productivity
   \begin{itemize}
   \item[-] What was the productivity of software development projects?
      \begin{itemize}
      \item[$\star$] Software Productivity Total (SPT)
      \begin{small}\begin{equation}
      SPT = \frac{\mbox{assembly-equivalent total source size}}
                 {\mbox{software development effort}}
      \end{equation}\end{small}
      \item[$\star$] Software Productivity Delta (SPD)
      \begin{small}\begin{equation}
      SPD = \frac{\mbox{assembly-equivalent delta source size}}
                 {\mbox{software development effort}}
      \end{equation}\end{small}
      \end{itemize}
   \end{itemize}
\end{itemize}

Hewlett-Packard\\
\indent
(under preparation)\\


IBM Rochester\\
\indent
(under preparation)\\

\end{large}


%========== New Chapter ==========
\clearpage
\begin{Huge}
\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
{\bf Chapter 4 \\
\hspace*{1in}\\
\hspace*{1in}A Model of Software Reliability\\
\hspace*{1in}\\
\hspace*{2in} --- The Rayleigh Model
}
\end{Huge}

\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
Contents:
\begin{itemize}
\item[4.1.] Software Reliability Model
\item[4.2.] Static and Dynamic Model
\item[4.3.] Rayleigh Model
   \begin{itemize}
   \item[4.3.1.] Basic Two Assumptions
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Software Reliability Model}\\

\begin{itemize}
\item {\it Software Reliability Models} are used to estimate
   \begin{itemize}
   \item[-] the reliability
   \item[-] the number of  latent defects
   \end{itemize}
\item[] of a software.\\
\item They are important
   \begin{itemize}
   \item[-] as an objective statement of the quality; and
   \item[-] for a resource planning.\\
   \end{itemize}
\item Two categories:
   \begin{itemize}
   \item[-] {\it static models}
   \item[-] {\it dynamic models}
   \end{itemize}
\item[] exist.   
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Static Model and Dynamic Model}\\

\begin{itemize}
\item Static Model:
   \begin{itemize}
   \item[-] It uses attributes of the project or program modules 
            to estimate the number of defects.\\
   \item[-] It has the form:
   \begin{displaymath}
   y = f(x_1, x_2, \cdots, x_n) + e
   \end{displaymath}
   \item[] where $x_1, x_2, \cdots, x_n$ are attributes of 
           the product (e.g., size, complexity, $\cdots$) 
           the project (e.g., number/skill of programmers, $\cdots$) 
           or the process (e.g., period cost, $\cdots$) 
           during the development, $y$
           is defect rate or the number of defects, 
           and $e$ is an error term.\\
   \item[-]Static models are static in the sense that contribution of 
           attributes are estimated based on previous projects.\\
   \end{itemize}
\item[]
\item Dynamic Model:
   \begin{itemize}
   \item[-] It uses current development defect patterns to estimate 
            end-product reliability.
   \item[]{\large (to be continued)}
\clearpage
   \item[]
   \item[-] It has two categories:
      \begin{itemize}
      \item[$\cdot$] those that model entire development process
      \item[$\cdot$] those that model the back-end formal testing phase,
      \end{itemize}
   \item[] the representative of which is:
      \begin{itemize}
      \item[$\cdot$] Rayleigh Model
      \item[$\cdot$] Exponential Model
      \end{itemize}
   \item[] respectively.\\
   \item[-] The denominator is always a function of time.\\
   \end{itemize}
\item Which model for what?
   \begin{itemize}
   \item[-] Dynamic model gives better estimation of the global 
            reliability, in general; but\\
   \item[-] static model is better for {\it hypothesis testing} 
            (to show, for example, that certain project attributes are 
             related to better quality) rather than for reliability; or\\
   \item[-] when the unit is more granular like program module level 
            the static model can be more powerful to provide clues to 
            the software engineers on how to improve the quality of 
            their design and implementation. 
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf The Rayleigh Model}\\

\noindent
$\bullet$ The Weibull distribution: \\
\begin{equation}\label{eq:weibull}
f(t) = (m/t)(t/c)^m \exp \Biggl( - (t/c)^m \Biggr) \\
\end{equation}
\begin{itemize}
\item[] where
   \begin{itemize}
   \item[$\cdot$] $m$ is shape parameter; 
   \item[$\cdot$] $c$ is scale parameter;
   \item[$\cdot$] $t$ is time,
   \end{itemize}
\item[] has long been used for {\it reliability analysis}, such as
   \begin{itemize}
   \item[$\cdot$] lifetime of ball bearings;
   \item[$\cdot$] electron-tube failures;
   \item[$\cdot$] overflow incidence of rivers.\\
   \end{itemize}
\item[-] Tail approaches zero asymptotically, but never reaches it.\\
\item[-] When applied to software $f(t)$ might represent
   \begin{itemize}
   \item[$\cdot$] defect density over time;
   \item[$\cdot$] defect arrival pattern,
   \end{itemize}
\item[] and its cumulative density function (cdf) form:
\begin{equation}
%F(t) = 1 - \exp \Biggr( -(\frac{t}{c})^m \Biggl)
F(t) = 1 - \exp \Biggr( -(t/c)^m \Biggl)
\end{equation}
\item[] means
   \begin{itemize}
   \item[$\cdot$] cumulative defect arrival pattern\\
   \end{itemize}
\item[-] To model software reliability $m=1$ or 2, usually. \\
\end{itemize}

\begin{figure}[h]
 \begin{center}
   \epsfile{file=rayleigh.eps,width=10cm,height=10cm}
   \caption {An example of Rayleigh pdf and cdf when $m=2$.}
 \end{center}
\end{figure}

\noindent
$\bullet$ The Rayleigh model is a special case when $m=2$:\\

\begin{equation}
f(t) = (2/t)(t/c)^2 \exp\Biggr( -(t/c)^2 \Biggl) \\
\end{equation}

\begin{itemize}
\item[]
\item[-] where $c$ determines the time $t_m$ at which the curve reaches 
         its peak the curve\\
\begin{equation}
t_m = c/\sqrt {2}\\
\end{equation}
\item[]
\item[-] The area below the curve up to $t_m$ is 39.35\% of 
         the total area.\\
\item[-] In actual case when $K$ is the total number of defects

\begin{equation}\label{eq:actual-rayleigh}
f(t) = K (2/t)(t/c)^2 \exp\Biggr( -(t/c)^2 \Biggl) \\
\end{equation}
\end{itemize}

%%\vfill
%%\begin{itemize}
%%\item[]{\bf Exercize 1.} 
%%   \begin{itemize}
%%   \item[] {\it Draw the graph of the equation (\ref{eq:weibull}) 
%%                assuming that $c=1$ and $m=1$ and 2.}
%%   \end{itemize}
%%\end{itemize}

\hspace*{1in}\\
\hspace*{1in}\\
$\bullet$ Examples of applications to software:\\

\begin{itemize}
\item[-] Software projects follow a pattern described by the Rayleigh 
         density curve (Norden, 1963). \\
\item[-] The defect removal pattern follows the Rayleigh pattern 
         (Gaffney, 1984). \\
\item[-] The month-by-month error pattern of software projects resembled 
         a Rayleigh-like curve (Trachtenberg, 1982). \\
\clearpage
\item[-] Defect counts at six phases followed a Rayleigh curve \\
         (Gaffney, 1984). \\
\end{itemize}

\begin{figure}[h]
 \begin{center}
   \epsfile{file=r-model.eps,width=10cm,height=10cm}
   \caption{An example of application of the model to software development.}
 \end{center}
\end{figure}

\hspace*{1in}\\
\noindent
Given the defect removal pattern, the goal is to estimate the defect rate 
when the product is shipped, which is called here {\it general-availability 
phase (GA)}.

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Basic Two Assumptions of the Rayleigh Model}
\footnote{Both assumptions give us the
{\it Do-it-right-the-first-time} principle, that is, if
errors are to be injected, they should be removed as early as possible.}\\

\begin{itemize}
\item[1.] Defect rate observed during the development is positively
correlated with the defect rate after release
\item[]\begin{center}$\Downarrow$\end{center}
\item[] The higher the curve (the more area under it) the higher 
        defect rate after release and vice versa, assuming the defect
        removal effectiveness remains unchanged. \\
\item[] Exercise 2.
   \begin{itemize}
   \item[] {\it Draw the two curves of the Equation 
                (\ref{eq:actual-rayleigh}) 
                with different $K$ values to show this assumption 1.
                Note that it is important to show the difference of 
                the tail of the two curves.}
   \end{itemize}
\item[]
\item[2.] Given the same error injection rate, if more defects are 
          discovered and removed earlier, 
          fewer will remain in later stages.\\
\item[] Exercise 3.
   \begin{itemize}
   \item[] {\it Draw two curves of the Equation 
                (\ref{eq:actual-rayleigh}) 
                the area under which are the same but peak at varying 
                locations.
                to show the curve that peaks earlier has smaller
                area at the tail.}
   \end{itemize}
\end{itemize}

%========== New Chapter ==========
\clearpage
\begin{Huge}
\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
{\bf Chapter 5 \\
\hspace*{1in}\\
\hspace*{1in} Reliability Growth Model\\
}
\end{Huge}

\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
Contents:
\begin{itemize}
\item[5.1.] Reliability Growth Model
   \begin{itemize}
   \item[5.1.1.] Exponential Model
   \item[5.1.2.] The Other Reliability Growth Models
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Reliability Growth Model} \\

\begin{itemize}
\item While {\it Rayleig Model} models defects during entire process,
      {\it Reliability Growth Models} are based on data from 
      formal testing phses.
\item Defect arrival or failure patterns during these test phases
      is a good indicator of reliability of the product after release.
\item when fails occur and defects are identified and fixed the product 
      becomes more stable and reliability {\bf grows} over time, which
      is the reason of the name of the model.
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf The Exponential Model} \\

\hspace{1in} --- Weibull model with $m=1$.\\

\noindent
$\bullet$ Its pdf form:\\

\begin{equation}
f(t)= \frac{1}{c} \exp \Biggr( - (t/c) \Biggl).
\end{equation}

\noindent
$\bullet$ cdf form:\\

\begin{equation}
F(t)= 1 -  \exp \Biggr( - (t/c) \Biggl).
\end{equation}

\hspace*{1in}\\
\noindent
$\bullet$ Graph of its pdf and cdf:\\
\begin{figure}[h]
 \begin{center}
   \epsfile{file=exponential-pdf.eps,width=10cm,height=10cm}
   \caption {Exponential pdf and cdf.}
 \end{center}
\end{figure}

\clearpage
\hspace*{1in}\\
\begin{itemize}
\item[-] $1/c$ is referred to as the {\it error detection rate},
         {\it instantaneous failure rate}, or {\it hazard rate}.\\
\item[-] Decline nonmonotonically to an asymptote,\\ (but never reach it.)\\
\item[-] The area under the curve is 1, i.e.,
\begin{equation}
\int_{-\infty}^{\infty} f(t)dt = 1.
\end{equation}
\item[]
\item[-] It has long been successfully used for the analysis of failure 
         data, such as:
   \begin{itemize}
   \item[$\cdot$] bank statement and ledger errors;
   \item[$\cdot$] payroll check errors;
   \item[$\cdot$] light bulb failure;
   \item[$\cdot$] automatic calculating machine failure;
   \item[$\cdot$] radar failure; and so on.\\
   \end{itemize}
\item[-] The importance of the exponential distribution in reliability 
         studies is analogous to the Gaussian pdf in Statistics.
\clearpage
\item[]
\item The model in Software Engineering:
   \begin{itemize}
   \item[-] Misra (1983) used the model to estimate the defect-arrival rates
            for Shuttle Ground System software of NASA.\\
   \item[-] Other examples:
      \begin{itemize}
      \item[$\cdot$] Defect-arrival data since the start of system test 
                     versus week when the development work was virtually 
                     complete.\\
      \item[$\cdot$] The pattern of defect arrivals during the test
           \begin{itemize}
           \item[---] indicative of the {\it latent-defect rate 
                     after release.}\\
           \end{itemize}
      \end{itemize}
   \item[-] When apply the Model:
      \begin{itemize}
      \item[$\cdot$] The more accurate the input data, 
                     the better the outcome.
      \item[$\cdot$] The more data points available,
                     the better the model fit to the data.
      \end{itemize}
   \end{itemize}
\clearpage
\item[]
\item The basic assumption:
   \begin{itemize}
   \item[] {\it The testing effort (e.g., person-hour) is homogeneous 
           throughout the testing phase.}\\
   \end{itemize}
\item[] Normalize if not!\\
\item[] \hspace{1in}$\Downarrow$\\
   \begin{itemize}
   \item[1.] Accumulate the total person-hours in testing for the entire 
             testing phase, and calculate the average number of 
             person-hours in testing per week $\Rightarrow$ $n$.\\
   \item[2.] Starting from the beginning of testing, calculate 
             the defect rates for each $n$ person-hour units.\\
   \item[3.] Allocate the defect rate observed for the first $n$ 
             person-hours of testing to the first week and so on.
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf The Other Reliability Growth Models}\footnote{Though
more than a hundred models have been proposed, they are not
so satisfactory, like
\begin{itemize}
\item[$\cdot$] the cost of gathering data is expensive;
\item[$\cdot$] some are not understandable;
\item[$\cdot$] some are not tested in practical environment;
\item[$\cdot$] some do not work when tested. 
\end{itemize}
} \\

\begin{itemize}
\item Two major classes (according to the dependent variables)\\
   \begin{itemize}
   \item[1.] The time between failures model:\\
      \begin{itemize}
      \item The parameters of the distribution of time between 
              consecutive failures should be related to the number 
              of latent defects remaining.\\
      \item The goal is to estimate meantime to next failure.\\
      \end{itemize}
   \item[2.] The fault count model:\\
      \begin{itemize}
      \item The number of faults in a specified time interval\\
            (CPU time, hours, days, week, or month.)
      \end{itemize}
   \end{itemize}
\clearpage
\item Examples of each classes:\\
   \begin{itemize}
   \item[1.] Time between failure models:
      \begin{itemize}
      \item[]
         \begin{itemize}
         \item[]Estimation of the time between $(i-1)$ and $i$-th
                failure, assuming $N$ defects at the beginning of testing.\\
         \end{itemize}
      \item Jelinski-Moranda (J-M) Model (1972):
\begin{equation}
\mbox{\large \{time between failure\}} = K\{N - (i-1)\}
\end{equation}      
      \item[]
      \item[-] under the assumptions:
         \begin{itemize}
         \item failures occur purely at random
         \item all faults contribute equally to cause a failure during 
               the test
         \item fix time is negligible
         \item the fix is perfect.\\
         \end{itemize}
      \item {\it Imperfect Debugging Model} 
            \hspace*{3in}(G\"{o}el-Okumoto, 1978)
\begin{equation}
\mbox{\large \{time between failure\}} = \{N - p(i-1)\}\lambda
\end{equation}
      \item[]
      \item[-] The assumption here is:
         \begin{itemize}
         \item fix is not always perfect but new defects may be injected
               when fixing. 
         \end{itemize}
      \item[]
      \item[] where $p$ is the probability of imperfect debugging and  
              $\lambda$ is the failure rate per fault.
      \end{itemize}
   \clearpage
   \item[]
   \item[2.] The fault count model:\\
      \begin{itemize}
      \item {\it Non-homogeneous Poisson Process Model}\footnote{Poisson
                 pdf gives a probability that an event whose average 
                 occurence is $lambda$ per unit time occurs $k$ times
                 during the unit time, being given as:
\begin{equation}
prob(X=k) = \exp(-\lambda)\frac{\lambda^l}{k!}
\end{equation}
}\\
\hspace*{3in}(G\"{o}el-Okumoto, 1979)
      \item[] This model is:\\
         \begin{itemize}
         \item To model the number of failures observed in given testing 
                 intervals, i.e.,
         \item the cumulative number of failures observed at time $t$,
               $N(t)$, can be modeled as a Non-homogeneous Poisson 
                 Process, i.e.,
         \item the time dependent failure rate follows a exponential 
                 distribution. \\
         \end{itemize}
     \item[] The model is given as:
\begin{displaymath}
prob \{ N(t)=y \} = \frac{\{m(t)\}^y}{y!}\exp\{-m(t)\}
\end{displaymath}
\begin{equation}
      \hspace*{4in} y=0,1,2, \cdots
\end{equation}
      \item[]
      \item[]where\\
\begin{equation}
m(t) = a (1- \exp(-bt))
\end{equation}
      \item[]
      \item[]\hfill (to be continued)
\clearpage
      \item[] is expected number of failures observed by time $t$
              in which $a$ is the expected number of failures to be 
              observed eventually, $b$ is the fault detection rate 
              per fault, and \\ 
\begin{equation}
\lambda (t) \equiv m ^{\prime}(t) = ab \exp(-bt)
\end{equation}
      \item[]
      \item[] means failure density.\\
      \item[]
      \item[] It is easy to check $m(t)$ and $\lambda(t)$ is 
              a {\it pdf} and {\it cdf} form of the exponential model,
              respectively,
              with $a = K$ and $b = \lambda$.\\
      \item[] G\"{o}el extend this into more general form \\
      \item[] \begin{center} $\Downarrow$ \end{center}
      \item[]
      \item[] Generalized Non-homogeneous Poisson Process Model\\
\begin{equation}
m(t) = a (1- \exp(-bt^c))\\
\end{equation}

\begin{equation}
\lambda (t) = abc \exp(-bt^c)t^{c-1}
\end{equation}
      \item[]
      \item[] where $a$ is the expected number of faults to be eventually 
              detected, and $b$ and $c$ are constants that reflect
              the quality of testing.
      \end{itemize}
   \item[] (2. The fault count model --- continued:)\\
      \begin{itemize}
      \item {\it Logarithmic Poisson Execution Time Model}\\
            \hspace*{3in}(Musa-Okumoto, 1983)
      \item[] The model denotes:\\
         \begin{itemize}
         \item observed number of failures by a certain time $\tau$;\\
         \item also a Non-homogeneous Poisson Process, but\\
         \item $m(t)$ has a different form.\\
\begin{equation}
m(t) = \frac{1}{\theta} \ln (\lambda \theta + 1)
\end{equation}
         \item[]
         \item[]where $\lambda$ is the initial failure, and 
                $\theta$ is rate of reduction in the normalized failure,
                i.e., intensity per failure.\\
         \item Under the assumption that fixes found later have a smaller
               effect on the software's reliability than the earlier ones.\\
         \item The logarithmic Model is superior when some functions are 
               executed much more frequently than others.
         \end{itemize}
      \end{itemize}
\clearpage
   \item[] (2. The fault count model --- continued:)\\
      \begin{itemize}
      \item {\it Delayed S Model} (Yamada et al., 1983)\\
      \item[] The assumption:\\
         \begin{itemize}
         \item Testing process consists of not only a defect detection 
               but a defect isolation.
         \item Because of the time needed for failure analysis, delay can 
               occur between the time of the 1st failure observation and 
               a time of reporting.\\
         \end{itemize}
      \item[] Here, $m(t)$ is defined as:\\
\begin{equation}
m(t) = k \Biggr( 1-(1+\lambda t)\exp (- \lambda t) \Biggl)
\end{equation}
      \item[]
      \item[] where $\lambda$ is error detection rate, and $k$ is
              the total number of defects.\\
      \item[]
      \item {\it Inflection S Model} (Ohba, 1984)\\
      \item[] This models:\\
         \begin{itemize}
         \item[$\cdot$] a software failure detection phenomenon with a 
                        mutual dependence of detected defects.
         \item[] \hfill (to be continued)
\clearpage
         \item[]
         \item[-] The more failures we detect, the more undetected 
                  failures become detectable.\\
         \end{itemize}
      \item[] Here, $m(t)$ is defined as:\\
\begin{equation}
m(t) = k \frac{1-\exp (- \lambda t)}{1 + i \exp (-\lambda t)}
\end{equation}
      \item[]
      \item[] where $\lambda$ is error detection rate, $k$ is
              the total number of defects, and $i$ is inflection factor.
      \end{itemize}
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
\begin{small}
$\Box$ {\bf Homework Assignment}\\
\begin{itemize}
\item For the three models we have learned so far, that is,
   \begin{itemize}
   \item Exponential Model,
   \item Delayed S Model,
   \item Inflection S Model,
   \end{itemize}
\item[]compare the degree of fitness of them to 
       a set of real data by the following way.
   \begin{itemize}
   \item[$\star$] For each of the three models:
      \begin{itemize}
      \item [(1)] Fill the blanks of the table below. In calculating
                  the column (C), you must adjust each of the parameters 
                  in the Model such that the model best fits the data
                  of the column (B).\footnote{The judgment of the
                  fitness depends on your eyes and therefore this column 
                  should be unique one. Do not use a set of results by
                  others. I won't believe any coincidence.}

\begin{table}[hbtp]
\begin{center}
\caption{A Comparison of a Model to a Cumulative Data of 
         a Weekly Defect Arrivals}
\begin{tabular}{c c c c c c c} \\ \hline \hline
             & observed & observed   & model      & normalized 
& normalized & absolute difference \\ 
week           & defect   & defect     & defect     & data
& model      & between \\
             & ratio    & cumulative & cumulative & cumulative
& cumulative & data and model \\ 
($x$)        & (A)        & (B)        & (C)        & $F^{*}(x)$ & 
$F(x)$       & $\vert F^{*}(x)-F(x) \vert $ \\ \hline
 1 & .353 & .353 & &       &       &   \\
 2 & .436 & .789 & &       &       &   \\
 3 & .415 &      & &       &       &   \\
 4 & .351 &      & &       &       &   \\
 5 & .380 &      & &       &       &   \\
 6 & .366 &      & &       &       &   \\
 7 & .308 &      & &       &       &   \\
 8 & .254 &      & &       &       &   \\
 9 & .192 &      & &       &       &   \\
10 & .219 &      & &       &       &   \\
11 & .202 &      & &       &       &   \\
12 & .180 &      & &       &       &   \\
13 & .182 &      & &       &       &   \\
14 & .110 &      & &       &       &   \\
15 & .155 &      & &       &       &   \\
16 & .145 &      & &       &       &   \\
17 & .221 &      & &       &       &   \\
18 & .095 &      & &       &       &   \\
19 & .140 &      & &       &       &   \\
20 & .126 &      & & 1.000 & 1.000 & 0 \\ \hline \hline
\end{tabular} 
\end{center}
\end{table}

      \item[(2)] Draw the histogram of column(B) as well as the 
                 curve by the model\footnote{You should draw the graph
                 of the equation 
                 (not by giving the points from column (C).}
                 on one coordinate.
      \item[(3)] Judge the fitness by the {\it Kolmogorov-Smirnov}
                 goodness of fit test, that is,
\begin{equation}
\mbox{if} \hspace{.2in} \max \vert F^{*}(x)-F(x) \vert < .294
\hspace{.2in} \mbox{then model fits the data.}
\end{equation}
      \item[]The answer hopefully should be positive.\footnote{This
means the answer is something like ``Yes, the model fits the data
because $\max \vert F^{*}(x)-F(x) \vert  = .262 < .294$.''}
      \end{itemize}
   \end{itemize}
\end{itemize}
\end{small}

%========== New Chapter ==========
\clearpage
\begin{Huge}
\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
{\bf Chapter 6 \\
\hspace*{1in}\\
\hspace*{1in} Complexity vs Quality\\
}
\end{Huge}

\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
Contents:
\begin{itemize}
\item[6.1.] Complexity vs Quality
   \begin{itemize}
   \item[6.1.1.] The Simplest Model
   \item[6.1.2.] Structure Metrics and Model
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Complexity vs Quality} \\

\noindent
This chapter studies relationships between metrics about the design and 
code implementation and the software quality.

\begin{itemize}
\item The simplest Model:
   \begin{itemize}
   \item Assuming LOC reflects complexity.\\
      \begin{itemize}
      \item[$\star$] The larger the size, the more defects?
      \item[]\begin{center}$\Downarrow$\\
             No! \end{center}
         \begin{itemize}
         \item[]
         \item[$\cdot$] Defects were mostly in the small module 
                        (Basili et al, 1984).\\
         \item[$\cdot$] The larger module contains small defect rate,
                        probably because of interface error which are more 
                        or less constant regardless of size
                        (Shen et al., 1985).
         \item[]\begin{center}$\downarrow$ \end{center}
         \item[]        The smaller modules has higher error density 
                        due to small denominator.\\
         \item[$\cdot$] Defect density decreases with size and then up 
                        again at the tail (Withrow, 1990).
         \item[]\begin{center}$\downarrow$ \end{center}
         \item[]        There might be an optimal program size.
         \end{itemize}
      \end{itemize}
   \end{itemize}
\clearpage
\item Structural Metrix and Models\\
\item[] We use:
   \begin{itemize}
   \item fan-in: \# of modules that call a given module.
   \item fan-out: \# of modules that are called  by a given module.\\
   \item[$\star$] The number of modules with a large fan-out have a 
                  positive correlation with the number of defect, while 
                  the number of modules with a large fan-in have a negative
                  or insignificant correlation?\\
   \item[(1)] Henry et al.'s Structure Complexity (1981):
\begin{small}\begin{equation}
\mbox{\{structure complexity\}} = (\mbox{\{fan-in\}} \times 
                                   \mbox{\{fan-out}\})^2
\end{equation}\end{small}
   \item[]
   \item[(2)] Card and Glass's System Complexity (1990)
\begin{small}\begin{equation}
\mbox{\{system complexity\}} = 
   (\mbox{\{structural intermodule complexity\}} +
   (\mbox{\{data intramodule complexity\}}
\end{equation}\end{small}
   \item[]
   \item[]Here, each term of the RHS is defined as:
\begin{equation}
\mbox{\small structural intermodule complexity} = \frac{\sum D(i)}{n}
\end{equation}
   \item[]and
\begin{equation}
\mbox{\small data intramodule complexity} = \frac{\sum f^2(i)}{n}
\end{equation}
\clearpage
   \item[]
   \item[]where 
      \begin{itemize}
      \item[$\cdot$] $D(i)$ is data complexity of module $i$ defined as:
\begin{equation}
D(i) = \frac{V(i)}{f(i)+1}
\end{equation}
      \item[$\cdot$] $V(i)$ is I/O variables in module $i$, 
      \item[$\cdot$] $f(i)$ is fan-out of module $i$,
      \item[$\cdot$] $n$ is the number of (new) modules in system.\\
      \end{itemize}
   \item[]
   \item[] Then model says:
      \begin{itemize}
      \item[$\cdot$] Fan-in is not an important complexity.
      \item[$\cdot$] Complexity increases as the square of fan-out.
      \item[$\cdot$] The more I/O variables in a modules the more 
                     functionality is needed, hence the higher internal 
                     complexity.
      \item[$\cdot$] ``more fan-out'' means that functionality is deferred 
                     to modules or lower levels, hence the less internal 
                     complexity.\\
      \end{itemize}
   \item[]
   \item[] And finally the Model give an equation by regression formula:
\begin{small}\begin{equation}
\mbox{\{error rate\}} = - 5.2 + 
                          0.4 \cdot \mbox{\{system complexity\}}
\end{equation}\end{small}
   \end{itemize}
\end{itemize}

%========== New Chapter ==========
\clearpage
\begin{Huge}
\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
{\bf Chapter 7 \\
\hspace*{1in}\\
\hspace*{1in}Customer's Satisfaction\\
\hspace*{1in}\\
\hspace*{1.5in} --- Measurement and Analysis
}
\end{Huge}

\hspace*{1in}\\
\hspace*{1in}\\
\hspace*{1in}\\
\noindent
Contents:
\begin{itemize}
\item[7.1.] Product Quality + Customer Satisfaction = Total Quality
\item[7.2.] How can We Measure the Customer Satisfaction?
\item[7.3.] Specific Attributes vs Overall Satisfaction
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Product Quality + Customer Satisfaction\\
        \hspace*{4.5in}\\
        \hspace*{4.5in} = Total Quality} \\


\begin{itemize}
\item Studies show:\\
   \begin{itemize}
   \item[$\cdot$]it is five times more costly to recruit a new customer; \\
   \item[$\cdot$]dissatisfied customers tell 7 to 20 people about their 
                 experience, while satisfied customers tell only 3 to 5.
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf How can We Measure the Customer Satisfaction?}\\
\begin{itemize}
\item By interview, phone, or mail? \\
\item The five-point satisfaction scale:
   \begin{itemize}
      \item[(1)] very satisfied
      \item[(2)] satisfied
      \item[(3)] neutral
      \item[(4)] dissatisfied
      \item[(5)] very dissatisfied\\
   \end{itemize}
\item To obtain representative samples:
   \begin{itemize}
   \item[-] Simple Random Sampling:
      \begin{itemize}
      \item Every possible sample of size $n$ has the same chance of 
            being selected, say, by using random numbers.\\
      \end{itemize}
   \item[-] Systematic Sampling:
      \begin{itemize}
      \item Simply goes down a list taking every $k$-th individual where\\
      \begin{large}\begin{equation}
                   k = \frac{\mbox{\{total number of items in the list\}}}
                            {\mbox{\{number to be drawn\}}}
      \end{equation}\end{large}
      \item[]\hfill (to continue)
      \end{itemize}
\clearpage
   \item[]
   \item[-]Stratified Sampling
      \begin{itemize}
      \item First classify individuals into non-overlapping groups 
            {\it (strata)}, and then select simple random sample from each 
            {\it} stratum \footnote{E.g., 
                                    network-users and stand-alone-usrs.}.
      \item It can be designed to yield greater accuracy for the same cost 
            or for the same accuracy with less cost.
      \item Proportionnal vs Dispropotional:
         \begin{itemize}
         \item The former being the sampling frachions within each stratum 
               is equal, while the latter being different when compose 
               different subpopulations that differ in size.
         \end{itemize}
      \item To yield the maximum efficiency:
         \begin{itemize}
         \item Decide sampling fraction for each stratum proportional to
               the standard deviation within the stratum and inversely 
               proportional to the square root of the cost of each case 
               within the stratum.\\
         \item[]\hfill (to continue)
         \end{itemize}      
      \end{itemize}
   \end{itemize}
\clearpage
\item[]
\item How large is a sample sufficient?
   \begin{itemize}
   \item[-] It depends on sampling method, the level of confidence and 
            margin of error.
   \item[-] A proposition for the simple random sampling, 
   \begin{equation}
   n = \frac{NZ^2p(1-p)}{NB^2 + z^2 p(1-p)}
   \end{equation}
   \item[] where $N$ is population, $p$ is estimated satisfaction 
           level\footnote{when no estimate of satisfaction available use
           $p=0.5$ which require the largest sample because $pq$ takes 
           the largest value.}, $B$ is margin of error, and
   \begin{eqnarray}
   \nonumber
   z &=& 1.28 \hspace{.3in} \mbox{for 80\% confidence level}\\
   \nonumber
   z &=& 1.45 \hspace{.3in} \mbox{for 85\% confidence level}\\
   \nonumber
   z &=& 1.65 \hspace{.3in} \mbox{for 90\% confidence level}\\
   \nonumber
   z &=& 1.96 \hspace{.3in} \mbox{for 95\% confidence level}
   \nonumber
   \end{eqnarray}
   \item[-] The required sample size decreases as customer satisfaction 
            increases, i.e., the larger the $p$ the smaller its variance 
            $p(1-p)$.
   \end{itemize}
\end{itemize}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf Specific Attributes vs Overall Satisfaction}\\

\noindent
CUPRIMDA\footnote{Each capital is abbreviation from
Capability, Usability, Performance, Reliability, Installability, 
Maintainability, Documentation, and Availability, respectably. 
These eight attributes are often used in software engineering.} 
vs Overall Satisfaction:

\begin{itemize}
\item A Survey\footnote{Data are from IBM Rochester whose sample size is
                        3,658.}
   \begin{itemize}
   \item[1.] Degree of Satisfaction
\begin{table}[hbtp]
\begin{center}
\begin{tabular}{c c c c c c c c}    \\ \hline \hline
Overall & U    & P    & R    & I    & M    & D    & A     \\ \hline
.855    & .841 & .911 & .938 & .753 & .829 & .733 & .945  \\ \hline\hline
\end{tabular} 
\caption{An example of Degree of Satisfaction}\label{table:satisfaction}
\end{center}
\end{table}
   \item[2.] Correlation 
\begin{table}[hbtp]
\begin{center}
\begin{tabular}{c c c c c c c c c}    \\ \hline \hline
  & Overall & U   & P   & R   & I   & M   & D   & A   \\ \hline
U & .61     &     &     &     &     &     &     &\\
P & .43     & .46 &     &     &     &     &     &\\
R & .63     & .56 & .42 &     &     &     &     &\\
I & .51     & .57 & .39 & .47 &     &     &     &\\
M & .40     & .39 & .31 & .40 & .38 &     &     &\\
D & .45     & .51 & .34 & .44 & .45 & .35 &     &\\
A & .38     & .39 & .52 & .46 & .32 & .28 & .31 &\\ \hline \hline 
\end{tabular} 
\end{center}
\end{table}
   \item[$\cdot$] Those who are happy with R, U, or I are happy 
         with the overall product.
   \item[$\cdot$] Those who are happy with R, I, or D are happy with U.
   \item[$\cdot$] Those who are happy with A are happy with P.
\clearpage
   \item[3.] Coefficient of Least Squares Regression\footnote{Coefficients
of the following equation
\begin{equation}\label{eq:regression}
y = a_0 + a_1 x_1 + a_2 x_2 + \cdots + + a_k x_k + \epsilon
\end{equation}
where $\epsilon$ is error term
are determined such that the total sum of the distance from data points 
to the hyper-line (\ref{eq:regression}) minimizes.}
\begin{table}[hbtp]
\begin{center}
\begin{tabular}{c c c c c c c}    \\ \hline \hline
R    & U    & I    & P    & M    & D    & A     \\ \hline
.391 & .247 & .091 & .070 & .067 & .056 & .022  \\ \hline\hline
\end{tabular} 
\end{center}
\end{table}
   \item[$\cdot$] Each of the $x_i$ is the expression of RUIPMDA 
                  expressed by the 5-point scale. The coefficients 
                  are re-arranged according to the order of magnitude 
                  and mean its importance.
\clearpage
   \item[]
   \item[4.] Logistic Regression\\
      \begin{itemize}
      \item[] $x$: dichotomous variable, i.e.,
         \begin{large}\begin{itemize}
         \item[$\cdot$]satisfied: \{very-satisfied\}+\{satisfied\} 
                                  $\Rightarrow$ 1
         \item[$\cdot$]dissatisfied: \{neutral\} + \{very-satisfied\} +
                                     \{satisfied\} 
                                      $\Rightarrow$ 0\\
         \end{itemize}\end{large}
      \item[] and\\
      \end{itemize}
\begin{large}\begin{equation}
\log \Biggr( \frac{\mbox{\{satisfied\}}}{\mbox{\{disstisfied\}}} \Biggl)
= b_0 + b_1 x_1 + b_2 x_2 + \cdots + b_k x_k + \epsilon
\end{equation}\end{large}
   \item[]
   \item[-]{\it Odds ratio} of satisfied and dissatisfied:
      \begin{itemize}
      \item[] How much more likely customers will be positive in overall 
              satisfaction if they were satisfied with specific UPRIMDA 
              vs if they were not.
\begin{table}[hbtp]
\begin{center}
\begin{tabular}{c c c c c c c c}    \\ \hline \hline 
           & R     & U    & A    & I    & M    & P    & D    \\ \hline
$b_k$      & 1.216 & .701 & .481 & .410 & .376 & .321 & .164 \\ 
Odds ratio & 11.4  & 4.1  & 2.6  & 2.3  & 2.1  & 1.9  & 1.4  \\ \hline\hline
\end{tabular} 
\caption{Logistic Regression Coefficient and Odds}\label{table:logistic}
\end{center}
\end{table}
      \item[]For example we can read this table that the overall 
             satisfaction occurs 11.4\footnote{The value greater than 1 
             has a positive effect.} times as often among those satisfied 
             than those dissatisfied.
      \end{itemize}
\clearpage
   \item[]
   \item[5.] The probability for customers being satisfied depending on
             whether or not they are satisfied with CPRIMDA:
\begin{table}[hbtp]
\begin{center}
\begin{tabular}{c c c c c c c c}    \\ \hline \hline
$p\{y=1 \vert x\}$           
           & U & P & R & I & M & D & A \\ \hline
.9632      & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ 
.9187      & 1 & 1 & 1 & 1 & 1 & 1 & 0 \\ 
.9552      & 1 & 1 & 1 & 1 & 1 & 0 & 1 \\ 
.9331      & 1 & 1 & 1 & 1 & 0 & 1 & 1 \\ 
.9287      & 1 & 1 & 1 & 0 & 1 & 1 & 1 \\ 
.7223      & 1 & 1 & 0 & 1 & 1 & 1 & 1 \\ 
.9397      & 1 & 0 & 1 & 1 & 1 & 1 & 1 \\
.8792      & 0 & 1 & 1 & 1 & 1 & 1 & 1 \\ \hline
.0189      & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
.0480      & 0 & 0 & 0 & 0 & 0 & 0 & 1 \\
.0260      & 0 & 0 & 0 & 0 & 0 & 1 & 0 \\
.0392      & 0 & 0 & 0 & 0 & 1 & 0 & 0 \\
.0132      & 0 & 0 & 0 & 1 & 0 & 0 & 0 \\
.1796      & 0 & 0 & 1 & 0 & 0 & 0 & 0 \\
.0353      & 0 & 1 & 0 & 0 & 0 & 0 & 0 \\ 
$\cdots$   &&&& $\cdots$ &&&           \\ \hline \hline
\end{tabular} 
\end{center}
\end{table}
\clearpage
   \item[]
   \item[6.] Regression coefficient vs Satisfaction\\
      \begin{itemize}
      \item[-] To determine priority of improvement:\\
         \begin{itemize}
         \item[(1)]Plot each CPRIMDA on the coordinate coefficient vs 
                   satisfaction level.\\
         \item[(2)]Pick up item by going from top to bottom.\\
         \item[(3)]If coefficients have the same level do it by going
                   left to right.\\
         \end{itemize}
      \end{itemize}
   \end{itemize}
\end{itemize}
\begin{figure}[h]
 \begin{center}
   \epsfile{file=improve-priority.eps,width=8.9cm,height=7cm}
 \end{center}
\caption{An example of Regression coefficient vs Satisfaction.
The data in Table \ref{table:satisfaction} and \ref{table:logistic}.
are used.}
\end{figure}

%---------------------------------------------------------------------
\clearpage
\noindent
$\Box$ {\bf How good is Good Enough?}\footnote{This title is from 
the book by S.H.Kan who cited Bebich (1992).}\\

\begin{itemize}
\item Though the long term goal should be of course 100 \%, 
      there might be better strategy, like:
   \begin{itemize}
   \item[-] Should we invest \$2,000,000 to improve satisfaction 
            from 85\% to 90 \%?
   \item[-] When our customer's satisfaction is 95 \%, should we invest 
            another million dollar or should we do it later?\\
   \end{itemize}
\item A model for prediction of future market share:
\begin{eqnarray}
\nonumber
A_{t+1} = A_t (1-x) + B_t y \frac{A_t}{A_t + C_t} 
                    + C_t z \frac{A_t}{A_t + B_t}\\
                    + G     \frac{A_t}{A_t + B_t + C_t}
\end{eqnarray}
\item[] where $A_t$($B_t$,$C_t$) is the number of A(B, C)'s customers, 
        at time $t$, $G$ is the number of new customers to market, and
        $x(y,z)$ is dissatisfaction level with A(B,C)'s products.  \\
\item[] $B_t$ and $C_t$ is also defined in the similar way.\\
\item[] An example of the plot of $A_t$, $B_t$ and $C_t$ with different
        satisfaction level is those in the next page.\\
\item[] Hence
   \begin{itemize}
   \item[]\begin{center} 
          {\it You must be better than your competitor!}
          \end{center}
   \end{itemize}
\end{itemize}
\clearpage
\begin{figure}[h]
 \begin{center}
   \epsfile{file=satisfaction-1.eps,width=10cm,height=7cm}
   \epsfile{file=satisfaction-2.eps,width=10cm,height=7cm}
 \end{center}
\caption{How the customer's satisfaction affects market share?}
\end{figure}

\hspace*{1in}\\
\vfill
\begin{center}
--------------------------------------------------------------------------\\
\end{center}
$\star$ This concludes the course of this semester.\\
$\star$ Thank you very much for your interest and attendance. \\
$\star$ See you.

\end{LARGE}
\end{document}



