\documentstyle [a4wide,11pt,epsf] {article} \parskip 0.05in \parindent 0in \topmargin -1.3cm \textheight 9.5in %\oddsidemargin -0.3in %\evensidemargin -0.3in %\marginparsep 0.2in \begin{document} %\parskip 0in \bibliographystyle{alpha} \begin{sloppypar} \title{Report on OLOS 1st summer school, 7-13 september 1998} \author{B.W. van Schooten} \date{version 3, 24 september 1998} \maketitle %\begin{abstract} %The OLOS group aims at interdisciplinary communication and research in the %area of safe and reliable human-computer systems. This includes the %disciplines of software, hardware, and human-computer interaction. %\end{abstract} \tableofcontents %\pagebreak %\part{Summary of lectures} \section{introduction} A classification into the actual disciplines HCI, hardware, software seems the most obvious at first. However, I found that only few lectures actually fit neatly into one of these categories, and several used the same methods in different disciplines. After a while of shuffling and re-shuffling, I decided on the following classification, based also on my own interests in methods: \begin{itemize} \item Methodological issues: this includes review of methodologies and standards. \item Schema and list (qualitative) methods: task analysis, component analysis, and trees and lists based on these. Some allow quantitative analysis when specified in enough detail. \item Statistical (quantitative) methods: fault frequencies and changes, MT* metrics, Markov models, Event Tree analysis. \end{itemize} Almost all lectures about methods do include some quantitative method, but each focuses on either the qualitative (how do I create a schema?) or quantitative (how do I calculate the values?) aspects only, making the distiction clear. A few lectures do not really fit into either of these and are mostly listed in `miscellaneous'. When a lecture explicitly focuses on software, hardware, or HCI, this is mentioned at the beginning of its summary. \part{Material} \section{Overview of the dependability handbook} Part of the material of some of the lectures can be found, in more detail, in the handbook. In this case, the appropriate reference is given in the `recommended literature' for that lecture. Part of the handbook is not covered in any lecture. \begin{tabular}{lll} 1.1 & Computer systems & only 1.1.4 and 1.1.5.1 covered \\ 1.2 & Human-computer interactions \\ 2.1 & Fault tolerance & covered \\ 2.2 & Fault removal \\ 2.3 & Fault forecasting & partially covered: \\ 2.3.1 & Ordinal evaluation & covered \\ 2.3.2 & Probabilistic evaluation w.r.t. physical faults & covered \\ 2.3.3 & Probabilistic evaluation w.r.t. design faults & covered \\ 2.3.4 & Probabilistic evaluation w.r.t. human errors & covered \\ 2.3.5 & Probabilistic evaluation w.r.t. malicious faults \\ 3 & Development of dependable systems & covered \\ A.1 & Dependability measures \\ A.2 & Human-machine task allocation \\ A.3 & Security models and policies \\ \end{tabular} \section{Miscellaneous lectures} \subsection{The man-machine relationship in railway transport safety} Lecturer: Gianpetro Monfardini. Discusses general (including organisational) aspects of safety. Some examples of the existing systems are given. \subsection{Introduction to cognitive psychology} Lecturer: Antonio Rizzo. Recommended literature: \cite{zhang94a} First, the symbolic and subsymbolic debate is reviewed. It is argued that the subsymbolic view is more useful as metaphor for understanding cognition. This is illustrated by the coin example: people only learn those features of a coin that are needed to separate them from other coins. This is similar to pattern recognition as done by a perceptron. Main characteristics: content addressable memory and context-sensitive rule selection. Errors are seen as the result of people's normal cognitive functions which happen to turn out badly due to adverse conditions. Classification into slip, lapse, mistake. The theory of distributed cognition (which is close to activity theory \cite{Kaptelinin:1995:ATB} according to Gerrit) maintains that cognitive activity is a combination of thinking (internal representation) and the use of artifacts (external representations). By choosing the right external representations, a task can be made easier and less prone to error. \subsection{Fault tolerance} Lecturer: Jean-claude Laprie. Recommended literature: handbook \cite{laprie98:dhp}, chapter 2.1, 1.1.4, 1.1.5.1, 2.2.4. Overview of fault tolerance. Def. of failure, error and fault, and their interrelation. Different kinds of faults and relative frequency. Dealing with faults. Review of fault-tolerant systems. Coverage factor. Examples of hardware redundancy. Fault tolerance testing, fault injection. Software fault tolerance: recovery points, design diversity, review. HCI fault tolerance: vicious cycle. \subsection{Future perspectives - remaining questions about safety} Lecturer: W. G\"orke General risk statistics. Fault-cause classification. Some individual accidents are reviewed. Some implications are shown for future safety-critical systems. \section{Standards and overall methodology} \subsection{Certification and licensing} Lecturer: Stuart Anderson Review of sources of evidence, reporting, kinds of criteria in certification. Review of therac-25 accidents. \subsection{Practical experiences in space systems} Lecturer: Uffe K. Mortensen Focuses on software. First reviews some specific space accidents. Then, reviews methodological issues in safety-critical software engineering. \subsection{Life cycle of systems and standards} lecturer: Mohamed Ka\^aniche and Corinne Mazet Recommended literature: handbook \cite{laprie98:dhp}, chapter 3. General review of safety and its relation to standards and certification. Review of many design cycle models. Review of verification and standards: (ergonomics) standards and human-centered development process; software verification process; IEC1508 (based on complete design cycle); hazard and risk level analysis. Safety can be checked using: verification, validation, assessment. Importance of independence of evaluating party. List of techniques. Dependability-Explicit Development Model: this model has the stages requirements, design, realisation, integration. A number of guidelines is given for each. \section{Qualitative methods (mostly)} \subsection{HCI design methods} Recommended literature: about GTA \cite{veer96:gta}, \cite{veer96:mcw}. Checklist \cite{roe88:crh}. \subsubsection{Interaction design} Lecturer: Gerrit C. van der Veer. Analysis: task model 1 (model of current situation), task model 2 (model of envisioned situation). Specification: User's Virtual Machine, User Action Notation, dialogue model classification. Evaluation: We may evaluate using the UVM, task model 2, formal specification, and a system setup with possibly varying degrees of realism. Various standard evaluation techniques are listed. The dialogue model classification sees natural language as a metaphor rather than a style. Typical natural language system must then be seen as switching between several styles (command language, question answering), but the switching itself could be seen as a interaction style. More problems: isn't direct manipulation just command language with a different modality? Aren't menu and form filling just question answering with a different modality? Doesn't VR belong in the `look and feel' section? \subsubsection{Methods of qualitative analysis of systems: task analysis of complex systems} Lecturer: Gerrit C. van der Veer. This lecture focuses on the analysis part of interaction design. The task analysis method used is GTA, which combines ethnography with more traditional task analysis methods like HTA (MAD), work flow analysis, object modeling. General procedure of GTA is explained, including task model 1 and 2, and a modeling tool. \subsection{Overview of human factors in safety critical systems + case study} Lecturer: David Embry Review of human failure data. Incident investigation using STEP (Sequentially Timed Events Plotting), based on general `incident model' based on the sequence failure-no defences-no recovery. A STEP diagram represents the causes which have lead to an incident in a cause-effect diagram, starting from the incident and working backwards. A BackStep diagram also allows working forwards from an unwanted situation (ex. alarm didn't go off) next to working backwards from incident. Causal tree analysis: one selects critical events from the STEP diagram, and draws causal trees for those. A tree represents all possible failures leading to an incident. STEP example. Description of: Hierarchical Task Analysis (HTA); Predictive Human Error Analysis (PHEA) based on HTA diagram. Examples. Case study: emergency response system in offshore oil rig. \subsection{Methods for dependability analyses of systems} Lecturer: Gerald Sonneck. Recommended literature: handbook \cite{laprie98:dhp}, chapter 2.3.1. The failure analysis methods and notations include human, software, hardware parts. A review of dependability issues and terminology. Analysis methods: Hazard Analysis/Hazard and Operability Analysis (HAZOP); Failure Mode and Effects Analysis (FMEA); Fault Tree Analysis (FTA); Event Tree Analysis (ETA). FTA and ETA may include calculation of probability figures (=quantitative). \subsubsection{OLOS case study: chemical plant} Lecturer: Gerald Sonneck. The case study is an example of the aforementioned methods. \section{Quantitative methods} Discussion of software risks \cite{littlewood92:rs}. Comparative research \cite{Fenton:1994:SSC}. \subsection{An introduction to quantitative analysis} Lecturer: Bruno Ciciani The examples focus mostly on hardware systems with unreliable components, but some of the mathematics (general statistics and Markov models) are useable for any kind of quantitative analysis. Statistics, examples. Server reliability. Markov processes. Reliability metrics (failure rate, MTTF, MTTR, fault coverage). Examples of Markov-based reliability analysis. Markov models with reward index, examples. Satellite system example. \subsection{Statistical software reliability} Focuses on software, but it is argued that such models may also be used for design reliability in complex hardware. According to Alberto Pasquini (+literature?), such models could also be applied to human reliability, but only when averaged over several people (what about human differences?). \subsubsection{Software reliability growth} Lecturer: Bev Littlewood. Recommended literature: handbook \cite{laprie98:dhp}, chapter 2.3.3. Discusses statistical prediction methods for predicting software reliability growth/change. Main data: time between failures. Best prediction method: u-calibration. \subsubsection{Software reliability in practice + case study} Lecturer: Karama Kanoun. Discusses methodology and some practical guidelines for using statistical analysis of software reliability using failure data. Dependability measures, trend analyses. Case study: software switching system. \subsection{Tools and techniques in human reliability quantification} Lecturer?? Recommended literature: handbook \cite{laprie98:dhp}, chapter 2.3.4. Human error data for basic tasks can be obtained from databases, literature or expert judgement (either absolute error or based on relative error). Error data can be combined to form an estimate of the error of a specific task. Hierarchical (event tree) or linear (sum of weighted errors) error model. Hierarchical: THERP. Linear: HEART, SLIM (single level/multilevel), IDEX. Some models also give error cause/means of recovery classifications. \subsection{Dependability analysis and evaluation} Lecturer: Karama Kanoun. Recommended literature: handbook \cite{laprie98:dhp}, chapter 2.3.2. General overview of dependability: terminology, statistical measures (availability, maintainability, MTTR, restoration rate). Probabilistic evaluation methods: Reliability block diagrams; fault trees; Markov chains including homogeneous ones. Review of methods with examples. \section{Formal logical methods} \cite{gurr94:sfr}: applicable to software/hardware design Lecturer: Stuart Anderson Overview of formal methods. Example using state charts. \part{Concepts} \section{Inventory of tools} After classification of the lectures, I found there is a large overlap in usage of some the terms and notations. A list of all notations together with their meaning and possible uses. \begin{tabular}{p{0.6in}p{1.8in}p{3.3in}} \hline HTA, MAD & Hierarchical Task Analysis & Hierarchical task notation by splitting a task into a set of subtasks (executed according to a given plan) recursively. \\ \hline UAN & User Action Notation & \\ \hline & Object modeling & \\ \hline & Workflow analysis & \\ \hline STEP, BackStep & Sequentially Timed Events Plotting & Plot event boxes on a timeline (horizontal axis), with cause-effect arrows between them. One works backwards from a major event (like a disaster) \\ \hline & Causal tree analysis & \\ \hline PHEA & Predictive Human Error Analysis & \\ \hline HAZ(OP) & HAZard (and OPerability) analysis & \\ \hline FMEA & Failure Mode and Effect Analysis & \\ \hline \end{tabular} Quantifiable notations: \begin{tabular}{p{0.6in}p{1.6in}p{3.4in}} \hline ETA, THERP & Event Tree Analysis, Technique for Human Error Rate Prediction & Events are plotted on a timeline, with the completion of a desired action at the end. A tree is plotted from left to right, which branches at every event: the upper branch means the event succeeds, the lower means it failed. Quantification requires success probabilities for each event. \\ \hline FTA & Fault Tree Analysis & A tree with nodes representing events, and an AND or OR gate between each parent and its children to denote failure condition. Quantification requires fault probabilities at each leaf. \\ \hline & Reliability Block Diagrams & \\ \hline HEART, SLIM, IDEX& & 2- or more-level trees; the nodes denote Probability Influencing Factors. A weight should be supplied for each PIF, and a rating (good...bad) for each leaf. The result is an index indicating (relative) success or failure probability. \\ \hline & Markov models & \\ \hline \end{tabular} %\section{Inventory of theories about human-machine systems} % %Distributed representations: choosing representations to %reduce error. % %User's Virtual Machine % % \bibliography{inf-2061-schooten} \end{sloppypar} \end{document}