--------------------------------Slide 1------------------------------
Contributions of a Social Science Based Evaluation for Adaptive Design
Projects

presented by Stuart Nelson

Focus: The focus of this paper is outlining a method for  

evaluating designs for adaptive systems


Areas Discussed:

I. Needs for Adaptation

	A. Determining the Potential for Adaptation

II. Objectives of Adaptive Designs

III. Conflicts of Adaptations

IV. Approach to Adaptive Design

V. Evaluation of Design

VI. Methods of Evaluation

VII. The Evaluation Process

-----------------------------Slide 2-------------------------------------
I. Needs for Adaptation

1. Computers have become more pervasive

2. Basic tasks undergo constant redefinition - adaptation

    could aid in task completion if tasks undergo 
    redefinition

3. End Users are different




A. Determining the Potential for Adaptation

1. Analyze the users work style

2. Analyze the users pattern of computer system usage

3. Analyze the users learning approach

4. Analyze the degree of autonomy the user has when 
    using  the system

------------------------------Slide 3---------------------------
II. Objectives of Adaptivity

1. Technical Operability- adaptations should follow a uniform
standard. They should not effect the overall performance of the
system.

2. Task Oriented Functionality - adaptations must lead to improved
system usability.

3. Organizational Impact - adaptations must blend into the
technological and informational infrastructure of an organization.

4. Individual Usability - adaptations should increase the user's
ability to perform tasks.




III.  Conflicts in Obtaining Adaptivity Objects

1. Too much personalization provided by adaptivity increases system
incompatibility among users.


2. System controlled adaptations can reduce the amount of autonomy a
user has in controlling levels of adaptivity.


---------------------Slide 4----------------------------
How do you design a system that accomplishes the goals of adaptivity?


IV. Take a problem oriented approach:

	This approach to system design focuses on the context in which
the adaptive system will operate.

A. To Determine the context of the system consider the following
social factors:

1. The Task - what things are attempted to be done with the adaptive
system.

2. The Organizational Framework - size, architecture of the
organization.

3. The Work Organization - degree of routinization, flexibility in
terms of sequence

4. Technological Infrastructure - the type of computer resources which
are available in an organization.

5.Informational Exchange - the type of information flow existing in
the organization

6. Internal Social Structure - how tasks are divided among workers.

7. Management Objectives - type of incentive systems, job profiles,
etc...

8. Staff Structure - kind of employees work in the organization.

-----------------------Slide 5--------------------------------
V. Evaluation of the System

A. Evaluation Tasks

1. Identify concrete adaptive needs, contradictions, and areas of
conflict in system design.

Accomplished by: inferences made based on current user tasks to be
performed with the system.


2. Provide mediation between project participants

The designers and system users all participate in the adaptive design
of the project. The evaluation should provide a neutral from the
discuss design issues.


3. Define overall outcome of the system

The evaluation should establish metrics to gauge the success of the
system.

------------------------Slide 6--------------------------------
How do you evaluate an adaptive system?

VI. Methods of Evaluation

A. Interviews/Questionnaires 

	1. Verbal Reports

		- Lots of data, can Give general impression of systems
                  overall acceptance.

		- Results can be biased, can be difficult to integrate
		  results.
	
	2. Standardized Questionnaires

		- Experienced driven angle, can pin-point specific
		  problem areas in the system.

		- Results can also be biased.

B. Controlled Experiments

	- Experiments can be replicated, good for gaging specific user
	  operation.

	- Can be costly, produces a simplified model of reality.

C. Checklists/Guidelines

	- Can produce comprehensive descriptions of the system,
	  addresses specific technical features.

	- Difficult to formulate guidelines

---------------------Slide 7---------------------------------------
VII. The Evaluation Process

1. Specify needs for adaptation - analyze a specific application
domain during the systems development via interviews, observations,
etc.


2. Feedback of Initial Findings- gather feedback from the initial
findings. This phase considers the designers and the users of the
system.


3. Test First Prototype - circulate first versions of the adaptive
software to run tests.


4. Feedback of first test results - rapid feedback of phase 3 to make
revisions of the first prototype.


5. Conclude Evaluation - similar to phase 3, provides documentation of
the design process and outcome of the design.


Conclusion:

The best approach to evaluating and adaptive system is to consider the
systems users on a social level, then formulate adaptations based on
the user, and combine the two by including the users in the design
process.

-----------------------Questions----------------------------
Questions


1. The author makes the statement, "Hiding all technical aspects of
computers would hinder users to actually learn anything about the true
technological basis of the adaptive system", is this true?


2. How do you make good inferences as an adaptive system designer
regarding the user's needs for adaptation.


3. How do you formulate theoretical guidelines to determine whether or
not adaptive systems meet their adaptive requirements.


4. What methods of adaptation can be used to facilitate learning?