Team Gromit

CS 6751B

March 12, 1999

Problem Description

Law firms, by the nature of legal work and court requirements, are required to keep hard copy and associated paper documents of their work. Unfortunately, the paper filing systems, used to keep these documents, have numerous shortcomings, resulting in excessive time to retrieve items, real and intangible costs, frustration, and in some cases lost documents and information. An inability to reliably retrieve information and documents, previously stored, can mean the difference between winning and losing a case. The loss of an original contract document can result in an unenforceable agreement with the associated loss of large sums of money. The nature of a law firm's work demands extremely reliable retrieval accuracy.

The retrieval accuracy of paper file systems can be improved by changing the faulty retrieval model currently in use. A major problem in life is retrieving information or an item that you have placed somewhere. To retrieve either, you must first recall, in human memory, the physical location of, or a map to, the information or item. Studies have shown that you can recognize much more information than you can recall. A system that aids mapping through recognition would greatly improve file retrieval speed and accuracy.

Requirements Summary

We believe that the retrieval of filed documents can best be enhanced by aiding the user to recognize categories, words, subjects and other mappings, uniquely associated with a document and the file system, rather than depending on a user to recall them. You can recognize much more than you can recall.

The current file systems, in the examined law firm, have a number of advantages. A major advantage is its existence for over 10 years. Personnel are familiar with it. Other advantages are its similarity with other file systems in other law firms and corporations. New personnel can learn it quickly. Retrieval is quick for small systems, even in the "date organized" filing "piles."

Although there are many advantages of the system, there are also many disadvantages. You must correctly file documents in a recallable location, relate the location of the document to the data entered, and be able to remember the recall keys (primarily numbers, dates and words) which you previously entered or scanned from the document.

Beagle will address a major drawback of the file system: remembering the keys, categories or words that allow you to locate a document. Beagle will operate on the principle of recognizing rather than recalling keys and categories. It will not preclude early recall, but will aid you in recognizing how you might recall. It will provide you with "cues." If the first attempt doesn't find the document, browsing and further prompts will help recall.

Design Progression

We focused on the main or everyday filing system rather than the portable system used for court and depositions. Initially, we developed three approaches. The approaches covered a spectrum from spoken language interaction to menu-based interaction to navigation through a document space. This section will briefly recount each of the previous designs and why they were not adopted. Our fourth and final approach, the SearchCone, was a variation on the menu-based approach. It also took concepts from the Camtree approach. It was developed to deal with the difficulties in understanding and specifying "and/or" searches in the menu-based approach. The final approach, being a new approach, will be more completely documented in the next section.

Approach A: Natural Language

This interface is based on computer understanding of human language. It uses the metaphor of a human assistant who goes out and searches for the documents needed. For practical purposes, we need to constrain vocabulary and grammar to avoid the ambiguity of everyday spoken language. Accuracy of the speech recognition engine will directly affect performance of the system. The output of the system would be a graphical display both because it allows for faster interaction and because it avoids the need to develop full interaction dialog. The system returns a document or set of documents that the user may browse through. If the user sees the document that he wants, he can select the document(s) and get their locations. Otherwise he can change the search parameters. Search parameters that have been entered will be displayed in the bins on the left of the display. Documents returned by the search would be sorted by title (or what ever field the user prefers) with a brief description by the title in the frame at the right. If the user wishes, he can call up a picture of the front page of the document for visual recognition. It would also be possible for the user to select a document and ask for more documents like the selected one.

Design Illustrations

Approach B: Menu and Dialogue

The Menu-based Document Browser (MBDB) utilizes a color graphics based computer display (PC) to search for documents. It attempts to retain the original search techniques the law firm uses to manually find documents in a paper based filing system (keyword searches and date ordered browsing). In addition, it provides for the capture of document images electronically and adds new database search techniques to find documents. The database interfaces and search techniques are designed for file clerks, secretaries and paralegals that use the law firm's filing systems. Although attorneys can easily learn the system, it is unlikely that they will become users due to organizational culture.

The search techniques are based on the principle that the user can recognize much more than they can remember. Multiple approaches, such as visual icon (picture) reminders, listings, key words and phrases and visual images of documents, are provided to prompt the user’s recall of a document. The search is menu-based prompted with common functions grouped together and color-coded. The main menu screen provides a "glimpse" of all the techniques that can be used to find a document. The screen nesting is limited to one level below the main menu. One may quickly return to the main screen to begin a new search, abort a search, view documents, add additional criteria or reset a menu.

Documents are scanned with an image scanner with OCR software to capture the text and visual image of the document in computer memory. The user familiar with the document may enter further information about the document. Therefore, both the words and phrases of the document as well as information about the users current relation to the document are stored. A sequential number is automatically assigned to the document during the capture process to speed manual file access for trials and depositions and indexing locations.

Design Illustrations

Approach C: Camtree

A Conetree is a 3D hierarchical tree. Each node may have sub-items below, arranged in a ring. The links between the node and the items form the outline of a cone. A variation of the Conetree is the Camtree. It is built from left to right, rather than up and down, which makes labeling the items easier.

In our implementation, the user can select which criteria are used to build the tree. The user may also modify the order of the criteria. To search for an item, the user can browse through the tree. Although this may sound like an extremely slow method of searching, the hierarchical arrangement allows fast comprehension of where documents are located. Also, nodes of the tree are labeled with the criteria and the ranges of values of the documents below. An additional highlight feature is also provided which allows the user to perform operations more like a traditional search. By specifying search criteria, documents of interest in the tree are highlighted. The user may also rearrange or select new tree building criteria to co-locate the highlighted documents in the same part of the tree. These search criteria can also be used to control which documents participate in the tree. This can focus the user's search by eliminating documents that are known to be wholly irrelevant.

Design Illustrations

Design Assessment

The design chosen for implementation of a legal document search interface is the Menu Based Document Browser (MBDB). The primary reasons for this choice over the other approaches of Natural Language and the Camtree, as discussed in Part 2 of the project are as follows:

1. Similarity to existing windows based graphic interfaces.

2. Personnel in this particular office are familiar with such computer interfaces.

3. All information about the different search approaches is shown in one menu.

4. Law firm users preferred it to the Camtree for a visual approach.

5. Exceeds the original HCI goals of consistency, dialog initiative and observability by adding familiarity, predictability, substitutivity, recoverability, responsiveness and task conformance. These are discussed in more detail in Section 5.2 of Part 2 of the project.

6. Has multiple ways of helping the user recognize clues to documents rather than having to remember them.

7. The approach is more feasible to implement in a reasonable time frame.

The most promising approach is the Natural Language design, but the tools to implement such a design are immature and expensive. However, the approach will probably be commonplace in the future or as an addendum to the present visual GUI approaches. This is the way a person naturally tells someone else to search for a document.

The Camtree had the advantage of being able to see the entire document universe at one time. However, the implementation is a major effort, requiring extensive graphic development tools and high end computing equipment for the implementation. The design also met with resistance from the law firm.

After the design choice was made, we discovered that there was a major problem with the presentation of the "and/or" function of the Menu Browser. You could easily forget what mode you were in, in the process of a search. Feedback of the mode on a continuous basis became a problem. We also questioned if the typical user would understand the Boolean "and/or" functions well enough to make an effective search. We decided to find a way to overcome that flaw. Our solution became the new design approach, the SearchCone, not conceived in Part 2 of the project.

Final Approach: SearchCone

Description

The SearchCone was developed to solve the "and/or" problem. It is a visual representation of the selection process chosen. The cone geometric representation shows intuitively that one is narrowing down the search process as you move toward the narrow end of the cone. This represents the "and" search in a universe of documents where additional criteria causes the search to narrow to fewer documents. If the user moves toward the larger end, the selection criteria cause you the number of documents to increase. This represents an "or" group of criteria which adds more documents as you add more criteria. Figure 4 and Figure 5 depict early versions of the SearchCone that allowed either an "and" search or an "or" search. Further refinement of the cone yielded Figure 6. The desire to achieve more general combinations of the two functions can be implemented with the RollingCone which is discussed later.

The SearchCone in Figure 6 has several major components.

1. Levels at which the documents and/or their descriptions are "stored."

2. A Magnifying Window which expands the view of the representations of the documents at a level of interest.

3. A Criteria Selection Window which shows the user the selections contained in the documents for the Category selected.

4. A Category Selection Button Bar to begin the search.

5. A Rolling Cylinder of SearchCones to expand the search and preserve history.

Levels

Each level in the SearchCone represents a selection of documents, which are visually represented by document icons, descriptions or the actual document images. The documents cannot be viewed directly because of the small size. However, each level, as it becomes populated with documents, represents visually and textually the document group you must search through. The user sees the document numbers decrease and expand in a level, as you select the criteria in a category. The user gets a perspective of the direction their criteria selections should be chosen to expand or reduce the document group. The searcher moves left or right to new levels depending on whether the search is expanding the group of documents or narrowing it. A search is constrained to move in one direction only, once the initial choice is made in a cone. It is either an "and" or an "or" search. More complicated searches are made using the RollingCone explained later.

Magnifying Window

The magnifying window expands the visible image or description of a limited subset of documents at a level to readable size. The window is visually mapped to the cone area it is magnifying. The user may scroll documents through this window by using the "scrolling buttons." The circular level visually rotates the documents through the window. If the user desires to see more documents, "clicking" on the "magnifying glass" button brings up a second full screen with more document images. (See attached representation). At the top and bottom of the magnifying window is a "Narrow Further" button, which moves the Magnifying Window to the next level to view a selection of documents based on additional criteria entered.

Criteria Selection Window

The Criteria Selection Window is used to see the universe of selections in a category. Represented in the attached figures is a selection of "people" from which you can recognize names to select for possible relations to documents.

The user may scroll through the selections or go to a second window with a larger selection displayed by clicking on the Magnifying Glass" icon. The user clicks on the name or names to indicate whether they are include in the search criterion. A particular selection of certain names will dynamically produce a selection of documents that have those names in them, for viewing at the next level of the cone. As you change your selections the next level adjusts the document selections accordingly. One can always move back and adjust the search criteria at any level and the corresponding higher or lower levels will adjust documents according to the revised search criteria. This window moves to the next level when the user wishes to select additional criteria for narrowing or expanding the search.

Category Selection Button Bar

The Category Selection Button Bar is the beginning of a search at any level in a cone. It is the first choice of selection criteria to begin the search. To search for documents with people in them, click on the button bar category "People." All the people associated with documents as determined by previous selections are shown in the Criteria Selection Window. The user may further narrow the criteria by clicking on selections in the window. When the user wishes to make the next selection criterion, the button bar is moved to the next level and the process is repeated.

RollingCone

The Rolling Cone (Figure 7 and Figure 8) was developed to solve the problem of multiple "and/or" searches. It has the advantage of preserving all the history of the search process and changing the mode of search. The user has full view of the cone presently being searched. To change the Boolean mode of the search, or to continue the search with larger viewing levels, simply use the "Send" button to copy the last level search to the new cone and roll it into view. The search can continue with its present direction (narrowing or expanding the number of documents) or changed to a different direction. A perspective view of the most recent cone remains adjacent to the new viewing cone. An unlimited supply of new cones is provided for new search directions while the entire history of the search is captured and viewed by simply rolling back to previously used cones.

Rationale

The design process began by trying to understand the problem of finding documents in a legal filing system. A law office was visited and extensive user interviews were done to understand the organization of the filing systems and the users. The task of finding documents in a file system was modeled by using Hierarchical Task Analysis. We researched the way human memory best remembers information. The major concept came when the literature revealed that human memory can recognize far more than it can remember. This reinforced a suspicion that the key to successful searches by personnel in a document search was best when they related to things about a document: people, places, subjects, etc. This oriented them toward where they might find a document previously filed. The usability criterion chosen for final evaluation of a prototype were consistency, dialog initiative, and observability.

The designers then designed 3 interfaces that would help you recognize items about a document rather than having to remember them. The three approaches were a Natural Language interface, a Menu Based Browser and a Camtree. Of these, the Menu Based Browser was chosen for the following reasons:

1. Similarity to existing windows based graphic interfaces.

2. Personnel in this particular office are familiar with such interfaces.

3. All information about the different search approaches is shown in one menu.

4. Law firm users preferred it to the Camtree for a visual approach.

5. Exceeds the original HCI goals of consistency, dialog initiative and observability by adding familiarity, predictability, substitutivity, recoverability, responsiveness and task conformance. These are discussed in more detail in Section 5.2 of Part 2 of the project.

6. Has multiple ways of helping the user recognize clues to documents rather than having to remember them.

7. The approach is more feasible to implement in a reasonable time frame.

The designers then discovered the "and/or" visibility problem in the Menu Based Document Browser and developed the SearchCone approach to successfully overcome it. The design was presented to users and the comments were captured which then generated further modifications and the Conclusions, which follow this section.

Design Scenario

Initial Evaluation

A preliminary evaluation was performed on March 9, 1999. Two secretaries at a local law firm gave us an initial evaluation of our project. We showed them a paper mockup of a possible sequence of events and saw how much they seemed to understand from that. Then we had discussion with them about what they thought about the system as a whole.

Initially they had some trouble understanding what was going on. While they understand from working with databases that a given document would have various fields which you could search through associated with it they could not tell, just by looking at the paper mockup, what to do from there. Once we explained what the basic principles of our interface were they seemed to understand how it would work. They did understand that you could either search through the different documents at a given level or that you could enter more fields to further narrow the search. They liked the idea that you could pull up a picture of the document to look at it. And they knew that if you didn’t find what you were looking for you could just restart the search.

They recognized very early on that our system doesn’t do anything that you cannot already do with a normal database. The one who was more experienced with databases thought that she would likely get frustrated by how slow our GUI interface was compared to her customary command line interface. However both of them agreed that our interface would be easier for someone to learn. This could be an important factor because apparently none of the people there has had any formal training in how to use a database.

Currently they do a keyword search on various different fields in a database. If they get a lot of hits they can put the documents in chronological order and get a text description of the contents. When a search fails it us usually because they have entered the wrong keys, the document is lost or they don’t have a copy of that document. Also their searches are generally fast, taking less than five minuets. They said that they have very long searches, 20+ minuets, about five times a year. The most common reasons for the longer searches are that either they were given vague information or that the document in question has been taken out of the filing system. They suggested that you should be able to print out documents from the pictures stored in memory, which would help increase the retrieval speed and alleviate to problem of having the only copy lost on someone’s desk.

Some concerns which they had were the cost of implementing the system and the amount of memory it would take up, and that some documents are not currently entered into a computer.

From the interview we have been able to deduce several things. One is that we would need to focus on speed in future versions due to their accustomed high retrieval speed. Another is that experienced database users will likely get frustrated with the slowness of a GUI interface, however we should be able to at least partially alleviate this problem by including an optional command line interface. We might also consider incorporating more features from more traditional databases to take advantage of users current experience with them. However we should strive to insure that our system is bot easy to learn and to explore without any formal training. This would be the major selling point of our system over more traditional databases.

Informed Consent

The following informed consent information was given to the two representatives of the law firm participating in the assessment.

This presentation and evaluation is a part of a quarter long project in a graduate course (CS6751) at The Georgia Institute of Technology College of Computing. The project has designed a Human Computer Interface for a new approach to finding documents in a paper document based legal filing system. The designers of this approach are Terry Hughey, David Krum, Crane Laws and Wasinee Rungsarityotin.

This is what is known as a paper design. That is, it is not implemented on an actual computer yet, although we will show you some of our thoughts using a computer.

The primary improvement of this interface, over other search methods, is based on taking advantage of the human memory principle that you can recognize far more things than you can remember. In other words, the interface seeks to prompt you with recognizable things that will help you remember where to find a document. The interface also uses a computer database search and a visual search technique that helps you progress in the direction most likely to aid you in locating where a document may be found.

As a user of filing systems in the legal profession, we would like your honest and candid assessment of this approach. In order to get your assessment, we will make you familiar with the approach and then ask you to use the approach to tell us how you would go about finding a document using this interface. We don't expect you to be familiar with all the aspects in such a short time. That is one of our interests. How can we make it easier to use and more friendly? Your assessment will help us do that. You should not feel uncomfortable in any way if you don't understand something. Ask us at any point if you don't. This is not an evaluation of your abilities but rather our abilities to design a good interface.

We want to know as much of how you are thinking about this design as you are willing to give us. Tell us you thoughts at any time during the evaluation, as you seek to find a document using this interface and as the approach is explained to you.

We want to be sure you are comfortable participating with us in the evaluation of our design, so feel free to ask us any questions about this approach, our project, the course or ourselves. The results of this evaluation will be used by the designers, professors and students associated with this course, in an academic environment, to evaluate and improve the design.

You may stop the evaluation presentation and process at any time you choose to. Just tell us. We appreciation your participation and time.

Observations

The following observations were made during the meeting.

Screens:

For the purposes of organization I divided the paper part of the presentation into three screens. The first screen is when they are populating the first level, the second screen is when they are populating the second level and the third screen is when they are populating the third level.

Screen 1:

They did manage to guess that you would use that screen when looking for a specific person, however they didn’t know what to do beyond that point.

Once we explained what was going on they understood that the purpose of the cone was to narrow down the search and that every document would have a person associated with it.

Screen 2:

They did manage to guess that the initial icons brought up a picture of the document.

One problem was that they thought that the magnifying glass icon started a search rather than magnifies an icon into a document.

They also thought that they would might recognize a document here or do a word search to find one.

Screen 3:

Initially they seemed a bit confused about why some documents were highlighted but eventually understood that those were the selected dates.

They also understood that if you didn’t find the document you were looking for to you could start the search over.

Critique:

One point mentioned is that they already do this with Microsoft Access. And that our presentation was a bit hard to understand because it was on paper instead of on a computer.

Other Notes:

They seemed to understand that you are pinpointing the document as the cone narrows. Another point was that their correspondences were not normally entered into a computer.

Currently when looking for a document they do a key word search on various fields in Microsoft Access.

Ex: By name and date

If they get too many documents they can put them into chronological order and get a description of the document.

If the search is unsuccessful the usual reasons are that they have used incorrect terms, that they don’t have the document ant that they have lost the document. When they don’t find the document they are looking for they restart the search rather than edit the history.

The one who is more experienced with databases (sorry, I didn’t catch the names) sees that we are not doing anything that she isn’t already doing in access. Also she doesn’t believe that she would be as satisfied with our GUI after using the command line. The one who is less experienced with database thinks that our system would be much easier to learn. Neither thinks that the lawyers would use it very much. Another point is that neither has had any formal training about how to use a database.

They mentioned that one of their uses for a database is to organize and search through large boxes of unorganized documents.

With their current system they can usually find a document in less than 5 minutes. They generally only have trouble when the specifications are vague or the document has already been taken out of they filing system. They maintained that they only take 20+ minutes to find a document about five times a year. Some others may take longer to find a document on average.

They have several concerns with keeping all of the documents on computer namely memory and price. Also some correspondence files are not very important.

They think that the picture idea is good because then you can print the document right away. Also they think that our system would likely save some time when finding a document.

Design Revision

The keyword search feature that the law firm currently uses in their portable file system was not in the SearchCone design. This was an oversight than can easily be corrected. Figure 18 and Figure 19 demonstrate that a keyword search option can be added to the level button bar and that a keyword search dialog box can be associated with a level.

Plan for Further Evaluation

Our original proposal used Consistency, Observibility and Dialog Initiative as our primary usability criteria. To measure Consistency we would use Heuristics early on in the design process. While the experts would be encouraged to report any other flaws they would be told that at this stage what we are primarily interested in is consistency. The testers would be shown how the different parts of the interface work, and then we would give them a set of sample problems to do. What we want from them would be whether or not the way that you search for different types of documents is consistent as well as whether narrowing the search down to the next stage of the cone tree is consistent with what you had to do at the earlier stages.

Dialog Initiative would be tested for using the Think Aloud technique. What we would do would be to get some secretaries and paralegals to test out our system. After giving them some preliminary training in how it works we would give them some sample tasks to perform. As they worked we would try to get them to talk about what they are doing and why they are doing it. We would pay special attention to any comments in which they express frustration about not being able to do what they want. We would most likely use a combination of notes and a video recorder to capture our data.

Observability would be measured with a Cognitive Walk Through. In this case we would step the users, again secretaries and paralegals, through several sample problems. While doing that we would ask them questions about what they think is going on at various points in the system as well as what they think that they should do next to complete the task.

Conclusion

The SearchCone and Rolling Cone has advantages over other search interfaces. First, the visual graphical interface lets the user see the progress of their search in numerous ways. It preserves the history of the search, allowing the user to return to previous levels to modify the search and watch the outcome in the number of documents. Second, it lets you observe the mode (and/or) of your search visually and graphically by the population of documents and the shape of the cone. This solved the defect in the Menu Based Browser. Third, it preserved the advantages of our original major goal by helping you to recognize criterion for searches rather than the existing window search engines or database search techniques which force you to remember words or phrases. Forth, it kept the original HCI design criterion of consistency, dialog initiative and observability and added additional criteria not originally conceived.

As can be seen in the user evaluation of the design, we did find several problems. The more experienced users of the law office filing systems did not react well, feeling it would slow them down in searches. The absence of a keyword entry technique was pointed out and should be added to bypass the constrained graphical search process of the SearchCone. The more experienced user preferred the Menu Based Browser approach. It should be reconsidered. This category of user is capable of doing Boolean searches with database software, and is analogous to the command line versus GUI arguments. Is command line faster for the experienced user? The keyword entry search capability should be added to the SearchCone for experienced users.

The more inexperienced the user, the warmer they were to the SearchCone approach. However, no user was able to evaluate the interface without some user training and explanation of the functions. We need to think of ways to make it still more intuitive. The RollingCone was confusing. Better rendering of this concept and the SearchCone, or a fully working computer prototype, would probably eliminate much of the confusion.