This duality of text and speech reinforces the messages the system wants to convey, and also allows the use of the system by those who have a hearing or sight deficiency.
This first sketch shows the interface as it appears to the customer upon first approach. The "virtual employee" is pictured on the left. This will be a recorded video image of an actual person, and will be used throughout the ordering process to give a feeling of human-human interaction to the customer. The face gives a greeting and some brief instructions, corresponding to the text printed on the right half of the screen.
The use of color is also shown here, with the instructions (very important for first time users) highlighted in red.
The second drawing is representative of the interface's appearance at some point in the middle of the process. The image of the virtual employee is smaller, allowing room for several new fields.
The large box in the top right of the screen is feedback to the customer on what has been ordered so far. An itemized list shows the customer precisely what the system has interpreted his/her order to be. This will lead to a better likelihood that mistakes will be caught before they are sent to the kitchen.
The two smaller boxes on the bottom of the screen give a textual representation of both sides of the dialogue between customer and system. When the customer speaks a command, the system puts its interpretation of the command into the "You say" box. Again, this is strong feedback to the user. The "Computer says" field is used for a textual duplicate of the spoken response of the system, just like the greeting in the previous sketch.
Again, color is used for coding. The image of the employee and the "Computer says" field are enclosed in green boxes, while the "You say" box is in brown. This gives a separation between the two participants in the dialogue.
The final sketch shows the interface as it would appear at the conclusion of a customer interaction. After the user gives one of the termination commands, the "You say" box disappears, and the "Items Ordered" field expands to the bottom of the display. The tax and total are calculated and displayed here. The system gives the total cost to the customer, in both verbal and printed form, and gives the user one last chance to make changes before the order is submitted, by asking the question, "Is everything correct?" An answer of "Yes" will submit the order, and an answer of "No" will put the user back into ordering mode, as shown in the previous sketch.
Red is again used as an "attention" color here, with the tax and total appearing in red. This should draw the customer's attention to these important numbers.
On to textual storyboard prototype