P1-schneider.fm

Constructive Exploration
of Spatial Information by Blind Users
Jochen Schneider, Thomas Strothotte
Otto-von-Guericke University of Magdeburg Universitätsplatz 2, 39016 Magdeburg, Germany ABSTRACT
One evidence for orientation as learning is that people first When blind people wish to walk through an area not fully orient themselves by routes and landmarks, and only with known to them, they have to prepare themselves even more experience acquire a mental overview of the full area ([2], thoroughly than sighted pedestrians. We propose a new pp. 214). In practice, when we prepare ourselves for a trip approach to support this preparation with the help of an through an unknown area, we first look at a map and try to interactive computer method, called constructive explora- learn how to travel through it. We might take the map with tion. Using this method, the user is guided in physically us during travel, but are often too busy looking for land- constructing the spatial arrangement to be learned using marks or traffic while walking or driving to update our cur- building blocks. We describe two implementations of the rent position on the map. We therefore have to memorize concept, one with a graspable interface with object tracking and the other employing a force feedback device. We reporton first tests of the implementations.
If we examine how blind people have obtained access tospatial information in the past, we find that it has mostly Keywords
been in a passive way: They have either listened to live or Orientation aids, blind users, augmented reality, force feed- pre-recorded descriptions of an area or a route, have been lead through the actual area by walking or have studied a INTRODUCTION
Information systems in general and those for blind people In this paper we propose to apply the idea of intimately in particular put the user primarily in the role of a recipient involving the user in the process of building a solution to a of information. For example, a search for information on spatial orientation problem. In particular, we propose to let the web can be considered successful if a page has been the user explore spatial information by actively building found with all the required data presented in a succinct certain parts of it as a model with the help of an interactive form. Indeed, many users complete their search by printing out a page or a small number of selected pages.
computer system. Our approach can be considered an appli-cation of the “learning-by-doing” principle. Map learning By contrast, students are taught not only by presenting for travel planning meets the prerequisite of that principle, material to them, but also by letting them actively work namely that “there has to be a real task in which the learner with it. For example, every good mathematics book will has some personal interest and investment” [17].
give exercises for the student to practice what they have learned. Students learn new material by the aid of physical The paper is organized as follows. In section two, we talk objects: “. one learns by building artefacts — whether about the background of our work. In section three, we they be formal reports, private scribblings, Lego choo-choo define the term “constructive exploration” and show how trains, or computer programs, the path to learning is strewn we implement it in actual systems. In section four, we with things, with externalizations” [17].
describe our implementations in detail. In section five, wereport on first tests of our approach with blind and sighted When pedestrians or drivers try to prepare themselves in an users. In section six, we conclude the paper and comment area not fully known to them, they undergo a learning pro-cess, even when they are not consciously aware of that fact.
BACKGROUND
Permission to make digital or hard copies of all or part of this work for personalor classroom use is granted without fee provided tht copies are not made or dis- Map Manipulation
tributed for profit or commercial advantage and that copies bear this notice and Letting people (re-)construct routes on maps with physical the full citation on the first page. To copy otherwise, or republish, to post onservers or to redistribute to lists, requires prior specific permission and/or a fee.
objects in connection with map learning is a well-known ASSETS ’00, November 13-15, Arlington, Virginia.
technique in the psychology of education: When psycholo- Copyright 2000 ACM 1-58113-314-8/00/0011.$5.00.
gists want to assess a child’s mental map of a route, one way is to ask the child to construct the route with card graphical objects by emitting a sound when the hand of a user touches the imaginary objects on a tactile grid. When a The “Tangible Bits” concept by Ishii and Ullmer aims at map is loaded into the KnowWhere system, it can therefore coupling physical objects and digital data, so that physical be passively explored. For a test, congenitally blind sub- skills can be applied during computer use [9]. One applica- jects explored country and state maps. They were able to tion of the concept is the exploration of a map and the find absolute positions and recognize puzzle pieces of the manipulation of its display with the help of physical Van Scoy et al. propose to use the Phantom force feedbackdevice from Sensable to let blind people explore a three- Research on Tactile Maps
dimensional model of a street block for mobility training Carved maps made by the Inuit three hundred years ago can [18]. In this approach, buildings are presented as three- be considered forerunners of tactile maps. They are highly dimensional models both for touch exploration and visibly abstracted as shapes and can be felt in the dark ([11], pp.
229). Today, tactile sheet maps offer blind people suitableaccess to geographical information [4]. The choice and tac- CONSTRUCTIVE EXPLORATION
tual representation of cartographic features to present con- By constructive exploration, we mean the process of learn- stitute the main challenges in designing such a map ([3], pp.
ing information through partly (re-)constructing it by fol- 206). Besides tactile maps for geography education, we dis- lowing instructions. Applied to the learning of spatial tinguish tactile orientation, mobility and topological maps.
information, during constructive exploration, users build Orientation maps provide a general overview of a certain parts of the spatial structure while they are given instruc- area. Mobility maps are tailored towards travellers and tions on how to assemble which parts. The parts are con- include orientation points. Topological maps show a certain structed with physical building blocks.
route. Other detail is left out, the presentation is simplified To implement a constructive exploration system for spatial information, we need ways to let users explore spatial Tactile maps constitute mature means of orientation and information with the help of the computer, let them choose can be carried along on trips. On the other hand, tactile map parts they wish to learn in detail, let them (re-)construct design is not fully standardized. This is one of the reasons these parts, and finally trace them until they have learned not all blind people are able to use these maps successfully, another being that inscriptions are done in braille, which The first implementation approach uses a graspable inter- not all blind persons can read. In addition, tactile maps are face [5] which enhances physical objects with interactive not readily available for all regions or at the time they are computer technology. It consists of a rectangular tactile grid required, because they are mostly made by hand. For these on which the interaction happens and which represents the reasons, there have been efforts to enhance tactile maps map which is loaded into the computer as digital map data.
through the help of computer systems.
On this grid, users move their hands to virtually touch map Electronic Travel Aids
objects to request information on them which is given Tactile maps can be enhanced through multimedia com- through synthetic speech. Users place physical objects on puter systems. A tactile map is placed on a touch tablet, an the grid to select routes and reconstruct them.
absolute two-dimensional input device. The touch tablet is A different implementation utilizes a force feedback device connected to a computer which has the map information which conveys the impression of touching a tactile map. In that is depicted on the tactile map loaded as a digital map.
addition to letting users explore the map, the system can The computer emits sound or text information on an object also give them information on objects on the map through when the user presses the object on the tactile map and synthetic speech and even guide them from landmark to therefore the touch tablet [7]. With this approach, the explo- landmark on a route. One can imagine additional imple- ration experience can be enhanced and braille labelling of mentations of the constructive exploration concept. Other features can be substituted with speech output, but there is input/output devices can be used, and information other than map information can be explored by users.
The MoBIC preparation system (MoPS) empowers blind IMPLEMENTATIONS
pedestrians to plan a walk through an urban area [15]. The Requirements
MoPS can be used to both explore maps and select routes in The spatial information systems described here fall into the them. Routes selected are transferred to an electronic travel category of orientation aids. General requirements for the aid, the MoBIC outdoor system, which then guides users design of orientation aids can be elicited by asking how during their walk outdoors. Users interact with the digital blind pedestrians prepare themselves for a walk through an map loaded into the preparation system through the cursor urban area not fully known to them. Mobility psychology keys of a PC keyboard. The system gives them information has found that blind people who wish to navigate indepen- on their current position through speech and braille. Abso- dently have to memorize the layout of a given area, learn lute spatial input is not supported.
path segments and angles between them, and have to recog- The “KnowWhere” system is a hand gesture recognition nize them during walking [6]. Therefore, a constructive system which conveys geographical information to blind exploration system needs to provide blind people with an people [11]. The system presents zoomable outlines of geo- overview of a given area on the one and teach them straight route segments and angles at route succession choice points the tactile grid is empty and the system starts in free explo- ration mode. In this mode, the user moves a hand freely on To implement the constructive exploration concept, there the grid underneath the camera. When the index finger are no large tactile input/output devices available to be used “touches” a cartographical object on the grid, like a street by blind people in the same manner as mice and graphics or a building, information on this object (e.g., its name) is screens are used by sighted people (apart from research given by the system through synthesized speech.
prototypes [16]). Therefore, new interaction techniques To change into route learning mode, the user places two were devised and implemented, while consulting a mobility route end pieces on street crossings on the grid. The system then searches a route between the two crossings, careful notto select unwalkable map segments like highways, andannounces to the user that it found a route. The user thenconstructs the route with long physical objects, guided bythe system. To construct a route, there are a number of game pieces ofdifferent lengths to choose from. The system tells the userwhich length the next piece to place should have. The userplaces the first piece next to the route start object. The sys-tem then tells the user through sound how to turn the object,with the start object as the axis, so that it lies on the route.
After the first brick has been correctly placed, the systemtells the user the size the second brick should have. Theuser then places the second brick guided by the system in asimilar way as for the first brick, with the end of the firstbrick as the connection point, and so forth for the rest of thebricks. When the last brick is correctly placed, the systemtells the user that the route has been successfully built.
In order to let the user place a brick, the system only needsto tell her or him the correct angle, because the position(actually the rotation axis of the brick to place) can bederived from the objects already placed. The angle is con-veyed trough a pulsed sound once a brick is placed at theend of the previous brick: the pulse frequency gets higheras the angle of the brick gets closer to the correct angle. Thesound stops once the brick is correctly placed. The routeend pieces stay on the pad during the construction of the Figure 1:
Screen shot of the visualisation of a map and a route. When they are removed from the pad, the system switches back into free exploration mode.
Once a route is chosen by placing the two route end pieces Implementation through an Image Processing System
on two crossings, the systems calculates the shortest path Our first implementation of the constructive exploration between them. The system then simplifies the path calcu- concept contains an image processing system for input and lated by merging two neighboring segments if no third seg- speech and sound for output. Users interact with the system ment leaves the node connecting the two and the angle with their hands and physical objects, whose positions are between them is roughly 180°, i.e., one is the elongation of tracked by a camera. The systems allows for the explora- the other. The route is then scaled down so that it starts and tion of digital map data and the selection and learning of ends at the border of the respective route end piece. A lin- routes in it. The digital map data is not displayed explicitly, ear scale is used in this step, in contrast to a system for pro- but is conveyed during interaction: Textual information is ducing tactile route maps [13]. The system then calculates presented through synthetic speech, other information how each straight route segment has to be scaled so that it can be filled with route bricks. Finally, the segments are On a table there is a tactile grid which delimits the interac- attached again to form a closed route.
tion area and gives the user a sense of where the hands and All graphical information like the map, the current position objects are on this area. Objects similar to the pieces of a of the index finger, the current object of which information game serve to select the two end points of a route and to is spoken, the position of the route end pieces, the current actively construct the route once it has been selected.
route and the route bricks placed so far are displayed graph- The systems supports free exploration and route learning in ically (see Figure 1). This allows for the cooperation of a two different modes between which users can switch with sighted and a blind user (and facilitated the development of the help of the construction objects, without the need to the system). Digital map data is loaded into the system as explicitly switch modes with a program command. At first, map files covering a certain area, e.g., a part of a city.
The physical objects used have a special design to enable tom, they are converted to a three-dimensional representa- the interaction just described. Bricks need to attach to the tion, which resembles engravings in a metal plate when last brick already placed (or the route start piece) so that they can be turned with the previous brick as the axis.
A three-dimensional model of a map was created, similar to Route end pieces and bricks already placed need to stick to a tactile map, to be explored by users with the “Haptic the pad so that they cannot be moved accidentally. Both VRML Guide”, a haptic exploration system for those mod- these requirements for the bricks are met by placing mag- els for the Phantom [10]. To create the map model, each nets on both ends of the bottom of the bricks. Magnets of map object was converted into a three-dimensional repre- two neighboring bricks stick together and route end pieces sentation in the modelling language used by the exploration and bricks already placed can be moved less easily when a system (VRML, see [8]). Under this scheme, segments are converted to cylinders. The VRML map is then manipu- The camera is mounted over a table, facing down. The tip lated with the help of a 3D modelling system by “subtract- of the index finger is marked with a colored ring, which ing” it from a block of suitable size, which makes the map enables tracking of its position through image processing.
“engraved” into the block. To explore the map, it is loaded Due to the nature of the image processing system currently into the “Haptic VRML Guide” system (see Figure 2).
used, bricks need to be fully visible to the camera in order (This system was developed by our colleague Henry König to be tracked correctly. On the other hand, bricks need to be tracked even when they are grabbed by a user. Bricks are Once the map is loaded into the system, users can trace therefore constructed in a way than they can be held in the streets will be able to listen to information on them given middle while still leaving the top unoccluded.
by the system through synthetic speech. In order to make Implementation through a Force Feedback Device
the exploration constructive, routes can be built with virtual For a different approach to implement the constructive objects in the haptic space. The construction will start at exploration method of conveying graphical information to one of the route end markers. A long, thin object the size of blind people, a Phantom force feedback device from Sens- the length of the first segment will be attached with one end able [12] is used. This device has the advantage that it can at one of the markers. It can freely rotate around that realistically simulate the effect of touching three-dimen- marker. The user can place the first segment by rotating it sional objects. (On the other hand, this effect is only until it engages at the start of the next segment. Then, an applied to one finger tip and the device will generally find object representing the next segment appears and can be no wide-spread use, because it is too expensive.) In order placed in the same way, until the whole route is con- to let users explore two-dimensional maps with the Phan- Figure 2:
Map data loaded into the system to explore it with a force feedback device (front view) EVALUATION
Vision, in Portugali, J. (ed.), The Construction of Cogni- The camera-based constructive exploration system was first tive Maps, Kluwer, Dordrecht, 1996, 215-246.
tested by a sighted user. He was able to successfully choose 7. Holmes, E., Jansson, G., and Olsson, E. Tactile and build a route with the help of the system. Route con- enhancement of reading a tactile map presented via syn- struction was also tested by a congenitally blind subject. He thetic speech. Proc. Maps and Diagrams for Blind and was asked to build a route with five route bricks. He was Visually Impaired People: Needs, Solutions, Develop- able to place the bricks at the right position and angle. It ments (Ljubljana, Slovenia, October 1996), Interna- was expected that the fact that route bricks do not snap into tional Cartographic Association, 1996.
each others’ ends would make using them harder, but thissurprisingly turned out not to be the case. After a short 8. ISO/IEC 14772-1, Information technology - Computer graphics and image processing - The Virtual Reality while, the subject was also able to build a route without Modeling Language (VRML) - Part 1: Functional speci- occluding the bricks with his hands.
fication and UTF-8 encoding, VRML Consortium, The user had fun with the system. After he knew how to correctly handle the bricks, it didn’t take him long to place 9. Ishii, H., and Ullmer, B. Tangible Bits: Towards Seam- them. The user is optimistic that he can learn routes with less Interfaces Between People, Bits and Atoms, in Pro- the system. Formal evaluations of both the camera-based ceedings of CHI ’97 (Atlanta, GA, March 1997), ACM and the force feedback-based implementations will be car- ried out in the near future. They will both be based on thehypothesis that routes can be conveyed better with the sys- 10. König, H., Schneider, J., and Strothotte, Th. Haptic tem than by verbal descriptions. In order to test this hypoth- Exploration of Virtual Buildings Using Non-Realistic esis, subjects will be divided into two groups. Each subject Rendering, in Proceedings International Conference on will construct a route with route bricks in one of two ways: Computers Helping People With Special Needs by either listening to a verbal description of the route or by (ICCHP) (Karlsruhe, Germany, July 2000). AustrianComputer Society, 377-384.
being guided by a virtual tactile map system. The routesconstructed by the two groups will be compared.
11. Krueger, M.W., and Gilden, D. KnowWhere™: an Audio/Spatial Interface for Blind People, in Proceed- CONCLUSION AND FUTURE WORK
ings of the Fourth International Conference on Auditory Our work has focused on blind users, who are mostly Display (ICAD) `97 (Palo Alto, CA, November 1997).
ignored by current design efforts for multimedia systems.
On the other hand, our constructive exploration approachand implementation methods can benefit sighted users, as 12. Massie, Th.H. and Salisbury, J.K. The PHANTOM well: Sighted people also find themselves in the position of Haptic Interface: A Device for Probing Virtual Objects.
learning the layout of an area they want to travel in and Proceedings of the ASME Winter Annual Meeting, Sym- posium on Haptic Interfaces for Virtual Environmentand Teleoperator Systems (Chicago, IL, November REFERENCES
1. Blades, M. Research Paradigms and Methodologies for 13. Michel, R., Computer-Supported Symbol Displace- Investigating Children’s Wayfinding, in N. Foreman, R.
ment, in Ottoson, L. (ed.), Proc. 18th International Car- Gillet (eds.), Handbook of Spatial Research Paradigms tographic Conference, Vol. 3, (Stockholm, June 1997), and Methodologies Vol. 1. Psychology Press, East Sus- 14. Papanek, V., The Green Imperative: Natural Design for 2. Downs, R.M. and Stea, D. Maps in Minds: Reflections the Real World, Thames and Hudson, New York, 1995.
on Cognitive Mapping. Harper & Row, New York,1977. 15. Petrie, H., Johnson, V., Strothotte, Th., Raab, A., Fritz, S., Michel, R. MoBIC: Designing a Travel Aid for Blind 3. Edman, P.K, Tactile Graphics. American Foundation for and Elderly People. The Journal of Navigation Vol. 49, 4. Espinosa, M.A., Ungar, S., Ochaíta, E., and Blades, M.
16. Schweikhardt, W., Interaktives Erkunden von Graphi- Comparing Methods for Introducing Blind and Visually ken durch Blinde (Interactive Exploration of Graphics Impaired People to Unfamiliar Urban Environments.
by Blind People), in Bullinger, J.-J. (ed.), Proc. Soft- Journal of Environmental Psychology 18 (1998), 277- ware-Ergonomie ’85 (Stuttgart) Teubner, 1985, 366- 5. Fitzmaurice, G.W., Ishii, H., and Buxton, W. Bricks: 17. Soloway, E., Quick, Where do the Computers Go? Laying the Foundations for Graspable User Interfaces, in Proceedings of CHI ’95 (Denver CO, May 1995), 18. Van Scoy, F.L., Baker, V., Gingold, C., Martino, E., and Burton, D. Mobility Training Using a Haptic Interface: 6. Golledge, R.G., Klatzky, R.L., and Loomis, J.M. Cogni- Initial Plans, in Proceedings Fourth Annual Phantom tive Mapping and Wayfinding by Adults Without Users Group (PUG) ’99 (Boston, MA, October 1999).

Source: http://cs.unc.edu/~welch/class/mobility/papers/p188-schneider.pdf