Implicit Navigation with Contextualized Audio Contents (Diploma Thesis)

Introduction - Scenario - Design - Evaluation - Publications - Press Coverage

Introduction

A variety of navigation systems have been developed that use a GUI-based interaction style. However visual navigation systems are often inappropriate in the dynamic mobile context since the user has to watch the device and cannot keep his eyes on his surrounding environment. Auditory navigation systems are more convenient, mobile users can easily interact with the system and are not visually distracted. But most auditory systems navigate the traveler by using precise spoken instructions and speech requires high attention.

The approach of Melodious Walkabout is that explicit instructions are not necessary to navigate a person. Melodious Walkabout is a wearable system that implicitly guides a user by providing him with awareness where the destination is located. It navigates users in a completely new way. It doesn’t use speech and allows the user to listen to his own audio contents while being aware of the location of the destination.

Scenario

Joy is located in the middle of a huge unknown town. She faces the everyday problem to find the way from one place to another. Usually she starts asking people for the way, struggles with maps and finally panics. This time it is different - Joy puts on headphones and selects her destination and favorite songs on her Personal Digital Assistant (PDA). The music begins to play and Joy follows the music through the city. After listening a couple of her favorite songs Joy arrives in best mood at the destination.

Design

Melodious Walkabout is a PDA-based auditory navigation system. It provides a mobile user with awareness where the destination is located by contextualizing audio contents the user is listening to. The mobile user wears headphones and hears audio contents that reach him from a certain direction. The direction of the virtual sound source unobtrusively tells him in which direction to go. An important feature of Melodious Walkabout is, that it can be used with any audio content. This gives the user the opportunity to listen to audio contents he likes to, while at the same time being aware of the direction and distance of the destination.

Melodious Walkabout consists of three main components:
1. A Bluetooth GPS-receiver for determining the traveler’s position and movement.
2. A Geographical Information System (GIS)
3. An Auditory Display that renders personal audio contents in real-time

Evaluation

A formal evaluation under field conditions proved that navigation with contextualized audio contents is efficient and intuitively understandable. Users were highly satisfied with the navigation support given by the evaluated navigation system.

Melodious Walkabout

Melodious Walkabout is the diploma thesis of Richard Etter in computer science in media at Furtwangen University of Applied Sciences. I designed, developed and evaluated the system at Fraunhofer Institute of Applied Information Technology. My thesis supervisor at Fraunhofer was Dr. Marcus Specht

(Pictures by Bastian Zapf)

Publications

>>> Melodious Walkabout - Konzeption, Entwicklung und Evaluation eines Systems zur impliziten Navigation von Personen mit kontextualisierten Audioinhalten (Diploma Thesis), University of Applied Sciences Furtwangen, 2005

>>> Melodious Walkabout - Implicit Navigation with Contextualized Personal Audio Contents, Adjunct Proceedings of the Third International Conference on Pervasive Computing, ISBN 3-85403-191-2, pdf

>>> Din A1 Poster Melodious Walkabout, Third International Conference on Pervasive Computing, pdf

Press Coverage Melodious Walkabout

>>> Melodious Walkabout in BPM, HOLLYWOOD, USA
I have been interviewed by BPM, America’s leading journal covering the electronic lifestyle. Michael J. Cervieri wrote a great article about my thesis Melodious Walkabout. They also created an excellent artwork for the article. BPM has a readership of about 350000.

Download

>>> Melodious Walkabout was selected for the 15 year celebration of the faculty Digital Media, Press Release of Furtwangen University of Applied Sciences (German)

>>> Fraunhofer Institute FIT presented Melodious Walkabout at LEARNTEC, the European Conference and Specialist Trade Fare for Educational and Information Technology.

>>> Melodious Walkabout was presented at CeBIT, the world’s largest trade show for information and telecommunications technology by Fraunhofer Institute FIT.

>>> GEAR live: “Melodious Walkabout Auditory Navigation System”

>>> Near Near Future:“PDA-based auditory navigation system”

This work was developed at the Ambiente Group (Fraunhofer IPSI)
by Richard Etter, Carsten Röcker, and Jennifer Heier

Social Radio is a novel approach for mediating awareness in small intimate groups. Instead of traditional communication media, music is used to inform users about the presence and mood of multiple remote peers. The system consists of several smart artifacts and an underlying multi-user communication infrastructure.

Design of Social Radio

It was aimed to develop a system that enables members of small intimate groups to stay in touch with each other. Each user has several artifacts at home that represent their personal circle of friends. Each artefact represents one remote individual and displays awareness information about that person. The presence of a remote person is indicated using ambient light. In addition, an artifact communicates the mood of a remote user by re-playing the music the person is currently listening to. When several artifacts are placed next to each other, the artifacts take turns in playing music.

Since each Social Radio artifact represents an individual remote person, it is important that the artifacts can be distinguished and that each one has its own character. Social Radio offers the user three ways to personalize the artifacts. Firstly by choosing the color of the illuminated surface, secondly by choosing a symbol for the speaker part, and thirdly by attaching an individual picture. This image can be handwritten, a photo of the person, or a shared piece of memory. When the artifact is active the image is illuminated.

Tangible User Interface

In order to provide users with lightweight interaction mechanisms the artifacts are controlled via a tangible user interface. Depending on the position, an artifact is switched off or in different operating modes.

Implementation

In each location, e.g., home, the system consists of several smart artifacts and a server. The server communicates wirelessly with the artifacts and manages them. Additionally, the server retrieves information from servers in remote locations and provides information for remote locations. The server uses the public API of iTunes in order to retrieve what a user listens to. The webserver provides playlists, audio data and presence information in standard XML via internet. This decentralized approach with multiple webservers was chosen, since it enables the system to support multiple locations and potentially several intimate groups without storing all data on one server.

Acknowledgments

We would like to thank the European Commission for supporting this activity as part of the Amigo project (contract IST-004182). We also thank our colleague Jennifer Heier for designing the Social Radio artifacts as well as Eugen Berlin and Vitaliy Rapp for their contribution to the production of a first series of artifacts.

Publications

R. Etter, C. Röcker
A Tangible User Interface for Multi-User Awareness Systems
Proceedings of the International Conference on Tangible and Embedded Interaction ‘07, Baton Rouge, USA, February 15-17, 2007.
PDF

C. Röcker, R. Etter
Social Radio: A Music-Based Approach to Emotional Awareness Mediation
Proceedings of the 10th International Conference on Intelligent User Interfaces, Honolulu, USA, January 28-31, 2007.
PDF

(Social Radio on the cover of the TEI 2007 proceedings)

Flame 5 (F5) is a jacket that connects to your personal mobile via Bluetooth and enables communication via heat. If a person sends an SMS (TEXT) to a remote person wearing Flame 5, the clothing heats up depending on the personal message. Flame 5 works with standard Bluetooth mobile phones.

This work was developed at the Ambiente Group, Fraunhofer IPSI (website) by

Researcher Richard Etter
Fashion Designer Nina Marquart
Research Assistant Eugen Berlin
Research Assistant Vitaliy Rapp

SMS 2 HEAT

The jacket can be configured conveniently via the personal mobile. Depending on the person sending the sms and its contents, different parts of the shirt heat up for different durations. The wearer can configure the jacket via his or her mobile phone. The users define which words or sentences heat up which parts of the clothing for how long. This is important, since it enables users to define their own language or heat patterns that suit best to them. E.g. heat near the heart for a sms from your loved-one. Let your imagination fill in the details from there. Since the interaction shall be simple, the jacket works with standard Bluetooth mobile phones.

Today’s mobile phones use sound, vibration and light to interact with the user and use rather alerting mechanisms. Flame 5 offers mobile phones a more sensual and calm communication via heat. Heat has been chosen since it is often associated with emotional connectedness. Moreover heat can be felt unobtrusively and in the periphery of the wearer’s attention.

Technology

Embedded in the jacket is light-weight technology that allows the mobile device of the wearer to connect to the jacket wirelessly and to specifically heat parts of the clothing. The following technical components are seamlessly integrated in the jacket. A Bluetooth module that enables communication with the mobile phone. A custom-built circuit board that is attached to the bluetooth module and controls the heating elements. A small battery pack. And finally heating elements that are connected to the circuit board. When developing the hardware the focus was on minimal size of all components. All hardware that is not water-proof is stored in small pockets and can be easily detached which enables to wash the jacket.

Exhibitions

>>> The Flame 5 jacket was invited to the SIGGRAPH Unravel Fashion Show in Boston, USA. The brilliant fashion show was organized by Amanda Parkes and attracted 4600 people. SIGGRAPH is the premiere international conference on computer graphics and interactive techniques.

Flame 5 coverage in newspapers

>>> Flame 5 in Boston Globe, USA
Geek chic on the runway. Design and gadgetry interweave as fashionistas begin to embrace tech.

>>> Flame 5 in The Phoenix, USA
Flame 5 - SMS 2 Heat

>>> Flame 5 in The Tech (MIT), USA, The magic of SIGGRAPH

>>> Flame 5 in The Press Enterprise, USA, Put your tech on

>>> Flame 5 in Timeswv, USA, Hot tech: the fashion of the future

Flame 5 coverage in magazines

>>> Flame 5 in Illustreret Videnskab, Kopenhagen, Denmark

Flame 5 coverage on the web

>>> Ehrensenf “SMS-Jacke” (German Web TV Station)

>>> ASAP “Hot tech: the fashion of the future” (Interactive)

>>> Mobile Magazine “Flame 5 jacket heats up when your phone rings”

>>> Ounae “Flame 5, mensajes que encienden pasiones”

>>> Gadget Garden “Flame 5″

>>> Gizmodo Japan

>>> The Coolest Gadgets “Bluetooth-Enabled Jacket”

>>> Nextnc “Hot tech: the fashion of the future”

>>> Gadget Bloggers “Is It Hot In Here, Or Is There A Call Coming In?”

>>> El Singular “Enciende la llama de tus comunicaciones”

>>> Teckie Diva “It’s Getting Hot in Here!”

>>> DotGizmo “Flame 5 Bluetooth-enabled jacket”

>>> Shiny Shiny “Flame 5 Bluetooth heated jacket”

>>> übergizmo “Flame 5 Bluetooth-enabled jacket”

>>> Textually “Flame 5 offers mobile phones a more sensual and calm communication via heat.”

>>> Gizmología “Flame 5, tus mensajes me calientan”

>>> Technochica “Flame 5, una chaqueta calientita.”

>>> Gizmowatch “A Jacket That Senses Mobile Phones”

by Richard Etter and Bastian Zapf

Periscope is a small piece of J2ME software for your cell phone that delivers you

• news (Google News & Yahoo News)
• events (Upcoming.org)
• and images (flickr)

depending on your current location.

Periscope has been downloaded via Nokia Mosh over 10000 times! Download it via Nokia Mosh (GPS Version & Non-GPS Version)

by Richard Etter and Diana Grathwohl

AwareCuffs are the 2nd item of the techophile fashion design AwareFashion. AwareFashion are clothes that sense invisible communication technology in the surrounding. They inform the wearer in an unobtrusive and aesthetic manner about their immediate digital environment.

AwareCuffs inform the wearer about wireless hotspots in the near sourrounding. If a person wants to check if wireless LAN is available, she usually has to boot a laptop or use a Wifi-finder, which in some situations is not appropriate. Instead the wearer of AwareCuffs simply glances on the cuffs. Small light patterns that appear on the cloth indicate if wireless LAN is available and the quality of service.

Connected ?

The left and right cuff look very similar. Except that one has an attached pocket and has flexible transparent glass fibres that are woven into the cloth. When designing the cuffs great attention has been paid to make the technology invisible and to use the visible part of the technology as fashion design elements.

The wifi sensing technology is hidden in the small pocket. The circuit board in the pocket displays the quality of service by LEDs (light emitting diodes) in different colors. The LEDs are connected to glass fibres, the light travels through them and small light spots appear at the end. When the technology pocket is detached the cuffs solely consist of cloth and glass fibres and can be washed. This modular concept also makes it possible to easily replace the technology function of the cuffs. Or even, to wear the cuffs without technology at all, just because you love the fashion design.

If you would like to see more AwareFashion or want to check where to see AwareFashion live please click here. Thank you Katrin for taking the photos and many thanks to Sandra for wearing the cuffs.

by Richard Etter, Diana Grathwohl and Sigmund Homolya

Introduction - Concept - Scenario - Shirt 1.0 - Festivals - Press Coverage

Introduction

Today an ubiquitous digital layer lies over the real world. Cities, houses, cars and people are increasingly connected to each other. Communication technology becomes smaller, wireless and invisible. The consequence is that communication technology is omnipresent, but we lack of means to be aware of it. Everyday we ask us questions like: Am I recorded? Am I alone? Do I have WiFi here? Am I connected? Can I call for help

Concept of AwareFashion

AwareFashion are clothes that answer the above questions. They react to invisible communication technology in the surrounding and thus enable the wearer to sense their immediate digital environment. AwareFashion has also a social dimension. Since most people wear mobile devices AwareFashion also enables the wearer to sense the presence of other people.

Scenario

The staff working for an opera has to make sure that every mobile phone is switched off. Instead of annoying announcements via the communication system the staff wears shirts that slightly glow at the end of the sleeves when switched-on mobiles are near. This enables them to perform various tasks while at the same time being aware of activated mobiles in their surrounding.

AwareFashion Shirt 1.0

a shirt that senses mobile phones

Design and Implementation

As an example of AwareFashion we designed a shirt that senses switched-on mobile phones in the proximity. The shirt looks like a fashionable normal shirt, however if mobile phones are near the shirt glows unobtrusively at the end of sleeves. The design is orientated at the human sensory perception. Humans sense their environment with sensory cells, that generate electronics impulses which travel though nerves tracts and are processed.

Similar to this concept we designed a shirt that is augmented with an antenna, a tiny circuit board, button cells and fiber optics woven into the cloth. The antenna detects radio waves of GSM mobiles (900MHz). The custom-build circuit board processes the radio waves and converts them to light. The light travels through flexible fiber optics to the end of the sleeves. There the light emits and indicates the presence of near mobiles. The detection range of mobiles is currently fixed, but we are working on a solution that enables the wearer to adjust the range by himself. This will further increase the usability of the shirt.

In regard to the design of the shirt special attention has been paid to avoid a technical impression. People wearing the shirt shall feel comfortable and not as a cyborg. Therefore the antenna and circuit board are small and lightweight (future circuit boards will be based on small SMDs). Furthermore these two components are hidden in a pocket that corresponds to the fashion design of the shirt. Also the fiber optics that are sewed in the cloth are used as fashion design elements, they resemble tribal tattoos. When a mobile is near not the whole shirt begins to glow since people would eventually mistake the wearer for an alien. Instead small light spots appear at the end of the sleeves. The wearer can check this place easily while performing other tasks. The color of the light can be chosen by the wearer, so he can match it to his style. He can choose red, green, yellow, white or blue.

The pocket can be easily attached and detached from the shirt. When the pocket is detached the shirt can be washed since it only consists of cloth and highly flexible fiber optics. There are no electronic contacts. After washing the pocket can be attached with push buttons. The bunch of fiber optics can be connected conveniently with a plug.

The flexible pocket system enables us to use the shirt with other technical units with different sensors and functionality. To see another item of the AwareFashion collection please click here.

Festivals

>>> AwareFashion will be presented at the renowned VIPER FESTIVAL. The technophile fashion design was nomiated in the category ‘processing’ and can be seen live in Basel, Switzerland. VIPER is one of the major European film, video and new media festivals.

Press Coverage of AwareFashion

AwareFashion has been mentioned by several newspapers and a variety of weblogs around the world. Now you can read about it in your favorite language Please find selected articles below.

>>> The AwareFashion Shirt will be featured in the forthcoming book “Mobile Audience” by Martin Reiser.

>>> An interview and article about AwareFashion appeared in Mobile & Wireless (Prag, Czech Republic).

Download

>>> The newspaper Electronics Weekly (London, UK) reported about AwareFashion in their last issue! With 43000 subscribers Electronics Weekly has an unrivalled reputation in the electronics community in the UK.

Download

>>> LARGEUR “Cette chemise … n’entache donc ni le look ni le bien-être.”

>>> Engadget “AwareFashion project provides cellphone alerts via clothing”

>>> EYEBEAM Art and Technology Center, article Glimpse of what’s to come

>>> Textually “The shirt that senses switched-on mobile phones”

>>> Smartmobs “Aware Fashion: Clothing That Detects Wireless Devices”

>>> Softpedia “Camasa care detecteaza activitatea celularelor” (Italian)

>>> Smartmobs “The shirt that senses mobile phones”

>>> Ira Vehementi “Detecta teléfonos móviles con tu camisa” (Spanish)

>>> ITNEWS.SK “Košela, ktorá cíti prítomnost mobilov” (Slovak)

>>> Mobile Bulgaria (Bulgarian)

>>> Onliner - Digital Lifestyle (Russian)

>>> MPort (Russian)

>>> Share you life AwareFashion… aware! (French)

>>> Charshija Queen Da ne bi sta propustili (Bosnian)

>>> mobileblog.it - “Illuminami i polsini” (Italian)

>>> Sensory Impact - “This week’s glimpse of whats to come”

(Pictures by Katrin Felicitas Schmid)

In conjunction with PERVASIVE 2005, 3rd International Conference on Pervasive Computing. Maritim Hotel, Goethestr. 7, München, Germany, 11th of May 2005.

Motivation for the workshop

Mobile contexts are a challenge for context-aware systems. In contrast to fixed indoor contexts, the mobile context is less predictable and more dynamic. Systems that understand the context of a mobile person and his environment and adapt to it can be a great benefit for the person. Especially in a mobile context auditory displays provide a variety of possibilities.

According to the current context a mobile system can:

The goal of the workshop is to address the potential of auditory displays for mobile context-aware systems. The workshop discusses different inference-models that adapt the usage of auditory displays and the type of information displayed according to the mobile context. In this workshop works from the background of the auditory display community and the ubiquitous and pervasive computing research will be presented. A moderated discussion on integrating audio displays for mobile context-aware systems will follow.

Please find more information on the workshop website.

by Ricarda Betz, Richard Etter, Stefan Nandzik and Marco Della Pina.

The short film ‘Sitzen’ (German for sitting) plays with words and locations. The film was shown at the International Media Festival Villingen-Schwenningen.

Trailer (40 seconds)