Listen to the art — a story about making art visible to people that can’t see.

The ŁAŹNIA Centre for Contemporary Art was one of the first public cultural institutions established in Poland after the transformations of 1989. It began as an initiative by local independent artists opposed to traditional models of art. In 2012, the gallery was one of seven Polish galleries listed among the 400 top places showing contemporary art in the world, by the prestigious Art Spaces Directory, published by New York’s New Museum of Contemporary Art in collaboration with ArtAsiaPacific.

In 2016 Laznia came to us with an idea to make art be visible for visually impaired and blind people. We’ve found this idea very interesting, so we started a research to make this plan become true.

In the beginning, Laznia employees Anna Meronk and Noemi Etush tell us more about their idea. They have seen the need to eliminate the social exclusion of blind and visually impaired people from participating in culture. Blind and visually impaired people want to actively participate in cultural events, however, often institutions are not prepared to accept people with such disabilities. The idea was to involved volunteers (previously trained) to be their eyes. Second thought was to ensure that where there are no volunteer people can also use the application.

Our task was to implement the idea into life.

About the app.

ArtSee application is addressed to blind and visually impaired people. The application is easy and intuitive, and its launch will be shortened to two clicks. The app works on a double click (the first click says what is under the given button, the second confirms the selection). Therefore, knowledge innovation and technology is not very necessary. CSW Łaźnia’ve purchased a few mobile devices, so that the fact of not having a smartphone, was not an obstacle in using the application.

The application is built in accordance with the requirements of designing applications for people blind and visually impaired.

A catalogue describing the project.

Functionalities.

Artsee app works on ANDROID. The application it is practically devoid of the selection and content function, after entering the application — it just turns on and it starts working. The blind person, joining CSW Łaźnia, will turn on the ArtSee application and will be led through the application from the entrance to the building to the exit.

In the first variant, a blind person will be assisted by a volunteer who in charge using the camera on the blind person’s phone, it will describe what he sees on the screen -this applies not only to the exhibition part but also to the whole environment. After enter to the exhibition part, the volunteer describes to the blind person what he sees.

An additional solution is the transmitters used in space — iBeacons send a signal received by telephones and cause a reaction in the form of notifications. iBeacons are supposed to perform an auxiliary function first — if a blind person, pointed the camera so that the volunteer will not see the item accurately, thanks iBeacon, volunteer knows exactly where the person is located.

In the second variant, when the volunteer will not be available, the application using beacons and audio transcription guides the user in the immediate surroundings, will describe what the interior looks like and it will include appropriate audio description adapted to the location of the person using the application.

Both variants combine the function of audio descriptive greeting and space description. After an audio recording or a person will connect with a volunteer or in a case unavailability of the volunteer — further audio description, run with beacons.

Audio description of the space of CSW Łaźnia is permanent, and the audio description of the exhibition space will change depending on the exhibition.

Our programmer during the fight with iBeacons.

The look.

FOR BLINDS and visually impaired

The application is built in accordance with the requirements of designing applications for blind and visually impaired people. The application consists of two views:

1. first (after opening the application) with the icon “I need help” and
2. second — connecting with volunteer or if the volunteer is unavailable, activating audio description along with a guide around the room.

FOR TRAINED VOLUNTEERS

A trained volunteer has access to the service panel available through a website that will work on computers and mobile devices. Every time the volunteer will have access to all hints related to the current exhibition and the position of the visitor through the calculated triangulation of iBeacons.

Scenarios of using the application

1. Blind person is guided by volunteer

A blind person enters the Łaźnia and Contemporary Art Center and on his device, he indicates that he needs support. Each of the volunteers gets prompt (notification on the phone and browser or SMS). The first available

the volunteer indicates the willingness to help, so he connects directly to the person needing support. A volunteer can help one person at the same time.

The above session includes a one-to-one connection. The number of sessions is not limited — it means that at a given moment there may be several people guided by several volunteers. After entering the room, by detecting the main iBeacon,the person being guided is played a sound file describing the room. During the time the volunteer prepares for the guided tour. The blind / visually impaired person directs the camera of his phone to places prompted by a volunteer. A volunteer describes via audio description artifacts seen on his screen. Volunteer in addition to the “live” view from the camera the guided person, sees the approximate location of the person guided on the base triangulation calculated by IBeacons.

2. The blind person is guided by iBeacons and audiodescripiton

After entering the room by detecting the main iBeacon a sound file is played describing the space. When moving around the room, next recordings are being played.

Technology

Whole technology stack is based opon WebRTC which enables peer-to-peer direct communication between people who visits the exhibition and curators that describe the art.

WebRTC (Web Real-Time Communication) is a free, open-source project that provides web browsers and mobile applications with real-time communication (RTC) via simple application programming interfaces (APIs). It allows audio and video communication to work inside web pages by allowing direct peer-to-peer communication, eliminating the need to install plugins or download native apps.

Back in 2016 to only way to get WebRTC working on android was crosswalk which is not maintained anymore, since the API is available with latest Chrome and Firefox for a while. Admin panel use node.js, express and React. Client application use React, socket.io and WebRTC wrapped in Cordova mobile container.

Admin panel use JavaScript iBeacons library that was delivered by kontakt.io hardware provider.

At the end of 2017 when Progressive Web Apps PWA had grown mature we added those feature, so at the moment user can experience without installing on their device.

Resources

1. Artsee website. This is a way to arrange a curated visit
2. Project description website