Accessible Live Casting, Almedalen 2010

Updated 2010-12-06 (and please consider the far more recent solutions concerning remote live sign interpretation).

The 2010 Almedalen experiment for accessible live casting is a project conducted by Westreamu AB, commissioned by the Swedish Disability Federation (HSO) and partially funded by The Swedish Post and Telecom agency (PTS).

This document describes the technology used and briefly summarizes the background and results. A full report is available (in Swedish) at westreamu.se/lab/pts.

Summary

During the Almedalen political week July 4-10, 2010 we conducted several experiments in order to improve live casted Web video with respect to accessibility. Our ambition was to develop and try new means in order to enhance the access for primarily people with hearing challenges.

In summary we successfully implemented novel technologies that featured:

  • live closed captions (show/hide subtitles)
  • live closed signing (show/hide signer)
  • remote captioning as well as signing (interpretation from a distance)

 

The tools and services we used include:

 

Background

The Almedalen week is an annual event where Swedish politicians, lobbyists, various organizations, and the public meet in a relaxed and open format. The whole event takes place in Visby, a county town situated at the Swedish island Gotland. This year the Almedalen theme was accessibility. Throughout the week over 300 seminars were arranged, of which more than 100 were broadcasted live on the Web.

Handikappförbunden (HSO, the Swedish Disability Federation), arranged nine seminars. Seven of these were interviews with representatives from leading political parties. The remaining two sessions were discussions about technology for accessible Web TV. One of these was a panel discussion with representatives from Hard of Hearing Young People (UH), Swedish Deaf Youth Association (SDU), Swedish public service TV (SVT), The Swedish Post and Telecom agency (PTS), and Swedens Ministry of Culture. All seminars, except the panel discussion, took place in a tent in the port of Visby.

Handikappförbunden’s aim was not only to live cast their seminars in an accessible fashion, but also to inspire others to start using these new possibilities. The latter group includes traditional TV broadcasters.

Some of the extra costs for the live casts we produced, and the technology developments required, has been funded by The Swedish Post and Telecom agency.

Results

To paraphrase an old proverb, a video is worth more than a million words. So let us start by watching how it looked like from the Web audience’s point of view. During the live events viewers could choose a player with captions or one with signings. Two videos, recorded from a web browser, shows the type of accessibility that each player offered.

The first video illustrates how the captions could be switched on and off by the individual viewer.  Featured in the short clip below is the use of an embedded player from Livestream. That live broadcasting platform is one of the few that support SUBply, the live closed captioning service we used.

In order to provide closed signing we developed a special Flash player. Below the second video illustrates the functionality of this player. That player has since been developed further with a better design and support for on-demand viewing. It will soon be released as “Open source”.

We also made local recordings of everything we live casted. Immediately after each seminar its video was published for on-demand viewing at hso.blip.tv. Later on we also uploaded re-synchronized caption files and post produced videos where the signers were added using chroma key technology. This material can be viewed in full versions at www.hso.se/webbtv.

Unique features

There are mainly three things that are unique in this experiment

  1. Open, flexible, and cost effective solutions
  2. Subtitling and signing from remote locations
  3. Closed signing (video and signer in separate streams)

 

We have used Flash as the underlying platform. Even though Flash is a proprietary technology, (owned by Adobe) it is the most used platform, widely available for web video solutions. All the software solutions we have used are easily available for general use. The applications we have developed in this project, mainly for closed signing, will soon be released as open source.

When building our live casting platform we have generally tried to use low priced software as well as hardware, mostly consumer electronics. Whenever accessibility is neccesary, that of course implies some extra costs. These mainly relate to professional subtitlers, signers, and web services ( like in our case Livestream and SUBply).

The use of remote subtitlers and signers facilitate increased flexibility and cost reductions. Particular in terms of reduced travel costs and setup times.

The possibility to have closed signing has several potential advantages. The most important is that the viewers may select if they want to see the signer, where it should be placed in the video, and how large it should be. Furthermore, separating the video from the signer might also make post production and on-demand publication easier.

Further use

Even though the focus on this project is on web video distribution, some features would most likely be useful also for live broadcasting of traditional TV. Even though terrestrial, satellite, and cable distribution cannot provide the same technical means that we used, there are still values to be obtained from a production standpoint. These include primarily the use of remote subtitling and signing.

Worth noting here is that we have focused on live casted video consumed via web browsers on personal computers. Mobile platforms (smartphones) do not yet provide widely available low cost solutions for neither (live) subtitling nor live signing. Our current trade-off is that the on-demand videos we published can be watched on mobile phones, albeit currently not with subtitles. Adding subtitles can easily be done, but lies outside the budget of this project.

Technical descriptions

Again a video is probably useful as a starter. This clip shows how it looked like in the tent where we produced the interviews.

The basic production setting

For all productions we used VidBlaster, a powerful yet affordable video production application for Windows. For the main video we used VidBlaster’s Broadcaster version, and for the sign channel the Professional version. The former was needed in order to conveniently transmit two video streams simultaneously. Similar tools like Tricaster, Wirecast, Ustream Producer, and Livestream Procaster could probably be used for these kinds of productions.

 

Illustration of the basic production settingA schematic overview of the equipment we used is illustrated in the image. Cam1 was an old Sony camcorder, used as our main camera. With it we zoomed and panned over the “stage”. Cam2 was a Logitech webcam, used as a fixed camera for overview. Cam3 was another Sony camcorder, used as a fix cam on the signer. Audio from two lavaliere microphones and one hand mike went to a mixer for PA and our video production. PC1 was a powerful gaming laptop that mixed Cam1, Cam2, and audio from the mixer with pre-designed lower third graphics and pre-recorded video clips. PC2 was a less powerful entertainment laptop, only used to mix Cam3 with audio. Both computers were connected to the internet via a switch hooked up to an Ethernet router provided by TValmedalen.

Live and remote closed captioning

Illustration of the setup for live closed captioningThis image illustrates how live captions were added to our video. In summary we used the SUBply live captioning service in connection with Livestream. In our production setting (1) one of the video streams from PC1 was sent to the Web broadcasting service Livestream. A professional subtitler (2) situated in Stockholm (200 km from Visby) watched the stream slightly delayed in the web based subtitling application provided by SUBply (3). The subtitler used a Veyboard keyboard in order to enter the text as quick as possible. Each text block entered was immediately sent by SUBply to Livestream. Viewers could then watch the result on the Web in Livestream’s player (A), with or without subtitles.

Live closed signing

Illustration of the setup for live closed signingIn order to provide live closed signing we had to setup a unique solution, illustrated in this image. The Livestream player (A) got its video and subtitles as described earlier. In addition we developed a special closed signing Flash player (B). From the production setting (1) PC1 also sent a video stream to a channel on Bambuser (another web broadcasting service). This stream became the background video in the signing player (B). PC2 sent its video stream to another Bambuser channel, which became the foreground video in the signing player (B). As the image suggests, subtitles was unfortunately not available in the signing player.

Remote live open signing

Finally, we tried remote open signing at two occasions. This image illustrates how that was setup. PC1 in the production setting (1) was running the video chat application ooVoo (1a). The advantage with chat video is that it is in real time, at the cost of somewhat reduced image quality and frame rate. Similar applications, like Skype, tend to have the same drawbacks.

Illustration of the setup for remote open signingPC1 sent the final video/audio mix to ooVoo. At Tolkcentralen in Örebro (Signing central, 300 km away from Visby) a sign interpreter watched the video in real time using ooVoo’s web application. In front of a web cam the signer interpreted what was said directly into a video window on PC1 (1a). This window was grabbed by VidBlaster and mixed into the final video production sent to Livestream (A), as well as the Bambuser channel for the background video in the signing player. As can be seen in the illustration, the signer always appeared on the video for all viewers, including the video the signer in Örebro saw. We still have to determine if the frame rate over ooVoo is sufficient enough to capture all signs.