Transcribing multimodally

By Emilee Moore, TLANG team, University of Leeds

Emilee Moore will be a visiting research fellow until the end of 2017 in the School of Education at the University of Leeds, linked to TLANG. Emilee is funded by the Government of Catalonia’s Beatriu de Pinós fellowship program. She is also a member of the Research Group on Plurilingual Interaction and Teaching at the Universitat Autònoma de Barcelona.

In her blog post on March 3rd, TLANG researcher Frances Rock wrote about how the fieldwork tools being used by the teams have been reconsidered and adapted to different sites throughout the project phases. In her blog post, Frances explains that in her team’s case, video recording was employed much earlier and more often in their research with young footballers, than had been the case at previous sites. She also touches on how not being able to video record in another site (a second sports club) during their Sports phase fieldwork led to the methodological challenge of how to document multimodal interaction in other ways (in their case, they employed drawings), and she reflects on the analytical implications of this.

Screen Shot 2016-03-15 at 13.41.49.png

Synchronised data from Leeds basketball site (2 cameras, 1 external audio)

The issue of how to analytically approach multimodal interaction, and especially the video data collected during the Sports phase, is one currently facing all of the TLANG teams. The Leeds team, for example, has collected video data from basketball and capoeira sessions. Viewing the data, it becomes clear straightaway that spoken language is just one – possibly not the predominant one – mode of meaning making in these settings. The intricate ways in which bodies coordinate in space and time with other bodies, to the rhythm of the berimbau, to song, to the bouncing and throwing of balls and to ‘non-speech’ sounds exchanged across courts (e.g. grunts), stand out immediately. A question that all teams raised at the last TLANG meeting in Birmingham was how to turn video data of this nature into more dynamic transcriptions that would allow the complexity of multimodality to be accounted for.

Screen Shot 2016-03-15 at 13.42.22

ELAN transcription of a capoeira lesson from the workshop

Responding to this challenge, the Leeds team organised the first of two full-day workshops on the transcription of multimodal data on March 14. The sessions have been opened up not only to TLANG members, but also to other researchers facing similar methodological and analytical challenges. In the session, we started with basic technical aspects of working with video, such as how to join and synchronise different media files. We then worked through a first analysis of some video data in order to share perspectives on what elements of the multimodal interactions we are working with should be accounted for in transcriptions. After that, we practiced with two different video transcription programs, ELAN and Transana, and weighed up their affordances and limitations for our research. Finally, we discussed different ways of presenting multimodal transcriptions on paper and in oral presentations, and considered the ethical challenges of presenting videos and still images and some solutions.

Screen Shot 2016-03-15 at 13.41.59

Subtitled video data from the Leeds capoeira site

Anybody interested in participating in the session coming up in April should contact Emilee Moore from the Leeds TLANG team ( . We are currently working to make the materials available online.


2 thoughts on “Transcribing multimodally

  1. Emilee ran a brilliant workshop which she had obviously put a lot of effort into organsing. Thanks to her and John for their super work on this. The next workshop comes highly recommended!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s