With this week’s exploration, we took a look at a few of the ways we can attach textual information to visual information. Without being able to explicitly attach contextual information to an image or video, we lose out on entire readings of works and ways of relaying interpretations to future viewers. This also becomes a problem of accessibility in multiple senses of the word. By attaching textual information to images, video, and even audio, they become discoverable. Text can be indexed in a way that non-textual information cannot, and when we are able to flag specific areas of note, we create a more tangible record of what that non-textual information contains. In the second meaning of accessibility, the presence of text as an interpretive layer allows for greater disability access. Subtitles allow for those hard of hearing to engage with video and audio, and for those with visual impairments, the presence of text is necessary for a screen reader to function. By including avenues to attach additional information to image and video viewers, it theoretically becomes easier to expand scholarship on specific works.
In his article “Annotating Upstream: Digital Scholars, Art History, and the Interoperable Image,” Matthew J. Westerby lays out the potential benefits of using annotations to build on analysis across multiple scholars in addition to his more technical descriptions of the functions of IIIF. The field of art history particularly benefits from being able to embed analysis and research findings directly into images (and even 3D models); it allows scholars to point to exact details, filling in missing context and elaborating on often sparse museum-created metadata. The IIIF viewer also allows for a higher resolution of viewing, which encourages close looking and analysis. By being able to reuse, share, and disseminate high quality interoperable images, we encourage the use of “thick data.” Of course, as with many digital projects, there are a number of challenges and shortcomings of this approach that Westerby identifies. Firstly, there is no centralized platform to share this thick data on; analysis can be built upon by pre-existing networks of scholars, but its reach is limited. If we want to annotate upstream in the truest sense of the phrase, there has to be an upstream to begin with. There are of course user-created workarounds and plugins to share annotations, but it can be intimidating for less technologically-inclined scholars to integrate into this workflow. And of course, it almost goes without saying at this point that maintaining thick data in this multi-part form is a digital preservation challenge because of its many moving parts and semi-obscure file types.
We got to try a little bit of video annotation ourselves to see how it looks in practice through Panopto. After the Westerby article, I was curious to see what more art historical annotations would be like, so I brought in a video from my favorite artist duo: Janet Cardiff and George Bures Miller. If you are unfamiliar with them or have never had the privilege of seeing one of their pieces live, Cardiff and Miller construct looping, immersive, multi-media narrative works featuring animatronics/moving parts that are sometimes contained to singular rooms or scenes. I originally considered annotating my absolute favorite work of theirs, Opera for a Small Room, but unfortunately it doesn’t film particularly well. There’s a moodiness and a physicality that just doesn’t come across in video; not being able to move around the piece and feel the sound breaks the immersion (and Cardiff and Bures Miller rely a LOT on sound!). I had originally thought that maybe annotation could fill some of those gaps, but after annotating, I was a little frustrated that it just couldn’t replicate the experience of watching this piece. So, instead, I looked at The Marionette Maker, which is a similar piece from Cardiff and Bures Miller with a less direct, linear storyline.
While this video representation is still not perfect and misses a lot of the aura of the original piece, I enjoyed being able to mark it up like you would be able to for a static piece. Since there are no spoken words in The Marionette Maker, I added a few bits of analysis and transcriptions for the general soundscape. I have also edited the video down to a more appropriate length for this example from the full 11 minute and 11 second video available on the artists’ website. Please forgive some of the awkward cuts; I tried to cut down on the amount of time the camera lingers on each setpiece, but had a bit of a hard time getting precise in Panopto. If you’re interested in the full video, here is the youtube link (and some higher resolution photos so you can see the craftsmanship!). Don’t forget to open this video in Panopto to see the added notes.
Practicing annotating videos led well into our discussion on oral history. While a small number of traditions prefer to focus on the purely aural aspects of a recorded interview, most rely on transcripts and annotations to make their content accessible (and indexable). As someone who has been working in a small museum for two and a half years, I’ve had direct experience with how vital (and difficult) it is to make oral histories accessible. Since the museum is owned by the City of Raleigh, we face specific municipal pressures to have a collection that is of and for the people of the city; this means that when we accept anything into the collection, it’s important that we are able to make it accessible to laypeople and not just researchers. However, because we don’t have transcription software, every bit of typing and indexing we do is entirely by hand. It takes a lot of work (and a few janky workflows) to get the dozens of oral histories in our collection into a state that is even remotely searchable.
Because of this, we have experimented with automated transcription for a project interviewing former students of Ligon High School. For some quick background, J.W. Ligon Junior-Senior High School was one of two of Raleigh’s black public high schools, and considered to have a particularly excellent teaching staff; many students went on to become STEM leaders in the Triangle area. The school operated as a junior/senior high school for 18 years before the desegregation of Raleigh public schools, and every year on the weekend of July 4th, alumni gather for a reunion. Some of these people have donated to the city’s historic collections before, so we have been trying to have a presence at these events, putting together small exhibits of Ligon items we have in the collection and operating an oral history station. Although the interviews only last 10-15 minutes, we still get about a dozen per year. Recently, when we had a several year backlog, we attempted to do some automation with our transcripts, which honestly did not save us much time. Transcription software tended to have a very difficult time deciphering the heavy accents of our interviewees, and it did awfully at understanding place names, so we had to go back in and basically rewrite the transcripts. It also of course did not understand any of the contextual information that we understand as stewards of this history- how could it possibly know that “the rat box” was what people called the movie theater on Hargett Street and index it appropriately?
I have a lot to say about oral history as a practice but in short, without being able to add that context back in, holes start to appear in the historic record.
Westerby, Matthew J. “Annotating Upstream: Digital Scholars, Art History, and the Interoperable Image.” Open Library of Humanities 10, no. 2 (2024). https://doi.org/10.16995/olh.17217.
Leave a Reply