Post by Richard Hewett, University of Salford

This post continues the ongoing “From Nottingham and Beyond” series, with contributions from faculty and alumni of the University of Nottingham’s Department of Culture, Film and Media. This week’s contributor is Richard Hewett, who completed his PhD in the department in 2012.

Alfred Hitchcock must rank as one of the most discussed and documented filmmakers of the 20th century, the number of books that focus on his life and work having turned into a cottage industry since his death in 1980. This year alone has seen the publication of a second volume of Sidney Gottlieb’s collected interviews and short stories, Hitchcock on Hitchcock, Alain Kerzoncuf and Charles Barr’s Lost and Found: The Forgotten Films, and a new biography by Peter Ackroyd. Over the years a number of screen documentaries have also been produced, the latest of which, Kent Jones’ Hitchcock/Truffaut, premiered at Cannes last month.

The director clearly continues to exert the same fascination that he inspired during his lifetime, but on television, at least, the love affair between documentarists and Hitchcock now risks falling into the same trap as Ackroyd’s biography: having nothing new to say, and no new way of saying it. As Hitchcock TV profiles are legion, I will focus here on those made specifically for British television. What emerges is a clear picture — in creative terms, at least — of diminishing returns, though it seems with no similarly diminishing interest in the man himself.

Prior to Hitchcock’s death, the number of UK documentaries were few in number, the filmmaker featuring more regularly in extended interviews for arts-centered magazine programs, many of which have since been culled (repeatedly) for biographical features. Unsurprisingly, Hitchcock’s visits to British TV studios were inevitably timed to coincide with the release of his latest cinematic work. Thus we have a 1960 interview for Picture Parade (BBC) (the year of Psycho’s release), a 1964 interview with Huw Wheldon on Monitor (BBC) (coinciding with Marnie), appearances on Profile (BBC) and Late Night Line-Up (BBC) from 1966 (surrounding Torn Curtain’s release), a 1969 NFT appearance (BBC), being interviewed by Bryan Forbes (circa Topaz), and so on. These appearances are notable primarily for the control Hitchcock himself exerts, both subtly and otherwise. Whether or not the questions were pre-screened is impossible to say, but the director is seldom led along any conversational routes he does not wish to pursue, with the result that the same anecdotes are repeated, sometimes verbatim. The one British TV documentary made during his lifetime, the 1972 Aquarius (ITV) entry “Alfred the Great,” is little different, featuring a combination of studio interview material and footage of Hitchcock directing the opening sequence from Frenzy (1972). Here, as elsewhere, Hitchcock both literally and metaphorically calls the shots. The sole occasion on which the director is visibly challenged in any of these appearances is, interestingly, the only one to take place in front of a live audience. In his NFT interview, Hitchcock effortlessly works an appreciative crowd until interlocutor Bryan Forbes, having pointed out that Hitchcock seldom concerns himself with social consciousness, takes issue with Hitch’s repetition of Samuel Goldwyn’s maxim that “messages are for Western Union.” Forbes cuts short the resultant rumble of appreciative laughter, curtly stating, “Yes; I don’t think the applause is actually well placed, because not all films that fall into that category are necessarily bad films, and Goldwyn was getting a cheap laugh, really, which is echoed here…”

At this point, Hitchcock visibly cools.

This moment does not appear in the majority of the television documentaries made following Hitchcock’s passing, though the stories he consistently regurgitated during his lifetime are, perhaps inevitably, employed as a framework upon which to build. Three examples stand out in this respect: the two-part Omnibus (BBC) documentary from 1986; another two-part entry in the Reputations series (BBC) in 1999 (Hitchcock’s centenary year); and Living Famously (BBC) from 2002. Each relates, often via reliance on identical archive-interview footage, the same tales: the infant Hitchcock being scared by his mother saying “Boo!”; Hitchcock’s fear of “everything,” but in particular policemen — the latter a result of being locked in a prison cell (at his father’s behest) while still a child; explaining the difference between suspense and surprise; bemoaning his lack of technique for having let the bomb go off in Sabotage (1936); and explaining that he never said actors were cattle, only that they should be treated as such.

It is entirely natural, of course, that documentary-makers should rely on such primary material, even if much of Hitchcock’s self-celebration goes unchallenged. His statement that after an initially unsuccessful screening for studio bosses, breakthrough film The Lodger (1926) was conveniently left on the shelf for three months before being dusted off and hailed a classic conveniently ignores the fact that Ivor Montagu was brought in to re-edit the film in the interim, removing several of the title cards. However, when the Omnibus and Reputations entries were made, there had been no significant UK television documentary focus on Hitchcock’s career for several years. (The BBC had in 1997 mounted a Close Up on Hitchcock retrospective, which featured actors and directors briefly discussing his work.) The inclusion of such notable (and now departed) talking heads as James Stewart, Joan Fontaine, Ann Todd, Teresa Wright, Farley Granger, Janet Leigh, Charles Bennett, Samuel Taylor, Hume Cronyn, Arthur Laurents, John Michael Hayes and Ernest Lehman provides valuable archive material, which would in turn be plundered for later such efforts.

There is also a sense, with these documentaries at least, of new discoveries and revelations waiting to be made — even if some of these later transpire to be false. Saul Bass’ claim, in Omnibus, that he in fact directed the Psycho shower sequence stirred up a storm of angry protest from many who had worked on the film (though Bass had been responsible for the storyboards), while Tippi Hedren’s reflection, in Reputations, on the post-Marnie rift provided a much-recycled sound bite: “I am totally responsible for it… No; I’m not, he is!”

By the time of Living Famously, this talent pool of primary sources, alas, starts to dwindle, and the same stories and reflections begin to derive from biographers such as David Freeman and Donald Spoto, or critics including Barry Norman, though the indefatigable Patricia Hitchcock O’Connell is still present to defend her father’s (and mother’s) reputation. There is perhaps a sense of the well running dry. As with the 1960s interviews conducted with Hitchcock himself, few new questions are asked. It is almost as though Hitchcock continues to exert editorial control over his legacy from beyond the grave, much as he did with the montage piecing together of shots in his movies. Given the paucity of new biographical detail, the sensible approach — and the one taken by the two most recent original TV documentaries, Paul Merton Looks at Alfred Hitchcock (BBC, 2009) and Jonathan Ross’s Perspectives: Made in Britain (ITV, 2013) — is to focus on lesser-addressed areas of the director’s life. Perhaps in a tacit admission that the old style of expository voiceover documentary has run its course, each of these takes a familiar media face and places it onscreen to deliver a personal take on the great man, both focusing on the British period, which allows an in-depth look at the more obscure (in terms of television airings) early work. Merton humorously acknowledges his reliance on archive footage by splicing himself into the interviews, introducing an irreverent note of which Hitchcock himself might have approved. The documentary closes with Merton loosening the necktie that he has (atypically) worn throughout the program, intimating that he is about to perpetrate the strangling (though presumably not rape) of Hitchcock in the style of Frenzy’s psychotic killer. Ross takes a somewhat different tack, using his and Hitchcock’s shared geographical origins (each hail from Leytonstone in East London) to unpack both the director’s early life and Ross’ personal relationship to his work. Though this verges on a vanity project at times, it at least introduces some new material, including Ross demonstrating how the effects shot from Blackmail (1929) was achieved.

And that, for the moment, draws a curtain (though not a torn one) over original Hitchcock documentaries on mainstream British television. The recent Talking Pictures entries, on Hitchcock and his leading actors (BBC, 2014, 2015), apply an entirely cut-and-paste approach, linking much archive footage that did not feature in previous programs (including a large segment of the NFT interview), with Sylvia Syms’ knowing voiceover. This is perhaps as far as can now be travelled in terms of a biographical or career overview, but British television documentarists of the future would do well to take a leaf out of Kent Jones’ book, or from Michael Epstein’s documentary Hitchcock, Selznick and the End of Hollywood (1999), to focus on particular periods, themes or relationships in Hitchcock’s career. Even the documentary extras which have proliferated on DVDs and BluRays over the last 15 years tend to address mainly the American films, and it would be gratifying to see a Hitchcock season — perhaps on BBC Four, which occasionally re-airs the Reputations and Living Famously material — that spotlights lesser-examined works, such as the wartime propaganda films (Bon Voyage [1944] and Aventure Malgache [1944]), the comedies (The Farmer’s Wife [1928], Mr. and Mrs. Smith [1941]) or (comparative) failures such as The Paradine Case (1947) and Under Capricorn (1949).

Until then, the Hitchcock TV documentary furrow would seem to have been definitively plowed.

Lantern is a search and visualization platform for the Media History Digital Library (MHDL), an open access digitization initiative that I lead with David Pierce. The project was in development for two years, and teams from the MHDL and UW-Madison Department of Communication Arts collaborated to bring version 1.0 online toward the end of July. After a whole bunch of testing, we decided that the platform could indeed withstand the scrutiny of the blogosphere. It’s been a pleasure to see that we were right. We’re grateful for supportive posts from Indiewire and David Bordwell and web traffic surpassing anything we’ve experienced before.

I will leave it in the capable and eloquent hands of David Bordwell to explain what the searchability of the MHDL’s 800,000 pages of books and magazines offers to film and broadcasting historians. In this Antenna post, I wanted to more broadly touch on how the search process works. I will address visualization more fully in another post or essay.

We run searches online all the time. Most of us are inclined to focus on the end results rather than the algorithms and design choices take us there. Cultural studies scholars such as Alexander Halavais have offered critical commentary on search engines, but it wasn’t until I began developing Lantern in 2011 that I bothered to peek under the hood of a search engine for myself. Here are five lessons I learned about search that I hope will prove useful to you too the next time you search Lantern or see a query box online.

1. The collection of content you are searching matters a lot.

It would have been great if the first time Carl Hagenmaier, Wendy Hagenmaier, and I sat down to add fulltext search capability to the MHDL’s collections we had been 100% successful. Instead, it took a two year journey of starts, stops, and reboots to get there. But in other ways, it’s a really good thing that we initially failed. If we had been successful in the Fall of 2011, users would have only been able to search a roughly 100,000 page collection comprised primarily of The Film Daily, Photoplay, and Business Screen. Don’t get me wrong, those are great publications. And we now have many more volumes of Photoplay and The Film Daily than we did back then. But over the last two years, our collections have boomed in breadth and diversity along with size and depth. Thanks to our partnerships with the Library of Congress Packard Campus, Museum of Modern Art Library, Niles Essasany Silent Film Museum, Domitor, and others, we have added a tremendous number of magazines, broadcasting, early cinema journals, and books. In 2011, a search for “Jimmy Stewart” would have probably resulted in some hits from the fan magazine Photoplay (our Film Daily volumes at that time didn’t go past 1930). Today, the Lantern query “Jimmy Stewart” yields 407 matching page hits. Take a look at the top 10 results ranked by relevancy. Sure enough, 5 of the top 10 results come from Photoplay. But there are also matching pages from Radio and TV Mirror, Independent Exhibitors Film Bulletin, and International Projectionist — all sources that a James Stewart biographer probably would not think to look. And who would guess that International Projectionist would refer to the star with the casual “Jimmy”? These sorts of discoveries are already possible within Lantern, and as the content collection further expands, there will only be more of them.

2. Always remember, you are searching an index, not the actual content.

This point is an important caveat to the first point. Content matters, but it is only discoverable through an index, which is itself dependent upon the available data and metadata. A search index is a lot like the index at the back of a book — it stores information in a special way that helps you find what you are looking for quickly. A search engine index, like the open source Solr index that Lantern uses, takes a document and blows it apart into lots of small bits so that a computer can quickly search it. Solr comes loaded with the search algorithms that do most of the mathematical heavy lifting. But as developers, we still had to decide exactly what metadata to capture and how to index it. In my “Working Theory” essay co-written with Carl and Wendy, I’ve described how MARC records offered insufficient metadata for the search experience our users wanted. In this post, I want to emphasize is that if something isn’t in the index, and if the index doesn’t play nicely with the search algorithms, then you won’t have a happy search experience. Lesson #3 should make this point more clear.

3. Search algorithms are designed for breadth and scale, so don’t ask them to search in depth

Open source search algorithms are better at searching 2 million short documents, each containing 500 words of text, than at searching 500 very long documents containing 200,000 words each. I learned this lesson the hard way. At the Media History Digital Library, we scan magazines that have been collected and bound together into volumes. So in our early experiments with Lantern, we turned every volume into a discrete XML file with metadata fields for title, creator, date, etc., plus the metadata field “body” where we pasted all the text from the scanned work. Big mistake. Some of the “body” fields had half a million words! After indexing these XML documents, our search speed was dreadfully slow and, worse yet, the results were inaccurate or only partially accurate. In some cases, the search algorithms would find a few hits within a particular work and then time out without searching the full document. The solution — beautifully scripted in Python by Andy Myers — was to turn every page inside a volume into its own XML document, then index all 800,000 MHDL pages as unique documents. This is the only way we can deliver the fast, accurate search results that you want. But we also recognize that it risks de-contextualizing the single page from the larger work. We believe the “Read in Context” option and the catalog descriptions offer partial answers to this challenge of preserving context, and we’re working on developing additional methods too.

4. Good OCR matters for searchability, but OCR isn’t the whole story

You don’t need OCR (optical character recognition) to search a blog or docx Word file. Those textual works were born digital; a computer can clearly see whether that was an “a” or “o” that the author typed. In contrast, Moving Picture World, Radio Mirror, and the MHDL’s other books and magazines were born in the print age. In order to make them machine readable, we need to run optical character recognition — a process that occurs on the Internet Archive’s servers using Abbyy Fine Reader. Abbyy has to make a lot of guesses about particular words and characters. We tend to scan works that are in good condition at a high resolution, and this leads to Abbyy making better guesses and the MHDL having high quality OCR. Nevertheless, the OCR isn’t perfect, and the imperfections are immediately visible in a snippet like this one from a 1930 volume of Film Daily: “Bette Davis, stage actress, has been signed by Carl Taemmle. Jr.” The snippet should say “Carl Laemmle, Jr.” That is the Universal executive listed on the page, and I wish our database model enabled users to log in and fix these blemishes (hopefully, we’ll get to this point in 2014). But — you may have guessed there was a but coming — our search algorithms use some probabilistic guessing and “stemming,” which splinters individual words and allows your query to search for related words (for instance, returning “reissue” and “reissuing” for a “reissue” query. The aggressiveness of stemming and probabilistic word guessing (aka “fuzzyness”) is something that developers can boost or turn down. I’m still trying to flavor Lantern’s stew just right. The big takeaway point, though, is that you’ll quickly notice the OCR quality, but there are other hidden processes going on shaping your results.

5. The search experience has become increasingly visual.

As my colleague Jeremy Morris pointed out to me during one of our food cart lunches outside the UW Library, the search experience has become highly visual. Googling a restaurant now renders a map within the results page. Proquest queries now return icons that display the format of the work — article, book, etc. — but not an image of the actual work. I’d like to think Lantern’s results view one-ups Proquest. We display a full color thumbnail of the matching page in the results view, not simply an icon. The thumbnail communicates a tremendous amount of information very efficiently. You quickly get a sense about whether the page is an advertisement or news story, whether it comes from a glossy fan magazine or a trade paper published in broadsheet layout. Even before you read the highlighted text snippet, you get some impression of the page and source. The thumbnails also help compensate for the lack of our metadata’s granularity. We haven’t had the resources to generate metadata on the level of individual magazine issues, pages, or articles (it’s here that Proquest one-ups us). By exposing the thumbnail page image, though, you visually glean some essential information from the source. Plus, the thumbnails showcase one of the strengths of the MHDL collection: the colorful, photo rich, and graphically interesting nature of the historic magazines.

Ok, now it’s your turn to think algorithmically. When you search for a movie star and sort by relevancy, why is it that the most visually rich pages — often featuring a large photo — tend to rank the highest?

The answer is that those pages tend to have relatively few words. If there are only eight words on a portrait page from The New Movie Magazine and two of them are “Joan Crawford,” then her name occupies a far higher word frequency-to-page percentage than a page from Variety that is jam packed with over 1,000 words of text, including a story announcing Joan Crawford’s next picture.

Should I tweak the relevancy algorithm so that image-heavy pages aren’t listed so high? Should I ascribe greater relevancy to certain canonical sources, like Photoplay and Variety, rather than magazines outside the canon, like New Movie and Hollywood Filmograph? Or should we weight things the other way around — try to nudge users toward under-utilized sources? I would be curious to know what Antenna readers and Lantern users think.

There are advantages and disadvantages no matter what you choose. The best approach, as I see it, may just to be to let the ranking algorithm run as is and use forums like this one to make their workings more transparent.