Particip@tions Vol. 3 (2) Article - Kate Egan & Martin Barker

Rings around the World: Notes on the Challenges, Problems & Possibilities of International Audience Projects

In a recent article, Sonia Livingstone has attempted to summarise some of the many problems, complexities and challenges which researchers face when conducting cross-cultural audience projects. This essay tells the story of the specific problems and issues encountered during the Lord of the Rings international audience project, in order to offer this project as a case study of how such problems can be anticipated or encountered and then managed, and to also offer up some new issues and questions that have emerged during the course of the project which, we feel, deserve to be considered and brought in to the cross-cultural debate. These include, in particular, the benefits and pitfalls of international web questionnaires, multi-dimensional forms of international research (reception research, questionnaires and qualitative interviews), and electronic and face-to-face forms of communication amongst research partners.

Key Words: international audience research; methods, issues and problems; audiences for Lord of the Rings; using web questionnaires

This essay tells the story of the Lord of the Rings (henceforth LOTR) international audience project, of how and why it came about, and how it was developed and pursued. Here we particularly focus on the ways in which we tackled the problems raised by trying to design and run a truly international project. Because the project required us to try to involve an indefinitely large number of countries, we faced a series of tough challenges. In a recent article, Sonia Livingstone, a researcher with a strong record of international projects, has summarised her sense of many of the problems involved,[1] many of which also emerged during the course of administering and running our project. Most prominently, these include: the problems of bringing together international research partners from a variety of disciplinary backgrounds, the disparities created by the different levels of funding that our partners were able to obtain, and the problems of working with terms and concepts that may hold a variety of different meanings and associations for audiences in different national contexts.

The account that follows will touch on these and many other issues, in terms of how they arose and how they were, in most cases, managed within the specific realms of our project. For we believe that while our project did have to encounter a series of very substantial issues, we did – sometimes by design and sometimes by sheer luck – manage to overcome most of them. With this in mind, what we hope to demonstrate in the following account is that while cross-cultural audience research is fraught with theoretical, conceptual and methodological difficulties and complexities, this does not mean that researchers should not recognise and embrace the possibilities and challenges of international projects of this kind. We certainly don’t consider our solutions to some of the problems we encountered as the only (or even the best) way to tackle some of these issues. However, we do hope that this account will serve both as an illustration of how all decisions made within an international project have consequences which should (and must) be acknowledged and reflected on, and as an expansion of Livingstone’s account which raises some further issues and questions that we feel should be brought into the cross-cultural research debate.

As two people centrally involved, we have tried to tell the story of this project honestly and fully, admitting the parts where we got it wrong, and the places where we ‘made it up as we went along’. We are telling it in this way because, Livingstone aside, we have rarely learned what we needed from reading many other people’s research reports. At best we hear about methods of data-gathering, about methods of analysis and weighing of the evidence, and about conclusions – all of which are posed in terms that could come from books on methodology. What we don’t much learn about is the reasons why the research was undertaken, the struggles to realise it, the mistakes made, and the lessons learnt about the process of research. Yet, in our experience, these are crucial, and complicated. Methods involve complicated ways of bridging the broad spaces between questions that researchers want to answer, and the pragmatics and possibilities of their research. This account, focusing as it does only on the international dimensions of the research, is but one part of a much fuller account which we are preparing, which we are happy to make available to anyone who is interested.

The LOTR project was first conceived in late 2002. Its aim was vast and, you could well say, over-reaching. Our central goal was to try to research world responses to the final part of the film adaptation of LOTR, with an eye to three aspects in particular: (1) the functions that film fantasy plays in the lives of different kinds of audience in as many countries where we could reach people; (2) the responses in different countries to a film with the complicated origins that the LOTR trilogy displays – a story with strongly English origins, a production process celebrating New Zealand, and financed by (a subsidiary of) the largest leisure-entertainment conglomerate, AOL-Time-Warner; and (3) the ways in which, in different countries, the ground for responses was prepared by everything from marketing, publicity and merchandising, through to local media coverage, and how audiences in situ made use and sense of those materials. This crazily large set of ambitions depended on so many things working out, not the least of which was our ability to recruit enough research partners reckless but excited enough to share our ambitions in this regard.

We began with an argument setting out the main grounds within our field for such a project. We explained our purposes in the following terms:

1. The on-going debate about ‘Hollywoodisation’, ‘cultural imperialism’ and the impact on national cultures of industrialised cultural production. This was important since it put emphasis on one aspect of the research: its international dimension. We had decided to try to find research partners in other countries. Hubris led us to hope that we might end up with a project wider and larger than had been attempted in the yet single largest project of which we knew – Janet Wasko, Mark Phillips and Eileen R. Meehan’s investigation of Disney’s audiences (which was important in another way, because we saw real weaknesses in this project’s methodology).[2]

2. The problems associated with the concept of ‘fantasy’. The debates around ‘fantasy’ have tended towards three poles: a ‘commonsense’ pole, in which fantasy is little more than irrelevant escapism (for a good discussion of an equivalent problem with the concept of ‘entertainment’, see Richard Dyer, Only Entertainment); a psychoanalytic pole, within which fantasy is the adult consequence of child-originated impulses (for an example of this approach, see Julian Henriques et al’s Changing the Subject); and a politicised pole in which fantasy functions as a means of political incorporation (of which the most famous example has to be Ariel Dorfman and Armand Mattelart’s How to Read Donald Duck).[3] The problem is that none of these is readily susceptible to testing by audience research.

3. We also related our project to the emerging interest in the associated bodies of materials that accompany a film – a body which came to be called, in the end, ‘prefigurative’. We are still not certain if this is the best term, since it implies that they come before the film, and we know very well that they are not always so encountered – it became a common theme in our project interviews in the UK, for instance, that people like to read reviews of a film after they have seen it, and the Extended DVD Editions of the film play a role in responses in the cinema, even though these weren’t released and looked at until many months later.

From our earliest thinking about the project, it was always going to have three stages. The first stage would be a study of those prefigurative materials – the whole range of publicity and marketing materials, merchandise and other licensed materials, press, magazine, radio and television coverage, the internet and website materials. This was UK project team member Ernest Mathijs’ centre of interest. It should be stressed that the gathering of these materials is the easiest part. Knowing what to do with them, once gathered, was a much larger, and much longer-debated, topic among us. To be honest, this is one area where the international reach of the project was not, in our view, as successful as others. Because of the inevitable different levels of commitment among our international partners, there was little consistency in the kind or amount of materials gathered, and our comparative analysis of these has been much more restricted than we would have wished. On the other hand, it has also become apparent to us, through analysis of our international project questionnaire data and UK project interviews, that the prefigurative materials that were most predominantly drawn on by audiences were non-nationally specific, including the international LOTR fan website, The One Ring, and bonus documentaries on the Extended DVD versions of the films.

The second stage of data-gathering was our project questionnaire. We knew that it would be modelled, in a broad way, on the questionnaire developed for the UK Crash project.[4] It would therefore aim to combine quantitative and qualitative measures, and it would do this by asking audiences to assign themselves on some simple dimensions, while simultaneously telling us what those choices mean to them. This was a ‘big’ methodological move in all kinds of ways. It took us into the dark heart of the long debates about the relations between quantitative and qualitative modes of enquiry, but sought to carve a new position within those debates. But also, crucially, we planned to make this primarily a web-based questionnaire. Drawing on the experience of one member of the UK team, Janet Jones, who had used a web questionnaire attached to the official UK website for Big Brother, we saw this as a key move – and one that would bring with it both signal benefits and some real risks. The potential benefits were: (a) that all the responses to the questionnaire could compile automatically into a database, making many kinds of automated analysis possible; (b) that the questionnaire could be made world-wide, and could even appear in several different languages (partly according to who we managed to persuade to join in with us – in the end, we operated in fourteen languages);[5] (c) that we might be able to generate large numbers of responses quite ‘cheaply’, through reaching people via email lists, web sites, and other electronic communication routes.

The potential risks on the other hand were: (a) that we had to be entirely on top of the computer skills necessary to achieve this, (b) that we would find it too easy to recruit enthusiasts in this way, but might have much more problems attracting the reluctant, the critical and the hostile. To mitigate against the second danger, we therefore also planned to use a paper version of the questionnaire, and to recruit helpers in various parts of the country to approach people at the cinema and get paper copies completed. In the UK, we were committed to achieving 2000 responses, combining web and paper versions. Across the world, we could only hope. But since, to be truly international, it was essential that we invite responses in more than one language, we had to face and solve a series of translation problems.

The final stage would be follow-up interviews. Our goal, in the UK, was specific and tricky – on the basis of an investigation of an interim data-run from the questionnaire, we hoped to identify general tendencies, or separate groupings, from which we could select individuals who appeared to fit the emergent patterns, and who had given us permission to contact them. We had promised our funding body in the UK, the ESRC, that we would be aiming to achieve 100 interviews.[6] In achieving this, we worked to a minimum of four per pattern, but then allowed ourselves to scale the number of interviews to the apparent scale of significance of the group – so that ultimately the largest group was 20 people, for what we identified as a central tendency in our responses. The point of the interviews was to flesh out in detail what was being revealed by the emergent patterns. But for that very reason, when we came to design our question schedule for these, we had to make sure that we did not impose our pattern onto them. So we were looking to develop a common schedule of questions, to be used with everyone we interviewed – and also, of course, one that might be used by any of our international partners.

In April 2003 we heard that our ESRC application had succeeded. Thereafter, the project had to move at real speed, to reach agreement on all outstanding issues and to construct, translate, test and mount the web questionnaire, in order to be ready to begin to collect responses from the first day of the film’s release, in most territories, on December 17, 2003.[7]

Recruiting Research Partners

An honest admission. When we proposed the research to the British Economic and Social Research Council in February 2003, we said – truthfully – that researchers in 15 countries had indicated their interest in taking part. But we had no certainty at that stage that any or all of these would ‘stick’. In the end we did better than that. During the main period of the research, research groups in 20 countries were taking part – albeit several of these ‘dropped by the way’ before the end.

How did we find these? By a combination of hard work, advertising and sheer luck. Perhaps unwisely, we did not build this from existing collaborations, or from networks within any of the main international organisations in our field, such as the Society for Cinema and Media Studies (SCMS), the International Association for Media and Communication Research (IAMCR), or the International Communication Association (ICA). Our first procedure was to ensure that one other country was committed with us. Circumstances dictated that it should be a group of colleagues in New Zealand, at Waikato University, who were already involved in other kinds of research on The Lord of the Rings. With their agreement, we prepared an Invitation which we sent out every which way we could conceive. We approached known research colleagues in other countries. We used various academic listservs. We Googled a lot of people who had published interesting work. We snowballed and foafed (‘friend of a friend’) possible people. Gradually, a core of enthusiastic people coalesced and, in a number of cases, began their own funding applications. Other individuals and groups became involved to the extent that their circumstances permitted – and sometimes, these were not very great.

Here, the overall shape of the project proved particularly beneficial. We were able to welcome people into the project at quite different levels of involvement. The bare minimum simply meant helping in the design, translation and publicity for the central project web questionnaire – and then, hopefully, looking at and analysing results. Beyond that, individuals or groups could use the paper version of the questionnaire, and input the results. They could choose to gather and analyse any range and extent of prefigurative materials. And they could potentially conduct interviews and focus groups, whether the recruitment to these did or did not derive (as we planned in the UK) from an initial search for patterns in questionnaire responses. In these respects, our research design consciously sought to offer opportunities for participation, however extensive or limited. This paid off, in spades.

At the point of the film’s release, researchers in all the following countries had indicated a willingness to join the UK in the project: Australia, Belgium, Canada, China, Colombia, Denmark, Germany, Greece, Italy, Netherlands, New Zealand, Norway, Russia, Slovenia, Spain, Turkey, and the United States. A little later, one researcher from France also became involved. In one or two cases, participation turned out to be quite notional. In another couple of cases, changed circumstances caused people to withdraw after a period of involvement.

The Consequences of Accidental Recruitment

It is important, though, to see the implications of the accidental nature of this international team, and that there were distinct downsides. Why were people taking part? There was no absolutely agreed basis. Although the initial invitation set out a research rationale and general plan, it did not, indeed could not, spell out in full detail the theoretical, conceptual and methodological bases from which we were working. In agreeing to join in with the project, it was not easy for us to know how far others would be in full agreement with our approach. And in the reverse direction, they often did not know very much about us – but took us on trust, for which we simply have to be very grateful. Indeed, quite reasonably, research groups in three countries chose to work with slightly different questionnaires (partly because of their separate ‘contracts’ with their national funding agencies), which had awkward consequences when we sought to merge the four resultant databases. In retrospect, we can see no alternative to what we did. It has had its costs. But we think the results are more than worth the sometimes problems and pain that have followed from the risks arising from this accidental recruitment.

The following are the main problems and decisions, which we address further below:

The core of the project, and the one part to which every international partner had to sign up, was the questionnaire. The design and construction of this was the most intensive single operation of the entire research. By and large, it was successful and generated rich materials allowing investigation from an indefinitely large number of directions – which was our aim. We made some mistakes, some technical, some procedural, which we record here in the hope that it can help other researchers avoid the same ones.

A draft questionnaire was produced early on – in the early summer of 2003 – and circulated for comments to all those then indicating a wish to participate. It was reworked, and recirculated more than once, towards a September deadline. It was also piloted during this period with students in Aberystwyth. 300 students were asked if they would try to complete the questionnaire on a trial website, based on their viewing of part two of the trilogy – and then to answer two further questions: were there any questions which they had had problems completing, and were there any missing which they would have liked to have the chance to answer? We then were able to look at their answers to the questions, and see if these revealed any misunderstandings, and also look at their meta-comments on the questionnaire. It was very important to us that the questionnaire should be ‘user-friendly’ – that is, not feel like an outside intervention by people not primarily interested in the film. Therefore when we received strong feedback that our initial list of questions lacked one which significant numbers said they wanted to be able to answer (‘Who was your favourite character?’), we had to include it. We honestly confess that we did not really want to do this, but agreed – and in retrospect are very glad, for a host of reasons. But there was a price. Because the questionnaire had to operate both on the web and on paper, there was a space limitation. Even using a Desk Top Publishing-designed paper version, using the advantages of sophisticated layout, there was a clear limit to what we had agreed had to be our maximum size for use at cinemas (2 sides of an A4 sheet). Therefore, to fit in the Favourite Character question, another had to go. This kind of compromise is inevitable in research of this kind and we should not be ashamed of it. No project can answer all questions, we should be happy if we generate rich enough materials to answer just some questions well.

This compromise was further tested when we produced paper versions of the questionnaire in translation for colleagues in several countries, and realised that the space required for the questions varied by quite a margin in different languages. The tight fit onto two sides of A4 was severely tested in several cases!

The questionnaire had to appear in a number of languages (in the end, fourteen), which meant getting agreement to its content in time to get it translated. ‘Translation’ is not an automatic process under any circumstances, but was particularly tricky in our case, because of the nature of the key terms we needed translating. There were few problems with a question like ‘What did you think of the film? Extremely / very / reasonably / hardly / not at all enjoyable’. But there were major problems with two questions, for different reasons:

1. Key questions for us were Questions five and six. Question five asked people to nominate up to three of a list of twelve terms, which might describe the kind of story they felt The Lord of the Rings was. After a lot of debate, the list in English read: Allegory, Epic, Fairytale, Fantasy, Game-World, Good vs Evil, Myth/Legend, Quest, SFX film, Spiritual Journey, Threatened Homeland, and War Story. (There was an option for respondents to nominate their own if none of these seemed right, although very few chose that option.) The problems of translation were considerable. First, not all these distinctions exist in all countries. Second, the terms may have different salience and resonance in other languages. Our aim was to use these choices not to give us answers but to do two things: to get a first indication of patterns of choices, by country, or by sex or age group etc., which we could then enquire into in detail by looking at answers to other questions; and to get a sense of the key vernacular terms whose meanings we would have to unpack. The problem was: you had to have terms that would register as vernacular in all the different languages – they had to be recognisable, in order to be meaningfully chosen.

This is one point where, we believe, we benefited enormously from the central design of the questionnaire. The decision to combine quantitative (multiple-choice) and qualitative (free-text) responses meant that with questions such as this we could do two jobs at the same time. We could ask people to choose from a list, but then immediately also tell us what that choice meant to them. This combination has proved invaluable, in a number of phases of the research. So, alongside identifying that certain kinds of audience were most likely to say that they had derived Extreme Enjoyment from the films, we have been able to explore what kinds of pleasures were involved for them. We have been able to identify those groups who were most likely to nominate a modality term like Epic, or Spiritual Journey, and then to explore what they mean by these terms. This meant that our research design already held a partial solution to the problems of translation – we could check, from respondents’ own accounts, what they understood by the terms in translation. So, Question six, the companion to Question five, asked people to put into their own words where and when they imagined Middle-earth to be, or in other words, to tell us the kind of imaginary world Tolkien’s story conjured up for them. But we should admit that in several phases of our analysis it has not been the expected question pair which has yielded the best ways of disclosing what audiences mean by their choices. Rather, we have benefited from the general commitment to the combination of multiple-choice and free-text answers, where, for instance, free-text answers to why the film was enjoyable have often been able to shed light on an audience member’s understanding of the kind of story they see the film to be.

2. A different problem altogether was thrown up by the demographic categories. None of the project’s research partners had any problems with asking for people’s sex, or indeed age (although two countries operating with their own databases chose to ask for year of birth, rather than follow the central project questionnaire’s age-ranges – but translation between them posed only a technical problem when we merged the databases). But there were other problems. We chose not to ask most further standard demographic questions. We could not ask about income, since there is no agreed international standard against which we could ask people (would it be Dollars, or Euros, or another kind of currency?). If the answers could not be compared, there was no point in asking. We chose also not to ask about educational level, again because there are no absolutely comparable international categories. But we did want to ask about Occupation. After some debate we developed a twelve-part categorisation of Occupations.[8] Considering that standard categories of occupation might also vary from country to country, we decided that we did not want to know the particular jobs people did, but the kind of jobs – as they perceived it. So, we built a categorisation which could include an element of attitude towards the job a person does. A person working in an advertising agency in New York or New Delhi might describe their work as Administrative, or as Creative, or as Professional, or indeed as Executive – this is their choice. A person working in agriculture in France or in Zambia might see themselves as skilled, unskilled, or indeed as in a service role. We wanted to do this in this way, because we did not simply want to find out how occupations, demographically conceived, might relate to responses to the film. Rather, since our central interest was in fantasy and how it functions in people’s lives, we wanted to see if we could open up for exploration how people’s perceptions of and attitudes to their jobs might relate to their response to the film. The votes are still being counted on whether we did this right. In small ways, we have identified groups where this might be valuable. For instance, in the UK, we found that people choosing Creative had different orientations to the film than others.[9] And we found, intriguingly, that a group of under-16s who declared themselves Unemployed, rather than as Students, did have distinctly different responses to the film. Beyond this, we are not sure.[10] Our decision on this, which in the end we simply had to go with because of pressure of time, was not universally liked among our research partners, and certainly those who operated separate websites chose to do this in the more classic sociological fashion. We will have to wait to see what they learn, using their approach.

We are not the first to use a web questionnaire for research purposes, by any means. But to our knowledge little has yet been written about the strengths and weaknesses, operational and intellectual, of their use; and yet surely their use must increase steadily, as access to the internet continues to rise. This project was an ambitious one. Here are some indications of this ambition. We were operating in fourteen languages, but with a common database – where some of those languages (Turkish, Greek, Chinese) use non-Latin character sets. That set technical demands, which we did not fully grasp until late on. When you translate a set of terms like Allegory etc, they may change alphabetical order, and we therefore had to be extremely vigilant at all stages, to ensure that entries were going into the right fields in all languages. When paper questionnaires were entered, we tried to ensure that in all countries they were distinguished by some marker, since it might prove useful to be able to make a comparison between the two sources (in the UK we did this, and found that we had recruited a significantly different overall group via this route – paper-completers were significantly older than those filling out on the web, had read the LOTR books less often, and were less enthusiastic in their responses to the film). Furthermore, when considering answers to the Occupations question, we had to remember that people using the paper version of the questionnaire simply filled in their specific job – they did not choose the category into which it would go. Thus our ambition to assess attitude to job would not be possible with these respondents, and we therefore had to make sure that we were able to eliminate them from consideration when assessing this dimension of peoples’ responses.

Analytic Traditions

Our processes of analysing the completed questionnaires have been complicated, and may not be the same in all groups. The UK research team offered, to all participating research teams, a series of examples of exploratory and analytic procedures, through which we believe that we have been able to map the international responses overall, and locate distinctive patterns within the map. Working with Microsoft Access as a relational database, we believe, facilitated this kind of search. It permits, with relative ease, the gradual formation of groups of associated responses, and then the gathering of the more qualitative materials for further exploration, and the location of individuals who exemplify the patterns who could be interviewed in the final phase. In the process of doing this, we discovered that our project web space could be utilised for another beneficial purpose – to mount large documents (including the complete, international data-set of questionnaire responses, and the UK and other research teams’ examples of analytic procedures, findings and research reports) for all our research partners to access and download with ease, via a private page on the web-site.

This sounds grand, and in the end we have no doubts that it was worthwhile. But there was a stage, in late January and early February 2004, when we were virtually ‘making it up as we went along’. We had to learn the potentialities of Access on the hoof. Rightly or wrongly, this led us to put an emphasis in our first searches on the discovery of discrete groups – ones who broke away from the main patterns and did something unusual. Some of these groups were quite small. Only subsequently did we return to the main core of responses, and address their nature. And only after that did we really exploit the ability of our overall responses to tell us much, just through an overall profile.

Even in retrospect we do not believe this was necessarily a bad thing. What it did encourage was a sense of simple discovery: rather than following a regime of pre-determined exploratory routes, we began by taking one of the first surprises which the data presented us, and used it as a basis for a set of searches. If, astonishingly, the gender split is overall so nearly a 50:50 split, when and where does that break down? With what choices are men more associated than women, and vice versa? Having begun in this way, we could – with a touch more numerical sophistication – do the same for age, and book-reading, and (eventually) occupation. And then these in combination, of course. This led us, within the UK questionnaire data, to the identification of a number of distinctive ‘Clusters’ (patterned differences arising from at least three dimensions); a good example would be the discovery of a group of older women who had chosen Spiritual journey – a Cluster made all the more interesting to us, when we were able to check this against the World Data-set, and realise that it does not seem to be reproduced in all other countries. Having found some 15 such patterns, we realised the need to return to the core, and began to look at the nature of the most commonly repeated responses – which in the UK turned around, we discovered, the choice of the term ‘Epic’.

The point of saying all this is to emphasise that, in research of this kind, there is a definite limit to how far one can predetermine the investigative and analytic processes. We ‘flew blind’ for a while, following some routes which led nowhere, along with some which produced puzzles, and some which produced really clear and exciting findings, before we could begin to ‘map the whole’.

But the difficulty was this, for our international partners. First, some did not have our resources. We had colleagues working entirely on their own, without any research support let alone anything as dedicated as a research assistant. Second, we had colleagues coming into this project from highly varied prior academic backgrounds. Some had never worked with quantitative procedures before. Some were richly experienced with the Statistical Package for the Social Sciences (SPSS) and had a working presumption that all results needed testing for statistical significance. Some were involved because of an overriding interest in the political economic aspects of the films and only marginally in the audience aspects. We had few, and had no right to expect, common methodological grounds. Furthermore, there were ways in which the framing of the questions owed more than we might even have recognised ourselves, to the slightly eccentric tradition of the project’s UK Director, Martin Barker. He had been developing concepts and associated methods which did not chime immediately with the ones most widely recognised in the field of audience research.

The project was distinctly unusual. The number of cross-country research projects is still, as far as we know the field, in single figures.[11] The consortium of researchers who joined together to pursue this project had not worked together before. Indeed, in a number of cases, we did not meet until a conference at the close of the project. The process by which people joined the project meant that, amongst the team, we had people of enormously varied disciplinary backgrounds and research experiences. The disciplinary backgrounds ranged across political economy, mass communications, sociology, and textual studies. There was a matching range of methodological preferences. What united us was simply a will to explore the international phenomenon that was The Lord of the Rings. To be frank, by the time quite a number of people committed their time to the project, the stages, and associated methodologies that would be pursued, were largely locked down – not least as a result of the application to the British ESRC. This meant that, willy nilly, many groups signed up to a research methodology of which they had no prior experience – they took it ‘on trust’. And given the highly fragmentary and contested nature of our field, this is almost inevitable. Yet many researchers recognise the deep need for more and better cross-cultural research, to test and give substance to many of the arguments about, for instance, globalisation.

Did the project’s results repay the trust? We believe so. Among the topics which the research was able to address, as a result, was a complex cross-cultural examination of the nature of responses in the twelve countries which had the highest overall levels of responses. Because we were drawing on a common database of responses, it was possible to make subtle comparisons, of a kind that to our knowledge has not been attempted before. A striking pattern emerged, connecting the length of time that Tolkien’s books have been available in different countries, the depth of reading revealed by the questionnaires, and the contrasting terms used by the most committed viewers, compared to the most commonly-used choice in each country, to describe the story-world of Middle-earth. The investigation of this pattern allowed us to disclose distinct ways in which the individual countries’ responses could be indicatively related to broader aspects of that country-context. The results of this investigation now constitute a central chapter to the book of core findings from the project.

As Livingstone argues in her article, maintaining good working relationships with all partners involved in an international project (particularly when the majority of the project team have not previously worked together, let alone met each other) is of paramount importance. Indeed, and as she also argues, the overall success and productivity of projects of this kind can frequently stand or fall on the basis of how successfully trust is maintained, information is disseminated, and dialogue and communication is perpetuated.[12] Clearly, and as with the use of web pages to disseminate key project files and documents, the increasing use and impact of electronic forms of communication within academia has aided this process – particularly, as Livingstone identifies, with the establishment of email as perhaps the key form of communication used within the international academic community.

However, a key question which arises from this is: how, within a project of this size and scope, are these flows of communication and information amongst project teams managed and administered? Within the LOTR project network, Kate Egan served not only as research assistant to the UK project team but also as co-ordinator of communication between the project’s international research partners. In this role, she dealt with the minute details of managing and operationalising the project – receiving (via email) translated versions of the web questionnaire from research partners, sending out important questions and up-dates, sending data-sets of questionnaire data to research partners and liaising with partners in terms of their specific requirements for such data-sets (e.g. which responses from which countries in which languages were required in particular data-sets, and in which format – Access or SPSS).

Clearly (and, as far as we are aware, this has not been acknowledged in any other accounts of cross-cultural research) having an administrative co-ordinator of this kind is essential within an international project, both in terms of the need for one central person to monitor the progress of the web questionnaire and the amount of ‘hits’ per country it was receiving, and so that research partners dotted around the world are kept informed of these and other problems and developments that emerge during the course of the data-gathering period of the project. In our project, this occurred over a period of more than a year, when, clearly, research partners also had other issues to turn their attention to, including teaching and other research projects). However, while this role is essential – at least if international projects wish to go beyond studies where data is gathered completely separately in each country and discussion between partners only occurs at the analysis stage – this does not mean that the use of such a figure doesn’t come without its own set of problems and pitfalls. For clearly, if the international team becomes accustomed to one national team (in this case, the UK) operating as both the central source of information and the primary instigator of debate, then this one-way flow of communication can, potentially, shut down specific forms of dialogue between other project teams outside of the UK. Late on in the data-gathering process, the UK team realised this potential danger, and Martin Barker began to encourage research partners, in email updates, to copy and paste the list of email addresses in the ‘to’ line of the email, and to feel free to use them, if they wished, to contact specific individuals on the list.

For us, this realisation highlighted a fundamental dilemma underscoring cross-cultural projects, and particularly those that utilise a central international web questionnaire as their primary research tool. Clearly, there needs to be a central figure within the project network who not only monitors the progress of the questionnaire, but also keeps all research partners involved, on board and ‘in the loop’ over a lengthy period of time, and operates as the instigator of not just debate but also friendly, interpersonal communication between the project instigators and the rest of the project network. Indeed, for what it’s worth, Kate Egan found this interpersonal communication to be the most enjoyable and rewarding part of her work on the LOTR project. However, at the same time, if the limits of this role are not discussed and debated by all partners at the beginning of a project (and, in hindsight, we feel the UK team should have organised an occasion where this could have occurred), then the establishment of this figure as the locus of data and information on the project could, potentially, close down communication and dialogue between other research partners, to the possible detriment of all the other research teams involved in the project.

Finding the ‘International’ in the International Lord of the Rings Project

Research projects are funny animals. To do them properly, you have to become so closely involved in them that for a time you can forget what you are doing them for. The goal of just doing it as well as you can becomes the end-in-itself. And we do admit that to an extent, and for a while, that happened with this research. We became so fascinated with the sheer task of gathering, then organising, managing and conducting first stage analyses of the materials, that we forgot our founding questions. Researchers are not supposed to admit things like this. The truth is, for a complex of reasons, we laid aside and didn’t return to one aspect in particular of our research: its strong cross-cultural ambitions. In a field where a corpus of cross-cultural researches into audiences is only now beginning to be established, we were in an almost unique position to produce findings, and this was particularly so since we had a common data-set. A number of other cross-cultural researches have been conducted on the basis of separate studies within different countries – a process which, curiously, can lead to exaggeration of differences, so it is not easy to see how similarities or overlaps could readily be found. But the fact remains, we let this slide.

When we did finally return to it, in early 2006, it was slightly adventitious. On the spur of the moment, we decided to see what might be learnt by taking those countries which had quite high overall levels of responses (so that complex enquiries into the sets would not result in dealing with small and unreliable quantities), and doing a series of in-depth enquiries into their similarities and differences. This resulted in us working with twelve countries, with a highest data-set of 4,700 (USA) and a lowest of 500 (Greece). The nice thing was that it included a range of different kinds of countries – the list comprised Australia, Belgium, China, Denmark, Germany, Greece, Italy, the Netherlands, Slovenia, Spain, the United Kingdom and the United States. What emerged from several days of data-crunching was, first, repeated senses of no pattern at all (gender, age, occupation patterns were all over the place – and indeed they revealed that while the world ratio of men and women was almost exactly 50:50, there were considerable variations by country); and then one stunning set of patterns, which reappeared each time we approached them with new materials and from new angles. They related to the differences between, and relations between, the overall most common descriptor of the film in each country, and the descriptor most commonly used by those with the highest commitment to and engagement with the film. A clear and strong patterning among these, and also with extents of readings of the LOTR books, emerged.

Working on these, and trying to make best sense of them, brought back into view the set of problems associated with doing this kind of international research. Some problems were acutely social: the problem of infringing on people’s time and energy yet again, to ask them to help us explore the patterns we had located. (This involved asking some people, in busy times, to translate 4,000 words of responses from their own language into English.) Others were acutely methodological. The fact is that there were no common methodological grounds across the world, among participants, for how to conduct and complete this analysis. There was a shared interest in the questions that the project sought to ask, but we would be hard put to identify anything more than negative agreements on how such research should be conducted, and how we should go about analysis of materials, once found. The negative would surely revolve around suspicion of mass communications models. But beyond that, there simply is not a shared set of methods, and in particular analytic procedures. And since we were trying to operate on the forefront, by managing a combination of quantitative and qualitative modes of investigation, the nascent problems just got more and more fierce. At the time of writing this, we are still completing this, and will leave the judgement on its success to others.[13]

Conclusion: New Issues, New Questions

To return to our opening remarks, while we acknowledge, like Sonia Livingstone, the myriad difficulties and pitfalls of international audience research, we do not have any doubt as to the virtue and value of our attempt – not only in terms of raising a range of new issues and problems that should, we feel, be considered and debated amongst cross-cultural researchers, but also in the sense that international studies can not only challenge us, but also broaden our knowledge and our resources After complicatedly merging the four databases from the central project questionnaire, and the Belgian, Dutch and Slovenian questionnaires, and cleaning them for some doubled cases, we had 24,739 analysable responses – we are sure this is a record for any kind of audience research, so far. This rich body alone, we think, justifies the risks we took, and the consequences of the decisions we made.[14]

Barker, Martin, Jane Arthurs and Ramaswami Harindranath, The Crash Controversy: Censorship Campaigns and Film Reception, London: Wallflower 2001.

Livingstone, Sonia and Moira Bovill (eds.), Children and their Changing Media Environment, Hahwah, NJ: Lawrence Erlbaum, 2001.

Livingstone, Sonia, “On the Challenges of Cross-National Comparative Media Research”, European Journal of Communication, 18.4, 2003, pp. 477-500.

Mitchell, Claudia, and Jacqueline Reid-Walsh (eds.), Seven Going on Seventeen: Tween Studies in the Culture of Girlhood, NY: Peter Lang, 2005.

Wasko, Janet, Mark Philips and Eileen R Meehan (eds.), Dazzled By Disney? The Global Disney Audiences Project, Leicester: Leicester University Press, 2001.

[1] Sonia Livingstone, “On the Challenges of Cross-National Comparative Media Research”, European Journal of Communication, 18.4, 2003, pp. 477-500.

[2] See Janet Wasko, Mark Phillips and Eileen R Meehan (eds.), Dazzled By Disney? The

Global Disney Audiences Project, Leicester: Leicester University Press, 2001.  While this book

undoubtedly contains a great deal of valuable material, particularly about the different histories of

Disney's presence in particular countries, its audience research dimension is weakened by a tendency

to inscribe into its methodology an assumption that Disney is a problem, and that if audiences do not

perceive this, that is in itself evidence of Disney's power.

[3] Richard Dyer, Only Entertainment, London: Routledge, 2002; Julian Henriques, Wendy

Hollway, Cathy Urwin, Couze Venn and Valerie Walkerdine, Changing the Subject: Psychology,

Regulation and Subjectivity, London: Methuen, 1984; and Ariel Dorfman and Armand Mattelart,

How to Read Donald Duck: Disney Ideology in the Disney Comic, NY: International General, 1976.

[4]On this, see Martin Barker, Jane Arthurs and Ramaswami Harindranath, The Crash Controversy: Censorship

Campaigns and Film Reception, London: Wallflower 2001.

[5] These were: Chinese, Danish, Dutch, English, French, German, Greek, Italian, Norwegian, Russian,

Slovenian, Spanish, Turkish and Welsh.

[6]  It is important to note our gratitude to the Economic and Social Research Council, without whose project

grant (RES-000-22-0323 ) this research would not have been possible at all.

[7] We had to remain aware, however, that the film was released later in some countries (including Italy and

Australia).  Indeed, the film’s official release in China didn’t occur until March 2004, meaning that, consequently,

we needed to make sure that we kept the project web questionnaire online and operational until June 2004, in

order to take account of the fact that Chinese respondents would be encountering the film much later than

respondents in, for instance, the UK and the US.

[8] For the record, these were: Clerical/Administrative, Creative, Executive, Home/Child Care, Professional,

Retired, Self-Employed, Service Work, Skilled Manual, Student, Unemployed and Unskilled Manual.

[9]  These findings were presented by Ernest Mathijs under the title ‘Professional Activity and the Enjoyment of

Popular Culture’, at: “The Art of Comparison: 6^th ESA Research Conference on the Sociology of the Arts”, 3-5

November 2004; Rotterdam (the Netherlands).

[10] For instance, while we have identified some interesting patterns amongst certain groups who have chosen a

particular occupation category, we have also identified at least one case where a respondent found this approach

confusing. For, when interviewing this respondent, it became apparent that she was an 18 year old who had just

completed her secondary school education and was about to embark on a university degree, and had merely

chosen the category ‘Unskilled Manual’ in her questionnaire response because she didn’t know of a more

appropriate answer (amongst the options available) to this question.

[11] The best-known of these is, without question, Liebes and Katz’s study of the cross-cultural reception

of Dallas (Tamar Liebes and Elihu Katz, The Export of Meaning: Cross Cultural Readings of Dallas,

Cambridge: Polity Press, 1993).  To it have been added a steadily growing number of other studies, for

example, Annabelle Sreberny-Mohammadi, Dwayne Winseck, Jim McKenna and Oliver Boyd-Barrett

(eds.), Media in Global Context: a Reader, London: Arnold, 1997; Sonia Livingstone and Moira Bovill

(eds.), Children and their Changing Media Environment, Hahwah, NJ: Lawrence Erlbaum, 2001;

Janet Wasko et al., Dazzled by Disney?; Claudia Mitchell and Jacqueline Reid-Walsh (eds.), Seven Going

on Seventeen: Tween Studies in the Culture of Girlhood, NY: Peter Lang, 2005; and Maya Götz, Dafna

Lemish, Amy Aidman and Hyesung Moon, Media and the Make-Believe World of Children: When Harry

Potter Meets Pokémon in Disneyland, Mahwah, NJ: Lawrence Erlbaum 2005.

[12] Livingstone, p.481-2.

[13]  These findings will appear as a chapter in the main book of the project, to be published as Martin Barker and

Ernest Mathijs (eds.), Watching The Lord of the Rings, NY: Peter Lang 2007.

[14] This database of responses is now available for other researchers to investigate, via the UK’s Economic and

Social Data Services (ESDS) website.