This collaborative piece explores the potential synergy arising from the confluence of two growing areas of research, teaching, and practice in composition (broadly defined): multi- (or trans-) modality, and trans- (or multi-) linguality. As we discuss ahead, these areas of concern have emerged simultaneously—at least within the context of modern composition studies—in response to changes in the means and identities of people in communication practices worldwide. These changes challenge compositionists to rethink all that composition entails.1
However, despite their common points of origination, discussions of modality have remained largely separate from discussions of translinguality, to the impoverishment of both. We find this situation to be most interesting and worthy of exploration.
This collaborative piece is meant to redress this impoverishment by exploring the overlaps, parallels, and points of intersection between the two areas of concern. The collaborators have each been associated primarily with one of these two areas of concern. And this fact, too, gives us pause for thought.
Bruce Horner’s work addresses the dominance of monolingualist ideology in composition and poses what is termed "translingualism" as an alternative set of beliefs to address those problematics.2 Cynthia Selfe's work has been at the forefront of efforts in composition to explore and engage responsibly with the affordances of digital literacies.3 Tim Lockridge's scholarship focuses on how texts are composed for and move through digital spaces. He works to raise awareness of, and build tools for, digitally accessible texts, resisting practices that efface differences in access. Tim's digital compositions also resist our understandings of single authorship, demonstrating the always already collaborative nature of composing complex online texts and developing a trajectory of scholarship and service that approaches multimodal scholarship as an inclusive collaborative effort rather than the purview of those who might possess coding expertise.7
Despite the different trajectories and limited perspectives of our own labors, we all sense a need for a more expansive view and practice of composition, whether in terms of modalities or languages of expression, and a sense that we can stimulate and support efforts toward that goal by identifying overlaps and parallels in work towards it from questions about both language and modality. That shared sense is what has brought us together and—with the addition of Tim Lockridge and his expertise in design and coding multimodal texts—given us the encouragement necessary to work on this project.
This project originally began and developed as a (mostly email) dialogue between Cindy and Bruce, with questions followed by responses followed by responses and questions prompted by these responses and so on. As this dialogue developed, we started to identify several key issues, explain ways these issues manifest themselves in specific teaching, research, and composing practices, and pose questions and challenges prompted by these manifestations.4
We've organized the discussion that follows in terms of these key issues, recognizing that there is significant overlap between and among them. We refer occasionally to some of the comments and questions raised in the email process leading up to this text to help explain what prompted our statements. These quoted passages from the original email dialog are signaled with large blue quotation marks and an image of the specific author who wrote them. Comments that occurred during the drafting of the article are attributed to specific authors, but not signaled by the blue quotation marks. Thus, readers can differentiate, if they wish, between our early explorations of issues and our later discussions about how to be more precise and illustrative about using visual and aural modalities in conversation with the alphabetic modality.
In the fall of 2013, Cindy and Bruce asked me to join them on a collaborative project that explored the connections between transmodality and translinguality. They had developed a working paper, in MS Word & PDF format, and were interested in moving their argument to a digital, multimodal artifact.
In preparing for this shift, Bruce and Cindy had used the Adobe Acrobat annotation tool to begin a dialogue via comments—discussing the possible elements that might go into a multimodal piece. You can see an example of this early correspondence in the following screenshot:
A screenshot of Bruce & Cindy's initial correspondence.
In a typical print production workflow, these marginal comments might have any number of fates: discarded during edits, sent to the bottom of a desk drawer, marked as “resolved,” or maybe recovered many years later in a personal archive. To lose these comments, however, seemed a shame. For me, following this marginalia was a pleasure—an opportunity to hear two senior scholars work through a range of ideas and allusions, trading links and negotiating a collaboration. Could the reader, I wondered, have the same experience? Could a hypertext piece document how a project moves from a series of emails to a working paper to a larger conceptual whole?
There appeared to be an answer in the musicality of Cindy and Bruce’s original working paper. Their conversation (part “paper,” part annotation) had a weaving, harmonizing feel—two voices, like instruments, interacting and diverging and harmonizing. I searched for a matching technical metaphor: A way to place the argument on horizontal planes, echoing a musical staff. A horizontal scrolling motif, I thought, might enable the reader to see the voices intertwine, and through the use of different planes (or staffs), we could perhaps show two levels of discourse: one level for the core conversation (the project’s main argument), and another level for allusions, additions, and marginal notes.
I drafted several paper prototypes and searched for an HTML horizontal scrolling mechanism or framework. This was the first complication and point of tension I encountered, and it’s one that is relevant for the development of accessible multimodal scholarship: many solutions require tremendous expertise—or the ability to build a tool from scratch.
I firmly believe that web-based scholarship should be built with standards-based and preservable technologies (HTML, CSS, PDF, ePub, etc). But when the goal is to move beyond a simple web page or HTML container, the difficulty level quickly ramps. In the case of this piece, it wasn’t feasible (based on my Javascript skills and free time to learn more) to create a tool that fit my needs. Instead, I needed to build from the open-source work of others. But even that kind of iteration requires a number of specific literacies. Producing these texts can be difficult, challenging work.
After experimenting with a few different frameworks, we decided on reveal.js—a platform for building web-based slide decks. Although it didn’t have the specific horizontal presence that I thought would be best for Bruce and Cindy’s exchanges, it did offer a two-axis system. With Reveal, a user can scroll horizontally, from slide to slide, but also vertically, allowing one to “dig” beneath each slide. Reveal isn’t perfect for a scholarly hypertext project, but it is built on basic HTML (facilitating preservation) and has a great deal of flexibility.
Reveal also introduced problems. The slide metaphor proved especially difficult. The platform requires the reader to move through the piece in a page-by-page motion, and each individual page lacks a scrolling mechanism. This meant we had to break the core piece into discrete chunks that could each fit on a single screen. Each new section—and there were many—prompted choices about where to break paragraphs, where to build new sections, and where to attach supplementary materials.
A screenshot of an early version of this project, built with reaveal.js.
An initial vision—and something from the first prototype I sent to Cindy and Bruce—included the use of cinemagraphs and animated gifs as backgrounds. I thought these would bring a metaphorical and artistic element to the piece, and I hoped they might also affect the overall cadence. If we were to parcel this piece into discrete units, could we use these animated images as a way of encouraging pauses? How might motion work with and against the text? This seemed like a point of exploration and interrogation—a way we might make the multimodal genre (much like the meditative nature of the piece itself) push against the norms of the academic text.
This incited several searches: I created cinemagraphs and animated gifs for use in the text, Cindy scoured the creative commons for images we might use, and Bruce sent links to scores, compositions, and musical selections that might serve a similar purpose.
These searches became an extension of that initial impulse: to have the intertextual portion of the document extend beyond the “core” text of the piece. Once we opened the door to incorporating asides and marginalia, we discovered new avenues for mediation and collaboration. We began pulling at the metaphorical threads of the piece, looking for new pieces to sew in and possible points of further weaving. In this, we found a rich moment for collaboration—but also tangible examples that showed the difficulties of collaborating on web-based texts.
Bach's Invention #6 in E Major. Public Domain.
Our project, then, is meant to operate at two levels: on one level, it carries out a discussion of the overlaps, points of intersection, and parallels between work on translinguality and multimodality; on another level, it also (and, for readers, simultaneously) engages in meta-analysis of just such discussions, leading us to conclusions about how to develop such collaborative work in the most productive ways possible.5
I remember wanting to use the Bach invention and the train track image as two countering representations of relations between discussions of transmodality and translinguality: parallel but never intersecting (the train tracks) or complementing one another (the counterpoint between the two voices in the Bach, starting from different points and coming together). The Bach invention helped me think how to conceive of the potential relationship between the two discussions, vs. the train tracks. But whether readers/listeners/viewers get those specific ideas from our introduction of the train track image or the image of the Bach invention depends, of course, on their reading/viewing/listening practices (including training in these. For example, the image of the musical score for the invention would only enable a sense of the two musical lines and their counterpoint for those practiced in reading such musical scores). The video of the two hands playing the invention does provide a visual parallel, with the hand movements, to the aural counterpoint—but a sense of that parallel may only be produced for those practiced in, well, piano playing.
We shift, therefore, back and forth between excerpts from our dialogue and commentary on that dialogue to bring out assumptions and problematics of the terms with which we do, can, and might explore translinguality, multimodality, and their relations. We all found this to be difficult and unfamiliar work, and that itself is another notable commentary on its unfamiliarity and its relative rarity—at least within our experience.
To get a sense of just how difficult and unfamiliar we found the project to be, here is a passage from my email to Cindy to initiate the project. Note how the notion of adding images and sounds is offered only as a kind of afterthought:
"One way of proceeding might be to start with two brief—say no more
than 2000 word—overviews of work in each, one on translinguality, one
on transmodality, giving a little history of the emergence of these
terms and research and teaching on them. . . . We could follow these up with individually authored questions and comments in response to those overviews, including questions and comments pointing to issues not raised in the overviews, then individual responses to these. Ultimately, I’m hoping we can end with a passage—how long would be something we could decide
later— [ . . .] identifying key terms, points of intersection, questions for research, and so on based on our conversation. So the piece would be presented as a kind of symposium. . . .
Of course, there’s an argument for producing a representation of such a conversation in a form other than verbal written text (even one potentially including, say, images and diagrams)."
Our overarching assumption is that such back and forth movement is necessary to the responsible conduct of any such work: our goal is to resist quick and easy sloganeering and the commodification of composing practices that might otherwise have the potential to transform the work, and the understanding of the work, undertaken in composition by teachers, scholars, and students, with the aim of extending our understandings.
Our hopes in exploring this potential and confluence are that:
we can better understand each area of inquiry by defining it in relation to the other;
we can, given what we believe are the significant overlaps and alignments in the concerns of each, re-define each in terms of the other; and
we can better identify important questions for future research as a consequence of our efforts here to outline the current state of affairs in the research and teaching of both.6
To be more specific from my own perspective, I sense a confluence emerging in composition studies with the growing and converging interests in multi- (or trans-) modality, and trans- (or multi-) linguality. As a digital media scholar, I experience these convergences most often in digital environments, when I see people communicating across conventional linguistic barriers using video and audio and alphabetic texts in a range of creative ways.
Here, for instance, is my most recent favorite example of this confluence, these convergences: a cat video that Diana George called to my attention. This small and delightful text is captioned in both French and English; it deploys music and humor, moving images and alphabetic text; it crosses borders (species, language, culture, geopolitics) and communicates effectively. It's not a weighty or consequential academic text; it's not an argument or a research paper, a persuasive essay or a lab report, but it is an example, I think, that can serve to remind compositionists of some important truths: that millions of people every day enjoy the process of composing self-sponsored vernacular texts outside classroom walls; that such texts are motivated by a variety of purposes and aimed at a variety of audiences; and that, in such contexts, many authors often choose to mix linguistic and expressive resources in creative ways in order to accomplish their rhetorical goals.
Both modality and language represent deep reservoirs of design resources available to communicators, and we can learn a great deal by discussing the intersections, overlaps, parallels, and relationships among these resources: how they are used by authors/designers, how they are taught, how they are deployed.
Bruce’s work includes:
Horner, Bruce, Lu, Min-Zhan, Jones Royster, Jacqueline, & Trimbur, John. (2011). Language difference in writing: Toward a translingual approach. College English 73(3), 303-321.
Horner, Bruce, Lu, Min-Zhan, & Matsuda, Paul Kei (Eds.). (2010). Cross-Language Relations in Composition. Carbondale: Southern Illinois University Press.
Horner, Bruce, & Lu, Min-Zhan. (2007). Resisting Monolingualism in ‘English’: Reading and writing the politics of language. In Viv Ellis, Carol Fox, and Brian Street (Eds.). Rethinking English in Schools: Towards a New and Constructive Stage (pp. 141-57). London: Continuum.
Horner, Bruce, & Trimbur, John. (2002). English Only and U.S. College Composition. College Composition and Communication 53(4), 594–630.
Horner, Bruce, & Kopelson, Karen (Eds.). 2014. Reworking English in Rhetoric and Composition: Global Interrogations, Local Interventions. Carbondale: Southern Illinois University Press.
Cindy’s work includes:
Ulman, H. Lewis, DeWitt, Scott L., & Selfe, Cynthia L. (2013). Stories that speak to us. Logan, UT: Computers and Composition Digital Press/Utah State University Press. 2013.
Selfe, Cynthia L., & Hawisher, Gail E. (2004). Literate lives in the information age: Stories from the United States. Mahwah, NJ: Lawrence Erlbaum.
Wysocki, Anne Frances, Johnson-Eilola, Johndan, Selfe, Cynthia L., & Sirc, Geoffrey. (2004). Writing new media: Theory and applications for expanding the teaching of composition. Logan, UT: Utah State University Press.
Selfe, Cynthia L. (1999). Technology and literacy in the twenty-first century: The perils of not paying attention. Carbondale, IL: Southern Illinois University Press.
Here, I think it might be cool to think about what it means to have a synergistic dialog—maybe some music that illustrates what happens when a dialogic exchange yields more than the sums of its two parts...
I'm thinking an excerpt from a Bach fugue with counterpoint which has a different meaning than the “point counterpoint” idea in common parlance: the two voices work in relation to one another to produce harmony, albeit necessarily with harmonic tension through deployment of alterations of dissonance and consonance.
Viewers will want to turn off YouTube’s "Closed Captioning" option to avoid double captioning interference on this video.
Captioning, too, operates in parallel with another text. Together the captioning and the original text form a third text that is more than the sum of its parts.
We might want to talk here about specific artifacts that mark/trace our own ongoing struggles to produce texts that more nearly approximate our thinking and the difficulties that involves. For instance, we might want to show examples of a range of texts that readers are disposed to read as multimodal and multilingual—in ways that extend beyond the dispositions they generally bring to the print articles we have done.
For my part, I'm thinking of three texts that illustrate a range: the last CCC piece I did where key audio files (which existed online) had to be referenced by URLs in print, the comic that Will Kurlinkus and I did in the issue of JAC that focused on the 2012 Watson Conference, and in Transnational Literate Lives, with Gail Hawisher and Patrick Berry, which exists as a born-digital book:
Selfe, Cynthia L., & Kirlinkus, William C. (2012). The Watson Symposium: What Might Be Missing and Why? Journal of Advanced Composition 32(3). Accompanying blog at: http://watsonresponse.blogspot.com.
Selfe, Cynthia L. (2009). The Movement of Air, the Breath of Meaning: Aurality and Multimodal Composing. College Composition and Communication 60(4), 611-663.
Berry, Patrick W., Hawisher, Gail E., & Selfe, Cynthia L. (2012). Transnational Literate Lives in Digital Times. Logan, UT: Computers and Composition Digital Press/Utah State University Press.
I would include my essay, "Relocating Basic Writing." Journal of Basic Writing 30.2 (Fall 2011[2012]): 50-69. Especially the use of diagrams on pp. 58, 59 to represent different ways of conceiving language relations and practices.
Tim’s work includes:
Lockridge, Timothy, George, Diana, & Trimbur, John. (2013). Cultural studies and composition. In Gary Tate, Amy Rupiper Taggert, Kurt Schick, & H. Brooke Hessler (Eds.), A guide to composition pedagogies. New York: Oxford University Press.
Lockridge, Timothy, George, Diana, & Lawson, Daniel. (2012). The new work of composing is much like the old, only different. In Debra Journet, Cheryl E. Ball, & Ryan Trauman (Eds.), The new work of composing. Logan, UT: Computers and Composition Digital Press/Utah State University Press.
Lockridge, Timothy, & Cover, Jennifer. (2009). Icons and genre: The affordances of Livejournal.com. Reconstruction: Studies in Contemporary Culture 9(3).
In the fall of 2013, Cindy and Bruce asked me to join them on a collaborative project that explored the connections between transmodality and translinguality. They had developed a working paper, in MS Word & PDF format, and were interested in moving their argument to a digital, multimodal artifact.
I remember wanting to use the Bach invention and the train track image as two countering representations of relations between discussions of transmodality and translinguality: parallel but never intersecting (the train tracks) or complementing one another (the counterpoint between the two voices in the Bach, starting from different points and coming together). The Bach invention helped me think how to conceive of the potential relationship between the two discussions, vs. the train tracks. But whether readers/listeners/viewers get those specific ideas from our introduction of the train track image or the image of the Bach invention depends, of course, on their reading/viewing/listening practices (including training in these. For example, the image of the musical score for the invention would only enable a sense of the two musical lines and their counterpoint for those practiced in reading such musical scores). The video of the two hands playing the invention does provide a visual parallel, with the hand movements, to the aural counterpoint—but a sense of that parallel may only be produced for those practiced in, well, piano playing.
To be more specific from my own perspective, I sense a confluence emerging in composition studies with the growing and converging interests in multi- (or trans-) modality, and trans- (or multi-) linguality. As a digital media scholar, I experience these convergences most often in digital environments, when I see people communicating across conventional linguistic barriers using video and audio and alphabetic texts in a range of creative ways.