TextTagger: An Augmented Reality Annotation Tool for Books

Vishesh Kumar Indian Institute of Technology Guwahati vishesh420@gmail.com Rishika Jain Mauli Pandey Indian Institute of Technology Guwahati Indian Institute of Technology Guwahati Prabhat Kumar Indian Institute of Technology Guwahati

Abstract
This paper aims to report current progress on the design and prototyping of a project, that aims to augment books with a layer of online annotations and contextualized discussions on top of the content called TextTagger. TextTagger, aims to bring the ability of contextualized discussions in the form of notes and comments, common to a large variety of content across the internet, to the content of books, especially physical ones as well! TextTagger, currently conceptualized for physical books, adds a layer of online content to physical books, accessible via an interface on a smartphone. TextTagger has been designed for people to be able to add comments in relation to paragraphs on the books, and these comments to be accessible and available for conversation, by anybody else reading the same book via TextTaggers interface. We imagine TextTagger enabling the reading of books to be an additionally active engagement, rather than a one-way consumption of content not only allowing for people to share their thoughts on the content of direct relevance, but be able to interact with other peoples critique and thoughts on the same.

Guwahati, Assam - 781039 India Guwahati, Assam - 781039 India

Guwahati, Assam - 781039 India Guwahati, Assam - 781039 India rishika@iitg.ernet.in mauli@iitg.ernet.in Harshit Agrawal Indian Institute of Technology Guwahati Guwahati, Assam - 781039 India harshit@iitg.ernet.in

Paste the appropriate copyright/license statement here. ACM now supports three different publication options: ACM copyright: ACM holds the copyright on the work. This is the historical approach. License: The author(s) retain copyright, but ACM receives an exclusive publication license. Open Access: The author(s) wish to pay for the work to be open access. The additional fee must be paid to ACM. This text field is large enough to hold the appropriate release statement assuming it is single-spaced in Verdana 7 point font. Please do not change the size of this text box.

Author Keywords
Authors choice; of terms; separated; by semi-colons Optional section to be included in your final version, but strongly encouraged.

readers ends is also being employed by news publishers, at least in the online form of their content. On the other hand, peoples desire to engage with content being consumed is prevalent, noticeable, and useful, even in static media like books. The act of annotating books, with highlighters, or making notes in the margins is fairly common a practice called Active Reading[1], is known to increase effectiveness of read content, as it involves greater assimilation and critical analyses and is common amongst students, researchers, and anybody carefully reading and thinking about what they read[2]. These practices of annotation would be seen as acts of vandalism if done with library books; a value for not tarnishing printed paper that hinders the extent to which readers are encouraged to interact with the content they consume. This permanence of physical media, perhaps, lends it a notion of trust, but at the same time, also leads to stagnation, greater propensity to be outdated, and lesser cross examination by readers. With the concept of TextTagger, we have conceptualized a platform to enable the addition of digital annotations and comments on all physical books and allowing these to be accessible by other users as well, thus inducing added engagement in the experience of a book i.e. sharing ones thoughts, readings others, and having conversations on the same on TextTaggers platform. In the context of HCI, and trends about how social networks are spreading, tools that try to associate conversation and interaction in the real world, have recently taken to using geo-local and spatial information, as in the case of MIT Media Labs Second

ACM Classification Keywords

H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. See: http://www.acm.org/about/class/1998/ Optional section to be included in your final version, but strongly encouraged.

Introduction
The practice of storytelling has even preceded the creation of language as prehistoric men would paint on the walls of caves. Humanity has witnessed a long journey in the same, from using the spoken word, to writing it down a significant shift having been onset by Gutenbergs printing press. A common feature to these practices, has been the unidirectional consumption of content, from the narrator to the listener or reader. The most recent paradigm shift has taken place with the invention of the internet wherein almost all the content receives a response; is discussed by people who feel passionately about it, and wish to voice their opinions. This has led to a visible proliferation in the variety of peoples outlooks, and their engagement with the content they consume. The efficacy of platforms that invite engagement is clearly visible in the popularity of tools networks like Facebook, Quora, and the vast array of blogs across the internet. The facility of comments to enable greater engagement, and active participation from the

Surface[6] and the local network Circle[10]. In contrast, TextTagger aims to build social and relevant interactions around textual content, beginning with that of books.

The project of XLibris [8] explores the premise of using computation for active reading. XLibris is a highresolution pen tablet display, with paper-like interface that allows key affordances of highlighting and annotating in physical reading.

Related Work
We Applications of augmented reality in the domains of education, design, maintenance and other fields have increased significantly. AR supplements the real world with virtual (computer-generated) objects that appear to coexist in the same space as the real world. Software platforms of Vuforia [3], Layar [4] and Metaio [5] allow display of interactive digital content atop print media. By tracking the visual elements of print surface and using sensors in mobile phones, the applications create AR contents to bring together online and offline materials. There have been prior works using AR technology to enable multi-user collaborations and interactions. The project of Second Surface [6], which uses Vuforia, is an interesting platform for virtual content generation, and allows users to interact with everyday objects. In Second Surface, users can place three dimensional drawings, texts, and photos relative to objects and share this expression with any other person who uses the same software at the same spot. Magic Book [7] is another multiple-user tool to support collaborations on three levels on the physical book where users can read together, by means of augmented reality displays to show virtual objects on books pages, and an immersive virtual space where users can see each others virtual avatars. Figure 1. Barcode Scanner using zxing library Entering comments: After the book has been chosen, the user is expected to take a photo of the page where he wants to add a comment. TextTagger, using optical character recognition tools (tested with the Vuforia platform by Qualcomm), reads the text from the page.
!

Implementation and Interface

We explored a variety of avenues when attempting to solve the numerous technical and design problems we faced in conceptualizing this project. The following is a list of the discrete interface screens to be seen by the user, which act as hosts of different actions: Identifying the book: TextTagger can accept the books name as a text input, or a photo of the books ISBN bar code which leads to search results of which book it is expected to be.
!

Following this, it allows for highlighting text, and adding comments related to the highlighted text. Viewing comments: When the book is identified, the user is also given an optional and cursory list describing the approximate locations of other users comments across the book, if any. This is in case the user wants to review the readers responses before beginning, or be aware beforehand of which paragraphs have associated discussions.
!

find relevant comments all the comments are loaded on identification of the book, and the reader can willingly browse them whenever he wants to. Interacting with others comments: TextTagger includes threaded comments, so as to enable discussion among the readers popular across numerous commenting systems, Reddit, etc. This has the obvious advantages of enabling conversation that does not fill up the primary level comments, and is clubbed within context, and people who find it relevant.
!

Comment types: Though our prototype design and implementation only supported text entries, we considered and believed that providing for graphic and audio entries would allow for a greater variety media usage in this electronic conversation.
!

Limitations
In terms of usability, TextTaggers implementation is visibly cumbersome, as it involves switching from the physical book to the mobile phones screen. Different concepts were imagined and considered an appealing but infeasible idea being the projection of the augmented content by the mobile phone, on top of the physical book itself. Similarly, wearable technology that makes augmented reality commonplace, like the Google Glass, would also offer a convenient access to TextTaggers interface, not requiring the discrete step of taking photos from the smartphone The issue of moderation of content has also been foreseen as a challenging hurdle as not only misappropriate content will need to be filtered, but strict guidelines would be required so that seeing the comments does not ill-affect the reading experience. For instance, comments will need to have spoiler alerts

Figure 2. Cursory comments access view A significant inspiration behind contextualizing comments discussion was the rising web design phenomenon of comments connected with paragraphs, popularized by the web-publishing platform Medium, and on the rise across numerous popular web content generating sites. This allows for comments to address a specific section of the content, without having to re-quote from the body, and simultaneously be channeled to users who found those particular paragraphs of interest. Also, the interface and working was designed such that holding the phone on top of every page is not required to
!

in case they refer events that happen later in the book, thus affecting the experience of the plot. TextTaggers identification of books, and the contents inside to correctly identify where the comments belong, also requires greater sophistication. ISBNs do not account well enough for the same book from different publishers and slight changes in the content will need to be understood across different editions, for appropriate placing of comments. This issue might arise, in case the user looks for comments on page 86 of his edition of the book, which has slightly changed text and this leads to the comments placed in the alternate edition not being shown at the requisite place.

If teachers were given access to mediate this discussion, TextTaggers interface can be of further use in exercises of comprehension and critical reading, while learning, as well as practice. Extension to magazines and journals TextTagger can be easily extended to include magazines and journals. This would offer great utility to academicians to share their thoughts, as well as offer feedback or complementary thoughts to the authors.
!

Conclusions and Future scope

There is a large scope of scenarios where TextTagger can be used, either as is, or with some modification. Collaborative Learning among students TextTagger has a great potential to encourage discussions among students, and academicians alike. Online discussions have been seen to have positive effects on students learning [9], and thus, the usage of TextTagger in schools can enable substantially improved discussions and learning. Through TextTagger, students can leave tips about a particular topic, discuss solutions to exercise problems, or even suggest reviews of the book, helping fellow students in book selection in libraries.
!

Actionable content TextTaggers interface could also be used as a decentralized channel for doing specific tasks on the content, for instance translation of books. By using a crowd sourced channel that is popularly visible, the errors caused by giving it to singular people is reduced, and the task is done without decontextualizing the text.
!

Integration with e-books Books are being increasingly released in physical as well as digital formats. Integrating TextTagger with e-books, say Kindle books, would enable this conversation to not span a greater number of readers, as well as have an interface where seeing the attached comments is easier and does not require a distinct device and interaction.
!

To conclude, TextTagger explores a fascinating new avenue to use the concept of augmented reality/content on. To enable a layer of thoughts and conversation on the plethora of existing printed content, appears to have a vast potential of uses, and invites greater exploration and application.

References
[1] Schilit, B.N., Golovchinsky, G and Price M. Beyond Paper : Supporting Active Reading with free-form digital ink annotations. Proceedings of CHI98, pp. 249256 [2] Adler, M.J. and van Doren, C.(1972) How to Read a Book. Simon and Schuster, New York, NY [3] Qualcomm, Vuforia: http://www.qualcomm.com/solutions/augmentedreality [4] Layar. http://www.layar.com [5] Metaio. Junaio. http://www.junaio.com

Emerging Technologies, ser. SA '12. New York, NY, USA: ACM, 2012, pp. 20:1-20:4. [7] Billinghurst, Mark, Hirokazu Kato, and Ivan Poupyrev. "The magicbook-moving seamlessly between reality and virtuality." Computer Graphics and Applications, IEEE 21.3 (2001): 6-8. [8] Price, Morgan N., Bill N. Schilit, and Gene Golovchinsky. "XLibris: The active reading machine." CHI 98 Conference Summary on Human Factors in Computing Systems. ACM, 1998. [9] Wu, Dezhi, and Starr Roxanne Hiltz. "Predicting learning from asynchronous online discussions." Journal of Asynchronous Learning Networks 8.2 (2004): 139152. [10] Circle. http://www.circleapp.com

[6] S. Kasahara, V. Heun, A.S. Lee, and H.Ishii. Second surface: multi-user spatial collaboration system based on augmented reality," in SIGGRAPH Asia 2012