D4.8.2 BLUE Experiment Progress Report v1.0

D4.8.
2 BLUE Experiment Progress Report

2013-04-08
Ioanna Lykourentzou, Yannick Naudet, Eric Tobias, Younes Djaghloul (Henri Tudor Public Research Centre); Angeliki Antoniou, George Lepouras, Costas Vassilakis (University of Peloponnese)
This intermediate report describes the current implementation and status of the BLUE experiment, as well as first experimental results obtained. It provides updates on the FHW venue constraints, the final scenario for the experiment and the associated technical IT infrastructure and architecture.
www.experimedia.eu
EXPERIMEDIA
Project acronym EXPERIMEDIA
Dissemination level: PU
Full title Experiments in live social and networked media experiences Grant agreement number 287966 Funding scheme Large-scale Integrating Project (IP) Work programme topic Objective ICT-2011.1.6 Future Internet Research and Experimentation (FIRE) Project start date 2011-10-01 Project duration 36 months Activity 4 Experimentation Workpackage 4.8 EX8: BLUE Deliverable lead organisation Henri Tudor Public Research Centre Authors Ioanna Lykourentzou, Yannick Naudet, Eric Tobias, Younes Djaghloul (Henri Tudor Public Research Centre) Angeliki Antoniou, George Lepouras, Costas Vassilakis (University of Peloponnese) Reviewers Remi Francard (FDF) Version 1.0 Status Final Dissemination level PU: Public Due date PM18 (2013-03-31) Delivery date 2013-04-08
Copyright Henri Tudor Public Research Centre and other members of the EXPERIMEDIA consortium 2013
EXPERIMEDIA
Table of Contents
1. 2. Executive summary ............................................................................................................................ 3 Experimental setup............................................................................................................................. 4 2.1. 2.2. Venue constraints...................................................................................................................... 4 Final experiment scenario ........................................................................................................ 4
2.2.1. Implementation of personalisation, recommendation, and profiling............................ 4 2.2.2. Implemented scenario .......................................................................................................... 6 2.2.3. Experimental phases............................................................................................................. 8 2.3. 2.4. 2.5. 3. 3.1. Experiment IT infrastructure ................................................................................................ 10 Final technical architecture .................................................................................................... 12 Experiment software (Implementation) .............................................................................. 17 MMS: Gaming on Facebook to derive cognitive profile................................................... 25
First experimental results................................................................................................................. 25 3.1.1. Cognitive style and decoration style/ fashion style........................................................ 25 3.1.2. Cognitive style and avatars ................................................................................................ 27 3.1.3. Cognitive style and game preference................................................................................ 28 3.2. Preliminary simulation results of a personalised museum visit ........................................ 30 3.2.1. Role and scope of the simulator ....................................................................................... 30 3.2.2. Internal simulator modelling ............................................................................................. 31 3.2.3. Simulator implementation ................................................................................................. 33 3.2.4. Experimentation status and future work ......................................................................... 34
4. 5.
Ethics and privacy ............................................................................................................................ 40 Conclusion ......................................................................................................................................... 41 Dissemination, meetings and efforts ........................................................................ 43 Consent forms .............................................................................................................. 44
Appendix A. Appendix B.
EXPERIMEDIA
1. Executive summary
The goal of the BLUE experiment of EXPERIMEDIA is to explore the use of visitors' cognitive styles and content interest in order to personalise their experiences inside a museum. The related experimental runs will be conducted at the Foundation of the Hellenic World (FHW) in Athens. The goals and requirements for the experiment have been described in EXPERIMEDIA deliverable D4.8.1. This deliverable provides a status of the BLUE experiment, both from the technical implementation and the experimental point of view. It details the experimental setup (software and hardware) as it will be implemented knowing the current characteristics of the FHW venue and reports the preliminary experimental results obtained off site. Section 2 details the experimental setup. It starts with an update on the constraints induced by the FHW venue, compared to what was reported in deliverable D4.8.1. Then the final experiment scenario is given, highlighting the personalisation and recommendation aspects, and a status on the experimental setup is given, in terms of hardware infrastructure and software architecture. Section 3 details our first experimental results. The first ones concern the assessment and parameterization of the My Museum Story (MMS) application, while the second ones are simulation results assessing the theoretical impact of recommendations in a museum. Ethics and privacy issues are briefly reported in Section 4, before conclusion in Section 5. Annexes are given related to dissemination, meetings and efforts expended.
EXPERIMEDIA
2. Experimental setup
2.1. Venue constraints
Exhibitions that can be considered have changed since the moment deliverable D4.8.1 was written. The preparation of our experiments has been done considering the following exhibitions: (1) the permanent exhibition on mathematics, (2) the temporary exhibition on Kazantzakis (if it is still be present when the experiments take place), (3) the Tholos with 5 scheduled shows, and (4) the REENACT experiment if it is reachable in FHW premises. At the moment we write this document, the Kivotos virtual reality hall is out of order and we are not sure it will be available when the experiments will take place. In addition, we have no guarantee that there will not be additional changes from FHW until the end of next September 2013. In order to increase the number of possible exhibition recommendations, it has been decided to consider the different sections (7) of the mathematics exhibition and build different virtual exhibitions clusters from their combination. Section 5 details how this will be considered in the experimental scenario. Finally, to make this scenario possible, it is assumed that there can be multiple entries to this exhibition. This still remains to be confirmed by FHW. To further increase recommendation possibilities, given required development time and feasibility, we investigate a new functionality that would allow the visitors to leave geolocalised messages for their Facebook friends in the museum.
2.2. 2.2.1.
Final experiment scenario Implementation of personalisation, recommendation, and profiling
To enrich user experience in the museum, BLUE offers both personalisation and recommendation. While the latter is also a form of personalisation, we refer to personalisation as the adaptation of something, in the form of an alteration, specifically for the user. Recommendation consists in offering suggestions to the user, tailored to his profile, usually including interests or needs. Both, personalisation and recommendation require user profiling. The latter, can be done explicitly by asking the user, or implicitly by observing their behaviour and recommendation consumption (in case of recommendation). In BLUE, the part of the user profile that will be exploited is mainly related to the nature of the cognitive style of the museum visitor. Personalisation in BLUE concerns the following elements which are personalised according to the cognitive profile and, where applicable, to the visitors personal interests and preferences: Written exhibition descriptions, specialised for each combination of the cognitive style dimensions. Language used in applications (MMS and MMG), tailored to the users origin, either Greek or English. User Interfaces of the MMS and MMG applications (Optional).
EXPERIMEDIA
Recommendations in BLUE are computed according to three main elements: (a) the visitors cognitive style together with a set of linked behaviour rules and interests, (b) personal interests (in relation with the museum exhibition topics), and (c) a set of rules related to the current context/situation (i.e.: time constraints, visitors current and previous activity, position of the visitor, status of a waiting queue). They comprise the following elements: Exhibitions to visit, respectively points of interest (POI) to go to. According to the FHW state and constraints, the considered exhibitions are: Mathematical exhibition and all the possible section groups, Nikos K. exhibition (if still present), Tholos shows, and the REENACT experiment if available. Other POIs are: shop, restaurant, places where to take photos, and messages left by friends. Type of ticket that matches user preferences and his time constraints. Path to follow inside the museum, i.e., the sequence of exhibitions and points of interests to see to complete the visit. Spontaneous actions to perform (e.g. Take a picture, Rest, Write a comment, etc).
The visitors profile to be used for personalisation and recommendations comprise the following elements: A cognitive profile, which will be obtained for each visitor after he has fulfilled a minimum number of steps through gaming in MMS or by going through several interactive screens in MMG. Cognitive profiles are predefined as a combination of cognitive style dimensions, where each possible combination constitutes a stereotype to which visitors having the corresponding behaviour will be affected. A set of behaviour rules is linked to each cognitive profile, pre-determined according to state of the art knowledge and previous studies. Common to each person belonging to a cognitive profile, these rules will be mainly used for action recommendations and content presentation. Finally, a set of interests will be also linked to cognitive profiles, related to museum exhibitions. They will be obtained principally from MMS through analysing objects collected by users within the game (which are related to museum exhibition topics) in a set of preliminary studies. Personal interests will also be retrieved from MMS and might be affected to a cognitive profile if evidences are retrieved from a majority of visitors with a same cognitive profile. As an option regarding the short timeframe of the project, they might also be deduced from observations on visitors behaviour in the museum regarding visited exhibitions and points of interests, and their consumption history of recommendations. Some demographic data (e.g. age, gender and linguistic background), extracted from Facebook visitor accounts. Current activity as determined by event scheme if applicable. Note that only user actions can directly be related to engaging in an activity. Implicit user activity deduction may be possible based on geolocation for instance. Time constraints regarding the availability of the user for a visit.
EXPERIMEDIA
2.2.2.
Implemented scenario
Following the initial scenario defined in deliverable D4.8.1, we describe in the following how it will actually be implemented knowing the constraints induced by the FHW venue. Figure 1 illustrates the inputs/outputs of the components involved in the personalisation and recommendations strategies of BLUE, namely: the visitor (user), MMS and MMG, the visitor profile, and the recommendation system.
Figure 1. Personalisation and Recommendation strategies in BLUE.
T0 - Before going to the museum. The visitor can play with the MMS application. After he has reached a sufficient play time, MMS determines his initial profile. From the list of profile elements described here above, this initial profile comprises a cognitive profile and personal interests, as well as useful demographical data. After the game play finishes, MMS calls the recommender system and proposes the visitor a personalised visit recommendation (i.e. visit recommendation including personalisation), presented as the following steps: 1) Exhibitions of interests are proposed, all with a personalised description and access to complementary information if available. This is an ordered list presenting exhibitions together with a weight related to visitor interests. 2) Once the visitor has made a choice among the recommendations, the system proposes a choice of sequence of exhibitions to see, including other points of interest, accounting for time constraints (of museum and of user if known) and personalised. Tholos shows are included at the time they have been scheduled by the museum. 3) When a sequence has been chosen, the system recommends the right ticket(s) to buy (single, sequence of singles, or one day ticket), different info about the exhibitions), potentially opening a new room in the MMS game. 4) Visitors are proposed then to make a reservation for a MMG device for the day of their visit. T1 - Before the visit, on the museums premises. MMG is given exclusively to visitors having reserved it, having bought a one-day pass or enough single tickets for different exhibitions justifying the interest for recommendations and guiding through MMG. The latter choice is
Copyright Henri Tudor Public Research Centre and other members of the EXPERIMEDIA consortium 2013 6
EXPERIMEDIA
justified by the limited number of devices for the experiment and to the lack of recommendation usefulness when only one or two exhibitions are accessed by the visitor. In case the visitor has not played with the MMS (i.e. T0 has not been realised), he will be asked to go through a series of screens in the MMG allowing to estimate roughly his cognitive profile. Then MMG calls the recommender system and proposes the visitor a personalised visit recommendation. Compared to T0, only step 2 is performed (a visiting path is given) since the visitor has bought a ticket, restricting what he can see. T2 - During the visit. MMG offers reactive personalised recommendations adapted to the visitors cognitive style and interests, context (position, activity, and time constraints), recommendation consumption history, and behaviour rules if existing. Recommendations are presented in a way that fits the visitors cognitive profile, demographic characteristics (i.e., language) and interests (i.e. their description is personalised). The MMG offers the following recommendations, having all the above described characteristics: Visit sequences (including exhibitions and other POIs), computed from the actual position of the visitor in the museum. Exhibition choice is limited by the ticket chosen and time constraints, which can also be modified during the visit. If the visitor has prepared his visit using MMS (step T0), MMG reminds him the visit sequence he has chosen. Each time the visitor reaches a POI that was not on the path proposed by the system, the possible sequences are dynamically re-computed. Actions to perform, at a specific place or according to a given user activity. Rules that are used to recommend actions are specific to each cognitive profile and take into account two dimensions of context; a) Item contextual elements, which are a set of variables related to the status of exhibitions (exhibition full, queue, ...), b) Users contextual elements (current activity, position...). All along the visit, the visitor is tracked and his recommendation consumption recorded. This allows to dynamically adapt some recommendations according to his situation, and to know at analysis time to which extent the recommendations have been accepted. Finally, the visitors can leave messages to their friends on Facebook at specific locations in the museum. Those messages are considered as POIs by the recommending engine. However, as it is thought to be an enticement, a kind of treasure hunt, finding the message is recommended without revealing any specific info to the visitor.
T3 After the visit. Once the visit is finished, before giving back the MMG device, the visitor can choose to upload his visit onto Facebook if he logged in. The MMG will post several items, including photos taken, to complement the visitors experience. At that time, whether the visit is uploaded or not, the visitors MMG session is purged from the device. The visitor then gives back the MMG device and he is asked to fill in a questionnaire to gather feedback. Such explicit feedback as well as implicit feedback (gathered from the observation of user behaviour through observables we have defined), will be used to refine both cognitive profiles (more particularly the attached behaviour rules and interests) and potentially personal interests in the case we can make multiple experimental runs with the same persons and if there is enough evidence to make such
EXPERIMEDIA
deductions. Depending in the numbers of participants, we might have to shift to qualitative data collection, possibly in the form of semi-structured interviews. The specific case of the Mathematical exhibition. As explained in the previous section, the Mathematical exhibition constitutes a specific point of interest in the recommendation process since it will generate multiple virtual exhibitions. Due to its specific structure, consisting of seven almost independent thematic sections, it is possible to ask the visitor to split his visit in parts, based on other restrictions (i.e. Tholos shows with predetermined times). The seven sections will be combined to form different groups of sections that can be seen altogether, according to a specific topical dimension (ex. antiquity or not, theory or practice...) or cognitive style. Hence different parts of the mathematical exhibition will be recommended, potentially at different moments in the visit and to different cognitive profiles. The full exhibition might also be interested to experiment the visiting style as we expected before starting the project. All of this is under investigation and the related choices, implementation and results will be reported in the future deliverable D4.8.3.
2.2.3.
Experimental phases
The experiment comprises three phases. The first phase is dedicated to preliminary studies and simulations allowing tuning the different components of BLUE correctly. The second phase is a controlled experiment that will be conducted with students from UOP. The third phase is an open experiment with common FHW visitors. At the current stage of the project, phase 1 has been realised (although new simulations will still be conducted), and phase 2 is expected to start in one or two months (i.e., April / May 2013). Phase 1, preliminary studies and simulations. This phase aims at calibrating the MMS component so that it can compute at best the cognitive profile of users, and at assessing the recommendation scenario in museum through simulations. Results obtained so far are presented in Section 4. Phase 2, controlled experiment. This experimental phase has as main objective to assess the validity of the initial set of interests and behaviour rules attached to each cognitive profile. Since initial choices are driven by previous studies, phase 1 results and common sense, it is important to confront initial settings to the reality and make necessary adjustments. From a recommendation point of view, the initial settings also avoid the cold start problem, but we can hardly expect having very good results without having learned from users behaviour and feedback. Consequently, phase 2 will be conducted with students from UOP (10-20) who will go through the complete scenario, i.e., from T0 to T2 as described hereinbefore. In order to reduce the variables influencing the experiment focus, the following constraints are applied for all students: All use a one day pass. Visitors time constraints are not taken into account. The visiting path is fixed for each visitor: the best possibility is computed by the system and they cannot deviate from the proposed path during their visit. An interview after the experiment will allow us to know if they liked or not.
EXPERIMEDIA
As a result of this phase, it is expected to refine the initial postulates and choices regarding the link between cognitive styles and corresponding visitors behaviour (i.e. interests and rules). This will be done by analysing the correlation between the proposed recommendations and the consumed ones, and using the students feedback as asked at the end of the experiment. A new and more accurate cognitive style rules mapping will be proposed and will be used in the next experimental phase. The UOP students are participating mainly for practicality reasons, although it is not a representative sample of the general population. However, BLUE does not target the general public but focuses on Facebook users and especially Facebook gamers. In that light, the choice of the participating sample can be considered as acceptable, since students are indeed expert Facebook users and in their majority they are also expert gamers. Phase 3, open experiment. Accepting the above issues of possible sample biases, it was decided to open the experiment to the general public visiting the museum. In this phase the experiment is less controlled, as visitors are not selected beforehand. It is expected to be conducted for at least one month during which visitors will be proposed to use MMG for their visit. Compared to the previous phase, the following changes occur: Step T0 is not necessarily realised. It cannot be guaranteed that all participants will play with MMS, nor that they will come to the museum having prepared their visit and bought a ticket following the recommendations of our system. In order to increase the chances to have T0 realised by visitors, advertisements for the experiment will however be proposed at the museum and on its web site (if possible), as well as through Facebook networks. Constraints of phase 2 are removed, making the visitors free to set their time constraints and choose their preferred visiting path.
In order to assess the usefulness and accuracy of recommendations based on cognitive profile, and their impact of QoE, visitors will be sometimes given recommendations actually computed from their estimated cognitive profile, and sometimes not. This will virtually create two groups: one guided from their cognitive profile, and the other guided by recommendations based on a same unique neutral profile. This neutral profile will be built so as not to reflect the behaviour corresponding to one true cognitive profile, or to reflect all the cognitive profiles, to derive recommendations that are independent from cognitive styles. This could be computed from averaging or aggregating over the set of real profiles; the choice will be made later on depending on the experimental context. We might also consider providing recommendations based only on personal interests without using the cognitive profile, if enough such interests can be gathered on a sufficient number of visitors. Both experimental conditions visitor satisfaction levels and use issues will be compared.
EXPERIMEDIA
2.3.
Experiment IT infrastructure
The server experimedia.uop.gr has been setup according to the EXPERIMEDIA Baseline Components v1.0 document. In more detail, the server has been setup as follows:
Functionality Operating system: Database: Messaging system (AMQP Bus for ECC external interfaces and ECC internal interfaces) Development tools and generic utilities: Software component Ubuntu 12.0,4 LTS PostgreSQL 9.1 RabbitMQ, release 2.8.4 Java 1.6 JDK/RT log4j 1.2.17 junit 4.10 hamcrest core 1.1 Tomcat 7 Apache 2 base64 2.3.8 gson 2.2.2 restletapi 2.0.1 restlet.ext.json 2.0.5 spring framework 3.0.5 jackson-mapper-asl 1.9.4 javax.servlet-api 3.0.1 json-lib 2.4 jQuery jqPlot Zurb foundation framework Date.js Modernizr Maven jetty plugin Nagios Core5 3.4.1 Juju beta (Ubuntu 12.04) 0install 1.6 (Ubuntu 12.04) SAM 16.0
Application container Web server EM/Monitoring system libraries: EM/Live metric view libraries/underpinnings:
EM/Experiment resource monitor: EDC/Service instance manager: EDC/Software installation system: ESC/Security modelling tool
The software has been configured regarding users and their credentials, access rights, initial database schemas, etc. Particular care has been taken to enhance the security of the system, so as to guarantee the integrity and privacy of the programs and, more importantly, the data it will host. To this end, a number of technical measures have been put into effect as listed below:
EXPERIMEDIA
Measure Remote administrative access to the machine, as well as traffic involving personal data (including passwords) should be performed only over encrypted connections. The software installation should be minimal, to limit the presence of vulnerabilities in the system. Frequent checks should be performed to detect whether security updates have been issued for the running software; security updates should be installed as soon as possible. Operating system-level accounts should be only granted to users that have a need to perform logins. A firewall must be installed, allowing access only to ports and services needed for the fulfilment of the tasks. Details
Fully implemented. ssh, postgres, rabbitmq, web access and application container access (tomcat) are provided on top of SSL12. The installed software has been limited to the absolutely necessary. Weekly automated updates have been scheduled to bring the software up-to-date, with minor disruption to the operation of the machine. The ufw firewall provided by Ubuntu has been used to filter network traffic. Access is only allowed to services that the machine should offer through the network, either to the public or to the development and administration staff. The services to which access is allowed are ssh, http, https, rabbitmq, postgres, and the application container (tomcat). All users have non-null passwords. Wherever it was possible, policy rules have been enforced to disallow null passwords. Default users were removed/disabled wherever possible; in all other cases, their passwords have been changed. A full weekly backup has been scheduled to be performed automatically.
Null passwords should not be used or allowed.
Default username/password combinations should not be used Backups should be made to guarantee that users can obtain a copy of their data, even after crashes.
The authenticity of SSL-based communication is ensured using a certificate obtained from the TERENA Certification authority (CA), which is recognised as a valid CA by all browsers. The only exception to this rule is the encryption of the Postgres network traffic, where a self-signed certificate is used. This is due to the fact that the process through which the certificate was obtained uses intermediate CAs (known as a trust chain) and this feature is not well-supported in Postgres. Database The Postgres DBMS running on experimedia.uop.gr will store the necessary data for running and monitoring the experiment, supporting the functionality of MMS and allowing for recommendation generation. To this end, two databases will be hosted on this DBMS:
1 2
Web: https://experimedia.uop.gr Application container: https://experimedia.uop.gr:8443
EXPERIMEDIA
a database for monitoring the experiment and reporting on it. This is compliant to the ECC metric model specified and detailed in Section 1.4.1, of the ECC metric model overview, a database for storing the data needed by MMS and the recommender. EXPERIMEDIA Content Component (ECC) The EXPERIMEDIA Content Component has been successfully deployed on server experimedia.uop.gr. The modules of the EXPERIMEDIA Content Components deployed are as follows: the RabbitMQ messaging system, the Postgres DBMS, within which a database for storing the experiment data manager metrics has been created (database (1), listed in the previous paragraph);
Access to all above resources is protected through passwords, while communications are encrypted using the secure socket layer (SSL). MMS The first version of the My Museum Story has been implemented using technologies such asHTML5, PHP, Javascript, CSS, jQuery and Ajax. The Facebook SDK for PHP (v.3.2.2) is also used in order to access the Facebook server-side functionality, including the features of the Graph API and FQL. FQL queries are implemented to extract the users Facebook data such as their names, age and number of friends. Apart from the already mentioned, SQL queries are also used to extract and commit information from, respectively to the Postgres database. These queries include the users game activity and Facebook data. . As far as the game functionality is concerned, in this version, users can choose their avatars, navigate their museums, play selected mini games, and earn randomly selected exhibits for placing in their museums. MMG The first prototype of the My Museum Guide has been implemented and tested using a Samsung Galaxy Tab 10.1 of the GT-P7510 family, upgraded to run Android 4.0.4. The development has later been switched to Samsung Galaxy Tab 2, the 7 inch version, of the GT-P3110 family which is shipped with Android 4.0.4. The latter device will also be deployed in the FHW which is why all current development is made using that tablet. 8 such devices have been bought so far, and we intend to have 2 or 4 more if possible. WIFI access points In addition to the existing WIFI access points available in FHW, we have bought 6 Netgear WG602-400PES 54Mbps WIFI access points that will be deployed in the museum. This will allow to have a correct calibration of the indoor location system.
2.4.
Final technical architecture
The global architectural diagram has undergone two small changes from the one proposed in D4.8.1. The revised architecture is illustrated in Figure 2. The ECC client implemented on the MMG does not support push behaviour in order to render it completely passive in that domain. Furthermore, the Recommender does not force a profile update as the cognitive profile is either
EXPERIMEDIA
know, by playing the MMS prior to using the MMG, or the MMG will require the generation of a rough cognitive profile in its next iteration. Therefore, the functionality has become obsolete.
Figure 2: Revised architecture
The MMG implements an ECC client which registers with the ECC instance running on the server. Once registered, the ECC will retrieve the metric generators and manage the data pulling schedule. The client runs in Android as a service to be separate from the GUI and be transparent for the visitor. The client is located in the lu.tudor.ssi.kiss.experimedia.blue.mmg.ecc package and consists of several classes used to implement all necessary interfaces as well as manage the data compilation process. The part of the EXPERIMEDIA Social Content Component (SCC) that is used in this experiment is the custom implementation of the SocialAuth library. It is used to provide visitors with an easy-to-use facility to log into Facebook and allow for the MMG to retrieve the visitors validatedID. This allows us to determine if the user has an existing cognitive profile or if the MMG needs to, in the next version, present some preliminary cognitive profile query screens.
EXPERIMEDIA
Figure 3: Package structure overview
The software enabling to conduct the experiment is logically separated into two packages as shown in Figure 3: 1) The lu.tudor.ssi.kiss.experimedia.blue.mmg package contains the implementation of the My Museum Guide. The implementation being done for Android, the structure separates the application by activities, fragments and tasks. The main package groups all Android activities that implement part of the application flow as seen in Figure 6, namely the splash screen, the legal information asking for user consent, the behaviour for not wanting to participate, the functionality to log in with an existing Facebook profile, and the main application window. Those activities are joined by an application activity that do not corresponds to a screen but is used as a mediator to hold application-wide data and to make sure Android lifecycle changes do not tamper with the data. The main application window includes tabs, Android fragments, which cater to delivering specific functionalities. Hence, they are separated into their own packages. The content package contains the fragment and necessary classes to display personalised content to the users. As of now, since we are still in the process of extracting and generating content, the package contains stubs. The recommendation package contains all classes necessary to build the list of recommendations gotten from the recommender. The visit package displays items that a user generated during his visit. This includes exhibitions visited, pictures taken, and comments made. This package relies on the correct attribution of visitor position to attribute these items to the correct visit item, or create the visit item in the first place. The geoloc package contains the Android service used to run the indoor localisation algorithm in the background as well as the fragment to display the map. In addition to the packages catering to Android activity and fragment behaviour, there are two packages to integrate EXPERIMEDIA components. The ecc package holds the implementation of an ECC client and the social package holds classes
EXPERIMEDIA
necessary to use the SCCs SocialAuth library extension. The util package holds classes necessary to use certain functionalities such as multi-tiered lists in the visit fragment. 2) The lu.tudor.ssi.kiss.experimedia.blue.reenex package holds the Recommendation Engine for EXPERIMEDIA (ReEnEx). The package also exposes a mediator that relays all service calls from the MMGs recommendation service. Incoming messages are either client registrations, calls for position resolution, or events. Position resolution is handled by the proximity package which will treat position related updates and queries and produce proximity related events for the recommender to react to. Events are implemented in the event package. The model of the data to be recommended is implemented by the content package. It defines recommendable items and their structure as well as restrictions the items can have in regard to availability and recommendability. They are used to build rules on which the recommender draws to be able to compute recommendations.
Figure 4: Package diagram showing high-level dependencies
The main recommendation algorithm that will be implemented is presented in the Figure 5. At the beginning, an initial filter is applied on the list of exhibitions. The filters criterion is the visitors time constraint. The result is a reduced list of exhibitions that can be proposed. One starts by this filter to optimise the recommendation creation in the next steps. Based on this list, one calculates the matchmaking between the visitors interests and each exhibition. The matchmaking is done by using two parameters; the content tags attached to each exhibition and the set of interests of the user. We decided to separate between the matchmaking based just on interests of the cognitive profile and those that are related to the specific user. The idea behind
EXPERIMEDIA
this decision is to keep the provenance of the recommended exhibitions in order to refine, after phase 1 and phase 2, the attached list of content tags for each cognitive profile. The user will select a sub-list of exhibitions. It will be considered as the main input for the mechanism of path creation. Each path is considered as suggested recommendation for this visitor. For each path on proposes also, a list of tickets that satisfy the recommendation. Since it is not allowed to by ticket during the visit the recommendation of tickets is available only before the real visit. During the visit, the context listener is activated, and the rule based recommendation is available. The latter consists of a set of rules that suggest spontaneous actions based on each possible modification the context of the visitor. The list of rules is attached to each cognitive profile. If the visitor does not consume a recommended action from the proposed path, a new recommendation is as proposed as done in before the visit but without proposing tickets.
EXPERIMEDIA
Figure 5. Main recommendation algorithm
2.5.
Experiment software (Implementation)
MMG As detailed in Section 2.3, the MMG runs on a Samsung Galaxy Tab 2, 7 inch model, hosting an Android 4.0.4 operating system. With the implementation tightly intermingled with bits and pieces of Android-specific code, we chose to present the functionality as implemented in the MMG prototype. Figure 6 shows the states the application will traverse from the initial launch to the user choosing to exit the application. Please note that the case when the application is killed by the operation system is not covered as it can never be detected by the application and the diagram was meant to be kept simple. The application features two complex states. The first unnamed state covers the initialisation process. All Android lifecycle events causing the application to pause such as
EXPERIMEDIA
Pause or Home, will cause the application to terminate and clear itself from Androids application stack. This is done to enforce that the consent is given prior to logging in and by the same entity as far as it can be controlled. The second state, Main, will not be subjected to such harsh constraints. Pausing or navigating Home will simply cause the application to reside on the stack. Note that upon return, the application resumes in the same state rather than in the Recommendation state, the diagram has been simplified in that regard. Take Picture is a special action and will result in the application to hand over focus and control to Androids camera application. This is also not modelled explicitly. The following paragraphs will guide through the screens associated with the states and explain some of the functionality.
Figure 6: State machine modelling MMG screen flow and transitions
My Museum Guide in its current version will see the user navigate a few introductory screens before being able to use the main functionality of the application. All named references in brackets refer to states from Figure 6. The initial splash screen (Splash), Screen 1: Splash, will show a logo to the visitor. The second screen (Legal), Screen 2: Legal, will present legal information to the user, namely the consent form. The user can choose to not accept the conditions upon which he will be presented with the goodbye screen (Bye), Screen 3: Bye. If he accepts, the login screen (Login), Screen 4: Login, will be shown and the visitor can choose to log in or to continue without logging in. If he chooses the first option, the activity holding the view will call upon the SCCs SocialAuth implementation to log the user in as shown in Screen 5. Upon navigating away from the login screen, the visitor is presented with the main application window (Main), the visitor will by default see his list of recommended items (Recommendations), Screen 6. The visitors are free to navigate the tabs. The content tab (Content), Screen 8, will show content regarding the closest recommended item or point of interest. This has yet to be determined as content extraction is still in progress and we are unsure
EXPERIMEDIA
what will be more meaningful to the visitor. The map tab (Map), Screen 9, displays the visitors position. A custom-made low zoom level map is currently being generated and will be available through TILT to convey a better sense of location to the museum visitors. During a visitors journey through the museum, the visit tab (Visit), Screen 7, will be populated with exhibitions he has visited. By default, the museum itself is always the first item and serves to hold visit items that cannot be attributed to a specific exhibition or point of interest. The presentation is a multilevel, expandable, list which allows the user to post comments and annotate pictures he has taken. Pictures can be taken any time by using the Take Picture Android Action Button. It will launch an Android Camera application, allowing the visitor to take a picture, and then return the control to the MMG application. Visitors are also able to exit at any moment. They are then confronted with several options: Post visit on Facebook (only available if logged in), exit without posting the visit, or resume the application.
Screen 1: Splash
Screen 2: Legal
Screen 3: Bye
Screen 4: Login
Screen 5: Login using the SCC's SocialAuth implementation
Screen 6: Main application showing Recommendation tab and expanded Action menu
Screen 7: Main application showing Visit tab with extended visit list
Screen 8: Main application window, placeholder for displaying content
Screen 9: Main application showing Map tab displaying the visitor's position
EXPERIMEDIA
Recommender The recommender service consists of two parts: the core recommendation service and the peripheral recommendation system which defines resources to recommend. These resources, and therefore resource definitions, are shared with the My Museum Guide application to ensure interoperability. The systems entry point conforms to the mediator pattern and is able to receive all calls from the My Museum Guide application and relay the request for a service to either the core recommendation service or a peripheral recommendation system. MMS The game is available on Facebook, implying that users have a Facebook account which they use to log in. The application stores some personal information that the user already provides on his Facebook page, like the users validateID, age, or gender, which can be used further in the personalisation process. The users validateID is important because he will use it to use the mobile application once at the museum. Without it, we would be unable to identify the user and match her to a cognitive profile. The age of the user is also important, since it correlates with a cognitive style dimension (Judger-Perceiver). Finally, we might be able to correlate the gender to different museum exhibits, once we have results from an ongoing study. The purpose of the game is for the user is to build her personal museum. Each choice the user makes is calculated in the estimation of her cognitive profile. The table below shows how the different game features, to be presented here, can be mapped to cognitive style dimensions.
Facebook Game element Extraversion Introversion Sensing iNtuition Thinking Feeling Judging Perceiving
Avatar: Mad Scientist Avatar: TV Persona Avatar: Judge Avatar: Diplomat Avatar: Engineer Avatar: Artist Avatar: Old-wise Avatar: Alien Avatar: Rapper Tool: book Tool: heart Tool: clock Tool: discoball Background music: classical (future work) X X
X X X X X X X X X X X X
EXPERIMEDIA
Facebook Game element
Extraversion
Introversion
Sensing
iNtuition
Thinking
Feeling
Judging
Perceiving
Background music: contemporary (future work) Pet: dog Pet: cat Pet: monkey Pet: gold fish Pet: owl Number of posts and comments: high Number of posts and comments: low Gestalt: general view Gestalt: details Choice of games: adventure corner, taking risks Choice of games: not choosing simulation games Choice of games: asking and inviting friends Decoration style: classic Decoration style: pop Fashion style (future work) Museum template (future work) X X X X X X X X X X X
X X
The first game screen asks the player to choose an avatar. The different avatars represent different ends of the cognitive style dimensions. The mad scientist uses the stereotypical view of an introvert individual, lost in his own studies. The TV persona represents extraverts. The Judge is for people that view justice very important (Thinkers). The Diplomat is for people that wish to avoid conflicts and find a middle solution (Feelers). An Engineer is for people that prefer clear instructions and create practical solutions (Sensors). The Artist is for individuals that have vivid imagination and increased creativity (Intuition). The Old-wise is for individuals that like to plan things in advance and like routines (Judgers). The Rapper is for individuals that like to mix work and fun, question rules and like action (Perceiver). Finally, the Alien is for people that feel different from the rest of the world (Introverts). Initial results (to be presented in a section below) showed that although certain avatars are very good predictors of cognitive style dimensions, other are not and this might be simply because of an ambiguity in the way users can perceive the different stereotypes we are implying. For this reason, we will accompany the
EXPERIMEDIA
different avatars with a key sentence for each one, stressing the stereotype we want to focus on. For example, a diplomat is someone that avoids conflicts and tries to find balanced solutions, etc. The following figure shows the avatar screen.
In the second screen the player chooses her pet. The dog implies high Intuition. Dogs can plan for the future since they can hide their bones for later (this characteristic will be known to the player). The cat implies high Sensing, since she will eat her food immediately and lives in the now. The monkey implies high Extraversion since monkeys like interaction with others and live in groups. The gold fish implies high Introversion and the Owl implies Judgers, since it is associated with wisdom.The following figure shows the second game screen.
EXPERIMEDIA
In the third screen the player chooses her tools and she can only use one of the following with different properties and functions in the games. The book represents Thinkers since it is associated with the search for facts and logic. The heart represents Feelers, since it is associated with sensitivity to peoples needs and feelings. The clock indicates a preference for punctuality (Judgers) and the disco-ball can represent both extraversion and need for fun (Perceivers). The following figure shows the third screen of the game.
Entering the museum the player can see that in the different museum rooms there are empty item holders and frames. The player needs to populate the item holders and the frames with items. The following screen shows an object holder with an item and an empty item holder. The player approaches the item holder once the cursor is on them, she can have a selection of game, she could play in order to gain an item. Once she finishes the game successfully, she is also presented with three different items she could choose one. All items are from three larger
EXPERIMEDIA
categories, similar to the categories of exhibitions currently hosted at the museum (ancient history, mathematics, natural sciences), as an attempt to capture museum specific preferences (to be used at the museum and the recommender).
The type of game the player chooses is also important and can provide information about different personality traits. The game ends when the user decides that their museum collection is complete, and arranges the items inside it. It can also end sooner but a minimum number of items have to be completed.
EXPERIMEDIA
3. First experimental results

3.1. MMS: Gaming on Facebook to derive cognitive profile
A number of pilot studies have been designed and some are already executed. These studies aim at checking whether all game characteristics (i.e. avatars, tools, pets, decoration style, etc.) correspond to actual cognitive styles of users. We used a small and biased sample, 51 - 66 (depending on the study, see specific results for the different experiments below) students from the University of Peloponnese, Department of Computer Science and Technology. Thus the results provided here, only offer indications of possible tendencies and correlations rather than proven facts. These pilot studies also provide a starting point for further data collection from a different set of users in the near future. For this reason, we are already designing an online study to distribute over social networks and collect data from a wider and more heterogeneous population. However, the following paragraphs provide the initial results.
3.1.1. Cognitive style and decoration style/ fashion style

The first such study involved the comparison of cognitive style and decoration style/fashion style preferred. Students were asked to complete the MBTI questionnaire and then they could choose between object photos. These objects were placed in three main categories (classic, modern, pop) and students were asked to choose one from each row (i.e. one chair, one dress, etc.). The following figure shows the items presented to the students.
EXPERIMEDIA
All the variables presented above were nominal/categorical. These nominal data, with multiple categorical values, formed several contingency tables. The contingency tables were statistically analysed with chi square tests. From the available chi square tests, the Pearsons chi square test was used in order to test the goodness of fit of the observed frequency distribution against the theoretical distribution and also to test the independence of the variables. Although the Pearsons test is the best known and most widely used for the analysis of contingency tables, there are several problems associated with its use when the expected frequencies are low. For those cases, when the expected frequencies were low, a more appropriate test was used, the Likelihood Ratio. Only the tests that provided significant or borderline significant results are presented here. From the MBTI dimension the one that gave significant results was the Judger-Perceiver dimension. According to the theory Perceiver are more open to new experiences and in our case, it was found that indeed they preferred pop style more than Judgers. Therefore, when the Judger-Perceiver dimensions was compared to roses (N=58), x (2, 5.883) = Pearson .05.
EXPERIMEDIA
In addition, multiple comparisons between the objects used in the study, revealed high correlations between them, implying that they are well balanced. That means that people who chose pop style bags for example, tend to choose pop style chairs, art, etc. In particular for chairs and boots x (4, 9.887) = Pearson 0.43. Chairs and coats x (4, 8.912) = Pearson 0.63 (borderline result). Chairs and dresses x (4, 8.324) = Pearson .08 (borderline). Bags and roses x (4, 15.224) = Pearson .004. Bags and coats x(4, 9.715) = Pearson .046. Colours and rooms x (4, 10.838) = Pearson .028. Roses and boots x (4, 8.039) = Likelihood Ratio .09 (borderline). Roses and coats x (4, 9.074) = Likelihood Ratio .05.
3.1.2. Cognitive style and avatars

51 students were asked to complete the MBTI questionnaire and choose between the different avatars designed for the game. Then we calculated the success rate of each avatar, how likely it is to predict a certain cognitive dimension. The following table shows the initial results.
N=51 Scientist TV persona Judge Diplomat Engineer Artist Success Rate 40 % 100% 50% 20% 70% 28,5%
EXPERIMEDIA
Old-wise Alien Rapper 60% 50% 33%
To improve the success rate of the avatars, we will add a small text that accompanies them, emphasizing their main characteristic. For example, next to the diplomat a bubble will say I want to avoid conflicts and find the middle solution or nest to scientist I want to work alone in my lab, etc.
3.1.3. Cognitive style and game preference

In the game, the player will be given a choice for the game she wishes to play. For example, in order to collect one item for her museum, the player can choose between a knowledge game and a card game. For this reason, we performed a study with 66 participants and it was found that different dimensions of cognitive style highly correlate with different games, like Extraversion _Introversion and fantasy games, Sensing_Intuition and fantasy games, Sensing _Intuition and chess, Sensing_Intuition and multiplayer games, Thinker_Feeler and card games and Thinker_Feeler and board games. However, although the fount correlations are high, it seems that the results can only go one way. That is if we know the cognitive dimension we could predict the game preferences but not the other way around. As an example, we use the comparison Sensing_Intuition and fantasy games to demonstrate the issue. The x (2, 7.466) = Pearson .024 < .05. Although the value is highly significant the bar chart below shows why the results can be only used one way. That means, if we know the person has a cognitive preference for intuition, then we could present her with fantasy game. From all the statistical analyses we performed the only one that can have direct use in the game is Judger_Perceiver and simulation games. Perceiver do not seem to prefer simulation games and the x (2, 4.515) = Likelihood Ratio = .086) (see bar chart below). However, this is a borderline significant correlation which implies that the finding can be only used as a additional factor for the calculation of the Judger-Perceiver dimension together with other factors, since alone it will not be a good predictor.
EXPERIMEDIA
In addition, we performed x tests for the different types of games we could have in our game. The categories presented to the participants were: action, adventure, horror, strategy, role, simulation, fantasy, cards, gabling, chess, board, knowledge, logic, multi-player. It was found that certain game preferences highly correlate. This finding can have a direct use in the game, since we could present the user with games that we expect she might like. The Pearson correlations found were (N=66):
Action and adventure, x (4, 36.840) = .00 Action and horror, x (4, 9.821) = .044 Action and simulation x (4, 10.145) = .038 Adventure and horror x (4, 9.823) = .044 Adventure and strategy x (4, 18.657) = .001 Adventure and fantasy x (4, 9.689) = .046 Adventure and chess x (4, 12.845) = .012 Adventure and multiplayer x (4, 10.673) = .030 Role and knowledge x (6, 17.830) = .007 Fantasy and knowledge x (6, 22.519) = .001 Fantasy and logic x (4, 14.248) = .007 Cards and gambling x (6, 16.505) = .011 Chess and board x (4, 9.198) = .056 Board and knowledge x (6, 19.369) = .004 Board and logic x (4, 14.290) = .006 Knowledge and logic x (6, 30.109) = .00
EXPERIMEDIA
Finally, in an ongoing study on Facebook, we are currently collecting data to study possible correlations between participants gender, cognitive style and museum themes, like ancient history, natural sciences, etc. Results are expected in spring 2013.
3.2.
Preliminary simulation results of a personalised museum visit
A crowd simulator has been built, in order to simulate the crowd movement of visitors inside the museum. In the following we present: i) the scope of the simulator and its role in the experiment, ii) the simulators mathematical modelling, iii) its Java-based implementation and iv) some preliminary results of its use.
3.2.1. Role and scope of the simulator

The crowd simulator was built on the basis of the overall target of the BLUE project, i.e. to improve the Quality of Experience (QoE) of museum visitors, by suggesting them interesting to see exhibits and by optimizing their routing inside the museum. Therefore, in the scope of the project, the simulator will be used as the basis to examine the expected outcomes of the recommendation and routing algorithms that are gradually being developed, and to identify their limitations and fine-tune them prior to their use with real users inside the museum. Based on the above, the requirements set for the simulator are the following. The simulator needs to: Simulate crowd movement inside museum o Different visitor characteristics (walking time, interests...) o Different crowd arrival rates (load) Simulate museum settings o Different museum size and density (rooms, exhibits per room, distances ...) Measure QoE metrics o o o o Walking time Interest for exhibits viewed Congestion found ...
The simulator also needs to be parameterisable (using elements such as visitor entrance rate, visitor speed, crowd tolerance etc.) so that we will be able to examine a broad range of different scenarios. Furthermore, it needs to have a Graphical User Interface (GUI),which will facilitate its use, and to be able to write directly to Excel Workbooks for the easier manipulation of its results. The simulator was built based on the above requirements. In the following we first present its internal modelling and then its implementation details. Finally, we illustrate and discuss certain examples and preliminary results of its use.
EXPERIMEDIA
3.2.2. Internal simulator modelling

The simulator models two basic elements: i) Museum and ii) Visitor, which then interact with one another during the simulation runs. Museum The museum is modelled as a set of exhibits . Each exhibit represents one exhibit inside the museum. For example, for the FHW museum, an exhibit can be an item of the Mathematics exhibition, a show in Tholos etc. Each exhibit is modelled to have a: Maximum crowd capacity , which corresponds to the maximum number of people that can simultaneously visit the exhibit . For example, in the case of a painting, M would mean the maximum number of people that can gather around the exhibit and be able to see it, while in the case of an exhibit that is a video-projection room, M would mean the maximum number of seats available.
Each exhibit also belongs to one museum room. Therefore exhibits can either be co-located in the same room or they can be located in different rooms. Each exhibit is connected, i.e. accessible via walking, to all the exhibits of the same room. Also, in this specific modelling, each room has one exhibit connected to the museum entrance, as this is the case for FHW (all exhibition, i.e. all rooms, are accessible directly from the entrance without the need to pass from other rooms first) . A room may be connected, always through exactly one exhibit, to another room. To model the above, we define: A| | | |exhibit positioning matrix positioning matrix P can be defined as follows: {
| || |
. Each element
of the
(1)
An| | | |exhibit distance matrix positioning matrix D can be defined as follows: {
| || |
. Each element
of the
(2)
where is the distance of exhibits within the same room, and is the distance between rooms, which is taken as the distance between the two exhibits connecting the rooms. The positioning matrix is used to find the connections between items of the same room and between rooms and the distance matrix is used to calculate the path distances. These two matrices can be given, if we want to experiment on a specific museum setting, or they can be generated using the following four parameters: 1) min and max exhibits distances for the exhibits
EXPERIMEDIA
of the same room, 2) min and max room distances 3) number of total rooms in museum and 4) number of exhibits per room. The simulation described here use the above 4 parameters to generate museum settings. Visitor Visitors are modelled to arrive to the museum with an inter-arrival rate , randomly distributed between two time parameters: . Each visitor j has the following characteristics: Interest per Exhibit , , where is a Real number and the
bounds 0 and 1 mean respectively no interest or perfect interest for exhibit i. Crowd Tolerance . The number of other people that the visitor can handle around the exhibit, without a decrease of his satisfaction from the visit. It is modelled per exhibit, as a percentage of the maximum crowd capacity of the exhibit, i.e.: , where is the maximum capacity of exhibit . Maximum available Time , where , and 1 time unit simulates 1 minute. The time that the user can spend inside the museum. It starts counting the moment the visitor enters. Once it is over, the visitor exits the museum. Walking speed , simulated in meters/second. Will be used to calculate the time needed to go to an exhibit, in the QoE function. Time spent per Exhibit . We assume that the time each user will spend seeing each exhibit is directly proportional to their interest for that exhibit: (( ) )
where and correspond respectively to the overall maximum and minimum times that the user can spend on one exhibit. QoE. Each user has a Quality of Experience metric. This measures how satisfied the user is from his visit inside the museum. We can intuitively assume that for each item the user visits, his QoE is proportional to the users interest on the item, and inversely proportional to the time it took him to reach the exhibit. Therefore, for a user that is currently at exhibit and he is going to exhibit , we model as follows: ( where and ) (3)
are calibration weights and is the distance between the current exhibit and exhibit . For the simulations,
and ( of Experience
) are also normalised in the [0,1] range. Finally, the total Quality at the end of the visitor's visit is:
EXPERIMEDIA
Dissemination level: PU (4)
3.2.3. Simulator implementation

To implement the above, we have developed an event-based crowd simulator program, based on Java. The results of the experiments are logged into MS Excel files, using the Apache POI3 (Java API for Microsoft documents) and a custom-made VB script to automatically update the excel files that correspond to multiple runs over the same experimental parameterization. Finally, the A* algorithm is used for the computation of the shortest path between exhibits (visitors move from one exhibit to another always following the shortest path). The simulator comprises a Graphical User Interface, which allows entering values for all the modelling parameters mentioned above (Fig. 7). Fig. 8 illustrates an instance of the simulators running output (showing the position matrix, distance matrix, and part of the output process).
Figure 7. The GUI of the crowd simulator.
3http://poi.apache.org/
EXPERIMEDIA
Figure 8. Running the simulator: an output instance. Here are depicted from top to bottom: the position matrix (dark grey) and the distance matrix (light grey) of the specific museum instance, and part of the running progress (dashed).
3.2.4. Experimentation status and future work

As illustrated in previous section, the developed simulator can be parameterised to simulate many different museum settings and visitor behaviours. In this section we present some preliminary experiments of its use, their results and the interpretation of these results for the purposes of the BLUE experiment. 3.2.4.1. Simulated systems We simulated two systems implementing two kinds of visitor behaviour inside the museum: 1) Random walk. Visitors enter the museum and they randomly select exhibits to see. If the exhibit they selected has reached its maximum crowd capacity , they move to another exhibit. For going from one exhibit to the other, visitors always take the shortest path. The resulting simulated system, also referred to as the no recommendations system, serves as the benchmark. 2) Smart. Visitors in this system are recommended exhibits, from a recommendation algorithm. In case the algorithm recommends them an exhibit for which their estimated
EXPERIMEDIA interest is above 7 (in a scale of 10):
Dissemination level: PU , then they follow the recommendation.
Otherwise they behave like in the random system, i.e. they randomly select an exhibit. It should be noted that the algorithms makes its recommendations based on the estimated interest of the user, which is modelled as the real interest of the user plus noise. The noise, which basically models the approximation error that the algorithm will have in reality, is set by the "noise "parameter of the simulator (also depicted in the GUI of figure 8), as a range between two numbers, each in the scale of [0,10], since interest is also measured in this scale. Then before each recommendation, the algorithm selects a random number in the range set for the noise and adds/subtracts it from the real user interest. This estimated value is the one used for the recommendation decisions of the algorithm. Example: Assume we set noise to be in the [1,2] range and a user with real interest 7/10. This means that the best possible approximation of this user's interest is 7 1 and the worst possible is 7 2. Therefore the algorithm estimates the user's interest as a random number in the [5,6] or in the [8,9] ranges. We developed two different recommendation algorithms: a) Smart 1.The algorithm optimises for the QoE function, as given in eq. (4). This is the classical content-based recommendation scheme where the recommender takes into account user interests and walking time to try to maximise the user satisfaction. b) Smart 2. Improves the smart 1, by also optimizing for congestion. The latter is a classical optimisation problem in routing. However in our case, it is handled in addition to content-based (or rather interest-based) recommendation. Given the above, we simulate 3 different evaluation scenarios (Table 1). Each scenario builds on the results of the previous scenario, improving one aspect of the latter.
Table 1. The evaluation scenarios. Each scenario builds on the results of the previous, examining one additional experimental aspect. Scenario Algorithm Recommendations? QoE attributes taken into account by algorithm Interest Walking distance No Yes Yes Crowd Tolerance No No Yes
1 (benchmark) 2 3
Random walk Smart 1 Smart 2
No Yes Yes
No Yes Yes
3.2.4.2. Parameterisation We evaluate the scenarios using the parameters illustrated in Table 2.
Table 2. The parameters used in the scenario evaluation
Parameter General Parameters Runs per result
Value 1
EXPERIMEDIA
Noise input to recommender Total simulation time Visitor parameters Maximum available Time Walking speed Time spent per Exhibit Visitor inter-arrival rate Museum parameters Exhibit distance (in the same room) Room distance Exhibit crowd limit Number of rooms Exhibits per room 1/10 18,000 units [1800, 5400] [0.83,1.83] [60,500] [0, 180] [2,5] [100,500] [4,10] 5 6
3.2.4.3.
Preliminary results
Scenario 1 Simple recommendation assessment. In this scenario we measure the QoE between the Random and Smart1 systems, for different levels of recommendation noise (i.e. the error between an ideal recommendation and the one actually provided by the system). This first scenario answers the basic question of whether a QoE-based recommendation increases the QoE of the visitors and, if so, the amount of our gain. Given that the recommendation algorithm that we will put in place inside the FHW museum will inevitably have an approximation error (regarding its estimation of user interest per exhibit), this scenario also answers the QoE improvement that we should expect in relation to the algorithms approximation error. Finally, one additional critical environmental condition (visitor arrival rate) is examined in relation to the algorithms response. As we may see (Figure 9), the recommendation noise directly affects the level of visitor QoE, since it affects the accuracy of the recommendation algorithm. In other words, the more inaccurately the algorithms estimations are, the lower is the final average visitor QoE, which is normal since QoE is computed from the user interests fulfilment. The very high levels of noise, for example above7, of the Figure correspond to the worst case scenario accuracy error that may be encountered for visitors that have not used the Facebook game and therefore their profiles are unknown to the algorithm when they first enter the museum. As they gradually move and respond to recommendations, we can expect that the recommendation algorithms accuracy error will drop. At both cases however (for high and low noise) we may observe that the visitors QoE is higher than the benchmark system.
EXPERIMEDIA
QoE
Mean Qoe for all movements 0.85 0.8 0.75 0.7 0.65 0.6 0 2 4 Noise Figure 9. Quality of Experience achieved with and without recommendations (recommendation algorithm used: smart1) 6 8 10 No recommendation Smart1
Figure 10 depicts the number of missed exhibits for different ranges of noise in the recommendation algorithms estimations. Missed exhibits denote those exhibits that the visitor is very interested in (internal interest >7/10) but that the visitor does not see because they are not recommended by the algorithm, because of the high estimation noise. As we may also intuitively expect, the higher the noise, the higher are the lost exhibits. However, we may also observe that at all cases, the recommendation-based system does not result in a significantly lower number of missed exhibits, thus the loss for the visitor, even at high levels of approximation error, is manageable.
Missed Exhibits
5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 0 2 4 Noise Figure 10. Average number of missed exhibits with and without recommendations (recommendation algorithm used: smart1) 6 8 10 Average Lost for all vistors
No recommendation Smart1
The Smart1 recommendation algorithm does not optimise from congestion, but only in regards to time and interest of the visitor. Indeed as we may see in Figure 11, both the benchmark and the smart systems present approximately the same congestion levels. Since in rush hours
EXPERIMEDIA
congestion may be an important factor, in the next scenario we examine a variation of the Smart 1 algorithm, which optimises also for congestion (Smart2 algorithm).
Congestion
Average exbhit missed per visitor 1000 100 10 1 0.1 0 0.01 50 100 Max visitor inter-arrival time No Recommendation Smart1 150 200
Figure 11. Congestion levels with and without recommendations (recommendation algorithm used: smart1)
Scenario 2 Optimization for Visitor Congestion As we may observe in the previous scenario, Smart1 algorithm improves the QoE of visitors and reduces the number of items they lose, i.e. they do not see although they have a high interest in. However, as we may also see (Figure 11) Smart 1 results in similar congestion levels than with the benchmark. In this second scenario we use Smart2 algorithm, which slightly alters Smart1, in order to include congestion optimization. The extra functionality of Smart 2, is that it first filters exhibits that have (at the time of the recommendation) a congestion less than X% (congestion threshold) of their maximum capacity , with . Then, it makes its recommendation to the visitor from the remaining exhibits, in the same way as Smart 1(according to the ranked estimated QoE function).
EXPERIMEDIA
QoE - Missed Exhibits Tradeoff

6 0.85 0.8 QoE 0.75 0.7 0.65 0.6 0 0.2 0.4 0.6 QoE 0.8 1 Congestion Threshold Missed Exhibits
Average Misssed Exhibits
5 4 3 2 1 0
Figure 12. The tradeoff between missed exhibits and congestion threshold (recommendation algorithm used: smart2)
As we may observe in Figure 12 if we lower the congestion threshold too much then the recommendation algorithm approaches in QoE the benchmark, because the algorithm has very few items to suggest to the visitor that fit the visitors interests. On the other hand, with a high congestion threshold, the number of missed exhibits is high. There is therefore a trade -off between visitor QoE and the items that they miss because of traffic considered too high by the algorithm, and this scenario helps define the optimal congestion threshold that the algorithm should place, to achieve a good trade-off.
EXPERIMEDIA
4. Ethics and privacy

Respecting ethics and rights to privacy is a major concern of BLUE. All ethical guidelines defined in the first deliverable have been observed. In regard to privacy, we have defined a consent form [refer to appendix] that informs all experiment participants of the experiments scope as well as giving them a clear list of the data, and nature of the data, that will be stored. The form also clearly mentions their rights as defined by the EU directive 95/46/EC as well as giving them instructions as to how to exercise their rights. Their consent will be asked on multiple occasions, on every possible entry point to the experiment. Entry barriers have been implemented in the My Museum Guide to prevent the participation of users not having explicitly given their consent as far as technically possible. Care has been taken to secure the database and security and access policies will be in effect before the first data is stored. The consent form (see Consent forms) has been dressed in close collaboration with Tudors legal department. The nearly finished form has been validated with Aleksandra Kuczerawy who is looking into the exact procedures to get the consent form submitted and validated by the EAB. The form has been submitted and accepted, there has been no negative return, by Luxembourgs national agency for data protection and privacy, the CNPD. Finally, this consent form has been translated in Greek by UoP (Appendix: "Participant Consent for Research Project_grk.docx"). UoP as a Data Controller has notified the Hellenic Data Protection Authority (HDPA) (www.dpa.gr) and is waiting for response.
EXPERIMEDIA
5. Conclusion
In this deliverable we describe the current status of the Experimedia BLUE experiment, regarding the technical and the experimental aspects of the project. First we summarise the work performed so far, provide details regarding the additional constraints induced by the FHW venue, describe the experimental setup (software and hardware) that has been decided in view of these constraints and the final experimental scenario. Next, we present our first experimental results obtained off-site. These are of two types: My Museum Strory Results and Perspectives. The first type of results concerns the assessment and parameterization of the My Museum Story (MMS) application. For the MMS application the initial results gathered can fall under three main categories: the ones that can be immediately applicable in the Facebook game, the ones gathered that can be used in a future version of the game and the ones that remain to be gathered in our future work. In this last category, we expect to collect results regarding music preferences and cognitive style dimensions. Although in our ongoing unpublished research, there are clear indications that music preferences indeed correlated with different cognitive style dimensions, the issue remains to be studied within the framework of the Facebook game as well. Another game element that needs to be studied further is the players choice of a museum template and how this choice possibly correlates with cognitive and/or visiting style. Finally, we have not used Gestalt images in the game yet. Where the user focuses on the different Gestalt images could provide extra information about her preferences in sensing or intuition. From the results we have collected but remain to be used in a future version of the game are correlations found between cognitive style and decoration and fashion preferences. In a future version of the game, the player will be also available to customise her avatars by choosing their outfits and also decorate her museum space according to her taste. Both choices on outfits and museum decoration can provide further information about cognitive style dimensions, especially about JudgersPerceivers. Finally, for the collected and applicable results, cognitive style and game choice will be partially used for the determination of the Judger-Perceiver dimension. In addition, the initial results for the avatar preferences showed that since their success rate spans from 28.5% to 100% m we will add a small text, putting emphasis on the stereotype we wish to use. Museum Simulation Results and Perspectives. The second type of results, at this stage of the project, assesses the expected impact of personalised recommendations and user routing inside the museum, using a custom-made crowd simulator. The simulator models and examines the movement of museum visitors inside the museum, their response to recommendations received in their mobile device and the Quality of Experience (QoE) that they obtain from these recommendations. At this stage the simulator is ready and it can be parameterised to examine many different visitor types, in regards to visitor arrival rate, movement speed, time spent per exhibit and maximum time inside the museum. Different museum settings can also be constructed, in regards to number of museum rooms, number of exhibits, density of exhibits (exhibits/room), room distances and maximum capacity of exhibits in visitors. Finally, different
EXPERIMEDIA
algorithms can be integrated into the simulator, as well as different interest approximation errors for each algorithm. Using the simulator we are able to generate and examine many scenarios (different visitor behaviours for the same museum setting, different museum settings etc.), to ensure the robustness of the recommendation and routing algorithms that will be used with the real museum visitors of FHW later on during the BLUE experiment. Given the frequent changes in the museum settings witnessed so far (change or closure of exhibitions as explained in Section 2.1 - Venue constraints) the simulator is even more helpful in fine-tuning the algorithms for the final experiment with real users. So far we have examined 2 scenarios, where 2 different algorithms (one optimizing the visitor QoE in terms of interest and walking distance and the other including also the optimization for crowd congestion around exhibits) where examined. Results are promising and show that indeed an optimization based on recommendations and personalised routing inside the museum can indeed significantly increase the QoE of visitors. The scenarios that we intend to examine in the immediate future are the following: Visiting style modelling. For the time being, visitor interest per exhibit is modelled randomly. However, literature shows that there are 4 predominant types of visiting styles in a museum. For the immediate future steps we plan to incorporate these styles as well in the simulations, to examine the responsiveness of the routing and recommendation algorithms under more realistic conditions. Approximation of the QoE function. In the previous scenarios, the weights of the QoE function are fixed and the same for every user. Moreover, they are considered known to the algorithm. In this scenario, users will be modelled to have different QoE functions, which are gradually approximated by the recommendation algorithm (smart 3), observing the visitor response to the recommendations. This will allow us to examine the speed of approximation convergence of the recommendation algorithm, in order to judge whether this is realistic and sufficient for the scope of our experiments. Calibration with FHW data. As soon as the first experiments start to take place in the premises of the FHW museum, we plan to calibrate the simulator and re-run the experiments with the real visitor data (user arrival rate, congestion levels etc.) in order to fine-tune the algorithms prior to the full-scale experiment.
EXPERIMEDIA
Appendix A. Dissemination, meetings and efforts

Dissemination So far one conference article has been accepted. Others are currently being written, accounting from preliminary results and will be submitted during April or later. Angeliki Antoniou, George Lepouras,Ioanna Lykourentzou, Yannick Naudet, Connecting physical space, human personalities and social networks: The Experimedia Blue Project Hybrid City conference, Hybrid City II conference, Athens, May 22-25, 2013 (http://uranus.media.uoa.gr/hc2/) Meetings During the period from October 2012 to March 2013, the BLUE team has attended the following meetings: Act 4 Teleconferences Act 2 Teleconferences + Integration Teleconferences (ECC) General Assembly, Madrid Face to Face meeting, FHW-BLUE-REENACT 2 Visits to FHW Skype meetings: TUDOR-UOP (4); TUDOR-UOP-FHW (1)
Efforts expended The corresponding efforts expended are detailed in the following table:
Organisation Effort expended
Henri Tudor Public Research Centre (TUDOR) 6,2 PM University of Peloponnese (UOP) 5,9 PM
EXPERIMEDIA
Appendix B. Consent forms

Participant Consent for Research Project BLUE is one of several projects in the scope of the European EXPERIMEDIA project (Grant Agreement No 287966), aiming at conducting experiments in live social and networked media experiences. If you choose to participate in the experiment, BLUE will allow you to establish your cognitive profile by playing a Facebook game and get custom-made, tailored content and recommendations during your museum visit. During the experiment you will have the possibility to share your results and your visit on Facebook if you choose to. All BLUE project partners, i.e. the University of Peloponnese (Greece) and the Public Research Centre Henri Tudor (Luxembourg), take due diligence in respecting the ethical and legal code of conduct of research as well as your personal rights regarding privacy and data protection. Your data will not be communicated to third parties or used to ends not covered by this document. It will only be used for scientific purposes. Please consider the following points before signing this form. Your signature confirms that you agree to participate in the study. Your contribution to the research will be participating, at your discretion, in a Facebook game which will present you with objectives that aim at determining your cognitive profile. During your visit through the FHW, you may use the My Museum Guide mobile application which will keep track of your position in the FHW as well as progress in the exhibitions. It will register your pictures, should you take any, comments, and likings you express using the application. The treatment of your data as well as their storage will take place in Greece. Recommendations are generated in Luxembourg however which requires that the data is temporarily transmitted, but not stored, to Luxembourg. All project partners will analyse your data to draw conclusions on the experiment. Your data is stored at most until December 31st, 2013. Within the context of the performance of the project, the partners agree to abide where necessary by the provisions of their national Law as well as EU Directive 95/46/EC on the protection of persons with regard to the processing of personal data. As such you have the right to, at any moment, request modification, deletion, or a complete excerpt of your data.
Both, the University of Peloponnese and the Public Research Centre Henri Tudor are considered Data Controllers in the context of protecting your data and you can address either of them by any means at your disposal such that your request can be taken care of. CRP Henri Tudor Data Officer 29 Avenue John F. Kennedy L-1855 Luxembourg, Luxembourg Phone: +352 42 59 91 1 University of Peloponnese Dr George Lepouras Department of Computer Science and Technology 22100 Tripolis, Greece Phone: +30 2710 372201
EXPERIMEDIA Mail: data.officer@tudor.lu
Dissemination level: PU Mail: remove-experimedia@uop.gr
Data that will be collected during the experiment is: your age, your Facebook validated ID (a number sequence, not your name), your gender, your interests regarding the content of the museum inferred by playing the Facebook game andby your Facebook profile, choices made, such as avatar or pet, playing the My Museum Story games, which My Museum Story games were played and with what frequency, your cognitive profile, your position for as long as the My Museum Guide is running (even in the background), recommendations proposed to you and them being followed or not, messages you leave for friends as well as their content, the number of pictures you have taken, comments you have made, and likings you have expressed, as well as the frequency of other social interactions on a per exhibit basis. Their content is not kept.
The content of messages will not be processed and is merely kept to be able to transmit the message. It is not our intention to infer data regarding ethnics, religion, health, sex, political or philosophical beliefs, but, depending on your profile, they may be inferred. We will not use this data and they will be disregarded. The project partners will try to design their systems such that these inferences are avoided if technically possible. Confirmation and consent I confirm that I have freely agreed to participate in the project. I have been briefed on what this involves and I agree to the use of my data. I understand that the processing of my data is protected by a code of professional ethics and by specific laws.
Participant Name: Date:
_______________________________ _______________________________
Participant signature: _______________________________
EXPERIMEDIA
BLUE EXPERIMEDIA ( . 287966), . , BLUE Facebook , . , , Facebook. BLUE, () Henri Tudor (), , . . . . . , , Facebook . ( ), " - My Museum Guide" , . , , . , . , . . 31 2013. , , 95/46/ . , , , , , .
, Henri Tudor . CRP Henri Tudor Data Officer 29 Avenue John F. Kennedy .
EXPERIMEDIA L-1855 Luxembourg, Luxembourg Phone: +352 42 59 91 1 Mail: data.officer@tudor.lu
Dissemination level: PU 22100 , : +30 2710 372201 Mail: remove-experimedia@uop.gr
: , Facebook ( ), , Facebook Facebook, , My Museum Story , , My Museum Guide ( ), , ( ) , , ("likes") ( ) . . , , , , , Facebook . . . . , . : : _______________________________ _______________________________
_______________________________

D4.8.2 BLUE Experiment Progress Report v1.0

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

D4.8.2 BLUE Experiment Progress Report v1.0

Hochgeladen von

Copyright:

Verfügbare Formate

D4.8.

2 BLUE Experiment Progress Report

Final experiment scenario Implementation of personalisation, recommendation, and profiling

Figure 1. Personalisation and Recommendation strategies in BLUE.

Null passwords should not be used or allowed.

Web: https://experimedia.uop.gr Application container: https://experimedia.uop.gr:8443

Final technical architecture

Figure 2: Revised architecture

Figure 3: Package structure overview

Figure 4: Package diagram showing high-level dependencies

Figure 5. Main recommendation algorithm

Experiment software (Implementation)

Figure 6: State machine modelling MMG screen flow and transitions

Screen 5: Login using the SCC's SocialAuth implementation

Screen 8: Main application window, placeholder for displaying content

3. First experimental results

3.1.1. Cognitive style and decoration style/ fashion style

3.1.2. Cognitive style and avatars

3.1.3. Cognitive style and game preference

Preliminary simulation results of a personalised museum visit

3.2.1. Role and scope of the simulator

3.2.2. Internal simulator modelling

An| | | |exhibit distance matrix positioning matrix D can be defined as follows: {

Dissemination level: PU (4)

3.2.3. Simulator implementation

Figure 7. The GUI of the crowd simulator.

3.2.4. Experimentation status and future work

EXPERIMEDIA interest is above 7 (in a scale of 10):

Dissemination level: PU , then they follow the recommendation.

Random walk Smart 1 Smart 2

Parameter General Parameters Runs per result

QoE - Missed Exhibits Tradeoff

Average Misssed Exhibits

4. Ethics and privacy

Appendix A. Dissemination, meetings and efforts

Appendix B. Consent forms

EXPERIMEDIA Mail: data.officer@tudor.lu

Dissemination level: PU Mail: remove-experimedia@uop.gr

Participant Name: Date:

Participant signature: _______________________________

EXPERIMEDIA L-1855 Luxembourg, Luxembourg Phone: +352 42 59 91 1 Mail: data.officer@tudor.lu

Dissemination level: PU 22100 , : +30 2710 372201 Mail: remove-experimedia@uop.gr

Das könnte Ihnen auch gefallen