• Über uns
  • Mitarbeiter
  • Aktuelles
  • ... mehr über uns
  • Kontaktinformationen
• Lehre
  • Offene Projekte
  • Lehrveranstaltungen
  • LV-Unterlagen
  • Abschlussarbeiten
• Forschung
  • Forschungsbereiche
  • Projekte
  • Publikationen
  • Partner

• TU Graz
• TUGRAZonline
• Impressum

 

 

 

 

Development of a Front-End-Module for New Web Based Training Software

 

 

 

Master’s Thesis

at

Graz University of Technology

 

submitted by

 

Werner Antensteiner

Matr.Nr. 9032540

 

 

Institute for Information Processing and

Computer Supported New Media (IICM)

Graz, University of Technology

A-8010 Graz, Austria

 

 

Graz, May, 1999

 

 

 

 

 

 

 

o. Univ. Prof. Dr. phil. Dr. h. c. Hermann Maurer

Ing. Mag. rer. nat. Mag. phil. Dr. phil. Andreas Holzinger

 

 

 

 

 

 

 

 

Entwicklung eines Frontend-Moduls für eine neue internetbasierende Lernsoftware

 

 

 

Diplomarbeit

an der

Technischen Universität Graz

 

vorgelegt von

 

Werner Antensteiner

Matr.Nr. 9032540

 

 

Institut für Informationsverabeitung und

Computergestützte neue Medien (IICM)

Technische Universität Graz

A-8010 Graz

 

 

Graz, Mai, 1999

 

 

 

 

 

Diese Diplomarbeit ist in englischer Sprache verfasst.

 

o. Univ. Prof. Dr. phil. Dr. h. c. Hermann Maurer

Ing. Mag. rer. nat. Mag. phil. Dr. phil. Andreas Holzinger

 

 

 

Abstract

 

A new generation of training-software aims to give the user the motivation for learning various fields even if it is missing now and then. This should be achieved by embedding the learning software in a game-show where the user should give his best together with a virtual friend – the VR-Friend –, trying to answer as many questions as possible.

The starting point for the concept of a training-software implemented in this thesis is the surprising success of Tamagotchis. These virtual beings made countless human beings – above all of course teenagers – deal with them. A similar effect should be achieved by the virtual friend that is acting as a partner in the game-show. As a result the user should play more intensively and more often in the game-show, in this way being busy with the learning fields.

After an introduction in chapter 1 the basic requirements of the learning software are laid down in chapter 2. Chapter 3 deals with the detailed specification of those modules that are part of this thesis. The implementation of them is covered by chapter 4. Finally chapter 5 allows a view on the running system.

The objective of this thesis is a prototype runable over the Internet and implemented in the Java programming language. It consists of a frontend that is running in a WWW-browser and a backend that uses a client/server SQL database system for storing all data needed for the system.

This prototype should subsequently enable scientific investigations whether this concept of a training-software actually results in positive influences on the learning behaviour and the motivation of the "players".

 

 

Zusammenfassung

 

Eine neue Art von Lernsoftware soll dem Benutzer die mitunter fehlende Motivation zum Erlernen verschiedener Stoffgebiete vermitteln. Dies soll erreicht werden, indem die Lernsoftware in eine Spielshow eingebettet wird, in welcher der Benutzer, zusammen mit einem virtuellen Freund – dem VR-Friend –, als Teilnehmer möglichst gut abschneiden, also so viele Fragen wie möglich beantworten soll.

Ausgangspunkt für das in dieser Diplomarbeit umgesetzte Konzept einer Lernsoftware ist der überraschende Erfolg der Tamagotchis, jener virtuellen Lebewesen, die unzählige Menschen – vornehmlich natürlich Jugendliche – dazu brachten, sich mit ihnen zu beschäftigen. Ein ähnlicher Effekt soll durch den als Spielpartner agierenden virtuellen Freund erzielt werden, sodass ein Benutzer sich intensiver und vermehrt mit der Spielshow und somit mit den zu lernenden Stoffgebieten beschäftigt.

Nach einer Einführung in Kapitel 1 werden in Kapitel 2 die grundlegenden Anforderungen an die Lernsoftware festgehalten. Kapitel 3 beschäftigt sich mit der detaillierten Spezifikation jener Komponenten, die in dieser Diplomarbeit behandelt werden und deren Implementation in Kapitel 4 beschrieben wird. Abschließend erlaubt Kapitel 5 einen Blick auf das fertige System.

Die Zielsetzung dieser Diplomarbeit ist ein über das Internet lauffähiger und in der Programmiersprache Java implementierter Prototyp, bestehend aus einem Frontend, das in einem WWW-Browser ausgeführt wird und einem Backend, welches sich eines Client/Server-SQL-Datenbanksystems bedient, in der die für das System notwendigen Daten gespeichert werden.

Dieser Prototyp soll in weiterer Folge Untersuchungen ermöglichen, ob dieses Konzept einer Lernsoftware tatsächlich positive Einflüsse auf das Lernverhalten und die Motivation der "SpielerInnen" bewirkt.

 

 

 

 

I hereby certify that all work included in this thesis is my own, that work performed by others is appropriately cited and disallowed aids have not been used.

 

 

Ich versichere, diese Arbeit selbständig verfasst, andere als die angegebenen Quellen und Hilfsmittel nicht benutzt und mich auch sonst keiner unerlaubten Hilfsmittel bedient zu haben.

 

 

Werner Antensteiner

Graz, May 1999

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Many names used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those names appear in this thesis, and I was aware of a trademark claim, the names have been printed in caps or initial caps, rather than listing the names and entities that own such trademarks or inserting a trademark symbol with each mention of the trademarked name. However, the absence of this notation does not imply the non-existence of a trademark.

 

 

Acknowledgements

 

I would like to thank all members of the VR-Friends team for the friendly cooperation during the design and implementation of the VR-Friends project. In particular I want to thank my adviser Andreas Holzinger for his continuous and untiring support and for correcting all draft versions of this thesis. Additional thanks go to Georg Stegmüller and Sabine Galler, forming the project’s staff, for their assistance mainly in technical aspects.

I want to express my special thanks to Horst and Elisabeth Demmelmayer for their enduring support over the last time of my study, first for relieving me of office work and second for making available technical equipment for the making of this thesis and the belonging software.

Next, I would like to friendly thank my brother Andreas for final correction of the draft version of this thesis.

Finally my most heartfelt thanks are due to my parents who have always been standing helpfully by me and who made my study possible by their effective help.

 

 

 

Table of Contents

 

I Preface *

II List of Abbreviations *

III List of Figures *

IV List of Tables *

1 Introduction and Overall View *

1.1 Motivation and Starting Point *

1.2 Further Investigations *

1.3 Chapter Outline *

2 VR-Friends Requirements *

2.1 User Requirements *

2.1.1 VR-Friends *

2.1.2 Course of the Game-show *

2.1.3 Modules *

2.2 System Requirements *

2.2.1 Overview *

2.2.2 Backend Servers *

2.2.3 Inbetween Modules *

2.2.4 Frontend Modules *

2.3 Example of a Game-Show (Learning Session) *

3 Specification of the VR-Friends Frontend and Backend *

3.1 Database systems *

3.1.1 Terminology *

3.1.2 The Relational Database Model *

3.1.3 The Object-Oriented Database Model *

3.1.4 The Entity-Relationship Model *

3.1.5 SQL (Structured Query Language) *

3.1.6 Choosing the VR-Friends‘ Database model *

3.1.7 Conventions for the database model *

3.1.8 Database Engine *

3.2 System Module Overview *

3.3 Reaction Database *

3.3.1 Mental States *

3.3.2 Appearance of a VR-Friend *

3.3.3 Database Model *

3.3.3.1 Definition of Tables *

3.3.3.2 Table TblVRFriends *

3.3.3.3 Table TblMentalStates *

3.3.3.4 Table TblLanguageData *

3.3.3.5 Table TblLanguages *

3.4 Status Database *

3.4.1.1 Definition of Tables *

3.4.1.2 Table TblVRFriendStatus *

3.4.1.3 Table TblUser *

3.5 VR-Friend Server *

3.5.1 Maintaining the Mental State *

3.5.2 Maintaining the Degree of Concentration *

3.5.3 Choosing Animations *

3.5.4 Choosing Reactions *

3.6 VR-Friend Display *

3.6.1 Displaying the VR-Friend *

3.6.2 Playing Animations and Reactions *

3.7 Auxiliary Modules *

3.8 Communication Channels *

3.8.1 Channel CommQuizmaster *

3.8.2 Channel CommVRFDisplay *

3.8.3 Channel CommGameShow *

3.8.4 Channel CommReactionDB *

3.8.5 Channel CommStatusDB *

3.8.6 Channel CommUserSelect *

3.8.7 Channel CommVRFSelect *

3.9 Course of Execution of the VR-Friends System *

3.10 Time Sequences of the VR-Friends System *

3.10.1 User Login *

3.10.2 Selecting a VR-Friend *

3.10.3 Starting the VR-Friend Display *

3.10.4 Starting a Game-Show *

3.10.5 Displaying a New Question *

3.10.6 Answering a Question Right *

3.10.7 Answering a Question Wrong *

3.10.8 Quitting a Game-Show *

3.10.9 Reselecting the VR-Friend *

4 Implementation of the VR-Friends Frontend and Backend *

4.1 General Implementation Objectives *

4.2 Communication Channels *

4.2.1 Quizmaster – VR-Friend Server (CommQuizmaster) *

4.2.1.1 Event ShowStart *

4.2.1.2 Event ShowQuit *

4.2.1.3 Event QuestNew *

4.2.1.4 Event QuestRight *

4.2.1.5 Event QuestWrong *

4.2.1.6 Event QuestUser *

4.2.2 VR-Friend Display – VR-Friend Server (CommVRFDisplay) *

4.2.2.1 Notification VRFShow *

4.2.2.2 Notification VRFHide *

4.2.2.3 Notification ReqAnim *

4.2.2.4 Communication from the VR-Friend Server to the VR-Friend Display *

4.2.3 GameShow Display – VR-Friend Display (CommGameShow) *

4.2.4 User Select – VR-Friend Server (CommUserSelect) *

4.2.4.1 Request UserLogin *

4.2.4.2 Request UserLoginKill *

4.2.4.3 Request UserLoginNew *

4.2.4.4 Request UserChangePwd *

4.2.4.5 Request LangSelect *

VR-Friend Select – VR-Friend Server (CommVRFSelect) *

4.3 Implementation Details of Modules *

4.3.1 Backend *

4.3.1.1 VR-Friend Server’s Course of Execution *

4.3.1.2 The VR-Friend Server Configuration File *

4.3.1.3 VR-Friend Server Class Descriptions *

4.3.2 Frontend *

4.3.2.1 Frontend’s Course of Execution *

4.3.2.2 Classes for the VR-Friend Display *

4.3.2.3 Classes for module User Select *

4.3.2.4 Classes for module VR-Friend Select *

4.3.3 Java Class Tree *

4.4 Implementation of the Databases *

4.4.1 Table Definitions of the Reaction Database *

4.4.2 Table Definitions of the Status Database *

4.4.3 Security Issues and User Management *

4.4.4 Connecting via the JDBC/ODBC Bridge *

5 A View on the Running System *

Appendix A: Bibliography *

Appendix B: Index *

 

 

1. Preface

Since Tamagotchis have become a worldwide success in 1997, experts have been wondering about the reasons and if the same effect could be produced at other possibly more useful things.

As a result an idea of Professor Hermann Maurer heading the Institute for Information Processing and Computer Supported New Media (IICM) of Graz University of Technology was to develop a completely new learning software that could considerably improve the quality of learning at the computer. This approach was called "VR-Friends" and should represent some sort of virtual learning-companion based on the Internet.

For the design and implementation of this new software a few theses were announced, each of them realizing some parts of the whole project.

By chance I read about these theses in a local newsgroup set up for study-related topics. Finally I started this thesis which is dealing with two parts of the whole project. First this part of the front end module or user interface that is responsible for the display of the VR-Friend itself. Second a database, located on some backend server, that is necessary for storing all available VR-Friends including their appearances and possible reactions.

Another thesis of the VR-Friends’ project is working on the other part of the frontend responsible for displaying questions and providing input possibilities for the user’s answers and the corresponding backends that store questions and all related topics as learning modules.

 

 

 

 

2. List of Abbreviations

 

In the following table all abbreviations used in this thesis are listed.

 

Abbreviation	Explanation
ANSI	American National Standards Institute
cf.	confer
cit.	citation
DBMS	Database Management System
DBS	Database System
DCL	Data Control Language, a component of SQL
DDL	Data Definition Language, a component of SQL
DML	Data Manipulation Language, a component of SQL
e.g.	exempli gratia
EER-Model	Extended Entity-Relationship-Model
GIF	Graphics Interchange File Format
HTML	Hypertext Markup Language
HTTP	Hypertext Transfer Protocol
ID	Identifying number of a record in a database that usually is unique
IICM	Institute for Information Processing and Computer Supported New Media
ISO	International Organization for Standardization
JDBC	Java Database Connectivity
JPEG	Joint Photographic Experts Group
K-module	Knowledge-module
LRN	Learner
LSY	Learning System
ODBC	Open Database Connectivity
OODBMS	Object-Oriented Database Management System
p.	page
pp.	pages
RFC	Request For Comments
SEQUEL	Structured English Query Language
SQL	Structured Query Language
URL	Uniform Resource Locator; unique address in the WWW
VRF	VR-Friend
VR-Friend	Virtual-Reality-Friend
WWW	World Wide Web

 

3. List of Figures

 

Figure 1: Traditional learning system *

Figure 2: VR-Friends learning system *

Figure 3: Overall VR-Friends system design *

Figure 4: Example for an EER-model *

Figure 5: Structure and commands overview of SQL *

Figure 6: Modules and Communication Channels *

Figure 7: Database model of the Reaction Database *

Figure 8: Query process of language dependent data. *

Figure 9: Database model of the Status Database *

Figure 10: Change of the mental state in response to the team’s answer *

Figure 11: An example for a mental state scale *

Figure 12: Calculation of new mental state *

Figure 13: Change of concentration degree in response to the team’s answer *

Figure 14: Changes of concentration degree and its step value caused by the mental state *

Figure 15: Execution flowchart of the VR-Friends system *

Figure 16: Time sequence for initialization of user login screen *

Figure 17: Time sequence for login of existing user name *

Figure 18: Time sequence for login with new user name *

Figure 19: Time sequence for changing the password *

Figure 20: Time sequence for selecting a VR-Friend *

Figure 21: Time sequence for starting the VR-Friend Display *

Figure 22: Time sequence for starting a game-show *

Figure 23: Time sequence for displaying a new question *

Figure 24: Time sequence for answering a question right *

Figure 25: Time sequence for answering a question wrong *

Figure 26: Time sequence for quitting a game-show *

Figure 27: Time sequence for reselecting the VR-Friend *

Figure 28: Execution flowchart of the VR-Friend Server *

Figure 29: Java class tree *

Figure 30: Login screen *

Figure 31: VR-Friend selection *

Figure 32: Selection of game-show *

Figure 33: Negative reaction of the VR-Friend *

 

 

 

4. List of Tables

 

Table 1: Definitions of database related terms *

Table 2: Definition of the table TblVRFriends of the Reaction Database *

Table 3: Definition of the table TblMentalStates of the Reaction Database *

Table 4: Definition of the table TblLanguageData of the Reaction Database *

Table 5: Definition of the table TblLanguages of the Reaction Database *

Table 6: Definition of table TblVRFriendStatus of the Status Database *

Table 7: Definition of table TblUser of the Status Database *

Table 8: Change of the mental state in response to the team’s answer *

Table 9: Effect of the ranking list on the step value for mental state changes *

Table 10: Change of the concentration degree in response to the team’s answer *

Table 11: Effect of the mental state on the concentration degree’s step value *

Table 12: Communication channels overview *

Table 13: Events of the CommQuizmaster communication channel *

Table 14: Notifications of the CommVRFDisplay communication channel *

Table 15: Events of the CommGameShow communication channel *

Table 16: Requests of the CommReactionDB communication channel *

Table 17: Communication events of the CommStatusDB communication channel *

Table 18: Requests of the CommUserSelect communication channel *

Table 19: Request of the CommVRFSelect communication channel *

Table 20: Header of byte stream block sent from the VR-Friend Server to the Display *

Table 21: Data header of byte stream block sent from the VR-Friend Server to the Display *

Table 22: Meanings of the possible contents of the byte stream block’s data header *

Table 23: Data part of byte stream block sent from the VR-Friend Server to the Display *

Table 24: Java methods implementing the communication between GameShow and VR-Friend Display *

Table 25: Constructor for module VR-Friend Display *

Table 26: Java classes for the VR-Friend Server *

Table 27: Keywords of the VR-Friend Server configuration file *

Table 28: Java classes for the VR-Friend Display *

Table 29: Java classes for module User Select *

Table 30: Java classes for module VR-Friend Select *

 

 

 

 

1. Introduction and Overall View

This chapter outlines the concept of VR-Friends, its origin and development starting from the Tamagotchis, thus explaining how the basic idea can be used for development of a new learning system. The chapter continues with a short description of possible investigations that could be started after the development of the VR-Friends learning system will be finished. A summary of the following chapters concludes this one.

 

1. Motivation and Starting Point

In learning systems currently available learning happens in traditional, strongly linear and goal-directed forms of learning. This means when learning some field the learner goes from topic to topic repeatedly reading or looking at the contents. Because the learning system (the same is true for a book for example) only provides the information to learn and usually the learner himself has to decide if and when he knows enough about a specific topic, the learner himself is required to maintain the motivation for continuing the learning. In the sense of successful learning – that means keeping in mind most information learned in a period of time that is as short as possible – this is a problem especially when the field to learn is difficult to understand or the learner has no personal interest in those topics but has to learn them for some reason [Holzinger & Maurer (1999)].

Figure 1 shows the way a traditional learning system works. The learner (LRN) interacts with the learning system (LSY).



Figure 1: Traditional learning system

As a consequence a new concept has been worked out that possibly eliminates or at least reduces the shown motivational problem. It is called incidental learning on the computer and is to be implemented in a new learning software [see e.g. Anderson (1985), p. 171f].

The basic idea of this concept came from the unforeseen success of the so called Tamagotchis [Eisenmenger (1997), p. 14f] which has not yet been scientifically analyzed. These "virtual living beings" started to spread all over the world in 1997. The extents of this spreading exceeded all expectations and psychologists all over the world could not give any explanations. Subsequently many children, parents and – of course – teachers were driven to despair by this mania.

The main idea of Tamagotchis is to represent some kind of virtual companion that periodically has to be fed, cleaned and even entertained to ensure that it will stay alive [Paul (1997), p. 22-37]. Estimations assume that 100 million hours have already been spent for "looking after" these virtual beings [Maurer (1997), p. 1126].

This idea of a virtual companion (or friend) led to the concept of the completely new learning software called VR-Friends (standing for Virtual-Reality-Friends). Figure 2 illustrates this concept. Now, there is a third party, the VR-Friend, involved in the learning process that influences the behaviour of the learner.



Figure 2: VR-Friends learning system

The difference between Tamagotchis and VR-Friends is mainly based on the technical implementation which is hardware-done at Tamagotchis but completely software-implemented at VR-Friends.

We can say that VR-Friends are virtual characters that should help the user to learn. In a first approach the VR-Friend is designed to be a very inquisitive person that is very eager to learn. Such a VR-Friend likes being taught and is constantly asking questions. We could consider this as an intelligent, human behaviour but VR-Friends have one big fault: They really do not know everything about any specific field which just happens to be the topic the learner has to learn.

Thus the learner has to explain the topic to his respective VR-Friend by himself while passing over from the learner to the teacher in this way. Because of explaining something, increased cognitive processes occur that directly result in thinking processes which again are causing learning processes.

This kind of learning is called incidental learning because the learning can be seen as a sort of side effect of the explanation process. In other words the learner has to prepare, analyze and organize his learning fields in order to be able to answer the persistent questions of the VR-Friend. By this means a VR-Friend should motivate a learner to learn without letting the learner to become aware of that.

However, just for that reason this approach to VR-Friends is problematical because the basic idea and appearance of the VR-Friends system is to be a learning system. This would prevent the learner from forgetting that the intended purpose of the system is to learn. As a consequence the user’s motivation is reduced and his learning success as well.

For these reasons the VR-Friends system is given a totally different overall picture: It is designed like a game-show. This means that the user and his individual VR-Friend are forming a team in such a game-show. The questions asked there come from a learning module (or K-module which is short for knowledge-module) that the user has to learn. The VR-Friend behaves like a colleague that helps the user in answering the questions. This has the advantage that the user does not get disappointed if he is not able to answer several questions because the VR-Friend helps out. Of course the VR-Friend is not always "able" to answer such a question correctly. This depends on some important characteristics of VR-Friends, which are their mental state and their degree of concentration.

A mental state of a VR-Friend means that a VR-Friend can get angry (as Tamagotchis also do) if the learner’s advances are unsatisfactory. In the same way VR-Friends can become happy if the user can answer most questions right which means that the learning-results of the learner make progress. For the sake of completeness it should be mentioned that it is refrained from giving the VR-Friends a state of health or in other words letting them to "feel" sick or even to die as Tamagotchis can do. This is not advisable for the intended purpose of VR-Friends. Reactions of the VR-Friends partially depend on their mental states but also on answers of the user (for example smiling after a right answer or sighing after a wrong one).

The degree of concentration is derived from the correctness of the user’s answers as well. The more answers are right the higher is the concentration degree which also enables the VR-Friend to give a correct answer with a higher probability in case of the user not knowing the particular answer.

In this connection another major difference between VR-Friends and known learning software is that VR-Friends are unpredictable in their behaviours. For this reason surprising reactions of a VR-Friend should help to give the learner enough motivation to increase his efforts on the learning field.

With the outlined concept of a learning software presented as a game-show, it is tried to create some kind of game that makes a user to learn without always being reminded that he is learning. Ideally a user should be able to play with the game without knowing at all that he is using a learning software. This again would be incidental (or implicit) learning.

Besides another advantage of this kind of learning software is the so-called Tamagotchi-effect. This means that some sort of relation arises between the (human) user and the virtual being which makes the user deal with the VR-Friend. The aim of this relation is to increase the overall amount of time, the frequency and the intensity of "playing" with the computer.

As a summary VR-Friends should make use of three fundamental psychological elements: incidental learning, increased motivation, and the Tamagotchi-effect [Holzinger & Maurer (1999)].

The specification and implementational details of VR-Friends are described in detail in chapters 2 and the following.

 

2. Further Investigations

Up to now there exist no known learning systems based on incidental learning. Therefore no scientific research could have been made to prove that incidental learning is more effective than the traditional kind of learning. By means of the implementation of a learning system including VR-Friends it will be made possible to compare both kinds of learning and to find out which advantages or adverse effects are the results of the use of VR-Friends.

Thus after having finished the development project of VR-Friends, one will be able to take among other things the following questions under a scientifical examination:

• How do VR-Friends (that is incidental learning) influence learning?

It is not only interesting to learn about the general influence of VR-Friends but also how VR-Friends effect learners with different fields of learning. For example there could be a difference in the influence between learning facts (vocabulary of a foreign language, geography or something like that) and learning conceptual know-how (knowledge of solving problems such as in mathematics or physics) [Holzinger (1997a), Holzinger (1997b) ].

• Which effects have VR-Friends on the motivation of learners [Holzinger (1998)]?

Do VR-Friends really get learners to work with a specific field of learning for a longer period of time, in an enduring manner and above all with (more) pleasure [Holzinger (1997c)]?

• Social impacts of VR-Friends to learners:

Does the use of VR-Friends in a learning system has social influences of some kind on the users?

• Human-computer-interaction:

How does the existence of a virtual person influence the learner’s interaction with the learning system? Is the result a more straight-forward "walk" through the field of learning? Does the presence of a VR-Friend reduce a possibly existing anxiety of learning at or rather with the computer?

• Tamagotchi-effect:

How is the learning-behaviour stimulated in dependence of the mental state of VR-Friends? In other words, does the changing behaviour of VR-Friends imply specific reactions of the learners (known as the Tamagotchi-effect).

 

1. Chapter Outline

Chapter 2 describes the requirements for all parts of the VR-Friends learning system. This includes system requirements as well as user requirements and an example of a possible learning session where a learner, the learning system and the VR-Friend are interacting.

Chapter 3 consists of the detailed specification of all parts of the VR-Friends learning system that are responsible for displaying a VR-Friend and all its reactions. These parts include the Reaction Database, the modules VR-Friend Display and VR-Friend Status as well as the communication protocols between the VR-Friend Display and all other modules.

In chapter 4 all details of the implementation of those parts of the VR-Friends learning system specified in chapter 3 are described.

In the appendix the bibliography and an index can be found.

 

 

 

1. VR-Friends Requirements

This chapter shortly describes all parts of the VR-Friends learning system from the user’s point of view first. A technical description of the learning system follows. An example of a learning session concludes this chapter.

 

1. User Requirements
1. VR-Friends

There are no restrictions, "what" can be used as a VR-Friend. It seems to be very obvious to take photos (or series of photos) and record voices of real persons. On the contrary comic figures can be used as well, such as Disney Figures or the Flintstones for example. Of course in this case the copyrights for the usage of those figures must be taken into consideration.

 

2. Course of the Game-show

When starting a VR-Friends learning session, the learner (in the following also called user) will have to enter a base-URL (which could be http://www.VRFriend.com, for example) into his or her WWW-Browser. After eventually displaying a welcome screen, the system should then provide a HTML-page where the user first is able to login to the system and can choose one of all available VR-Friends as his or her companion for the game-show then.

This can be accomplished by providing a page with thumbnail views of all VR-Friends. Then the user can select the desired VR-Friend by clicking on its picture. Up to now we can see this as if the user is "meeting" a VR-Friend at the URL-location.

Thereafter the VR-Friend appears on the screen. After selecting the desired game-show (that is the learning module) "together", the quizmaster (or moderator) of the game-show welcomes both and the game begins.

The quizmaster then repeatedly asks questions to the user. If his answer is right, the team will get some points. If the user fails, the question will be passed on to the VR-Friend, that may or may not be able to give the correct answer. However, if the VR-Friend’s answer is correct, the team will also get points but less than for the case the user can answer the question. All obtained points are added and displayed during the game.

After finishing the game the received points are stored in a high-score table (also called ranking list) together with the scores of all other teams that played the same learning module. This should give the user additional motivation to try harder.

During the game-show the user can hold up the game to display additional information about a question if the user wishes to get more information.

 

3. Modules

Learning modules are the basis for the questions that are asked in a game-show. Essentially these are databases that store several kinds of questions as well as their respective correct answers. Kinds of questions can be multiple choice (choose one or more answers from a pool of possible answers), picture (click onto the right area on a picture) and completing a text (enter the missing words).

An important reason for the user’s motivation is the reaction of the VR-Friend. To supply those informations a database is needed that comprises spoken and written words (each in different languages) and gestures of all available VR-Friends.

For the mentioned databases wizards are needed. These are special programs used for maintaining the databases.

 

An additional feature is the barometer of opinion or user feeling. This means that from time to time questions are asked about the contentment of the user. Such a question could for example be "I am happy" or "This VR-Friend is really funny". The user then should rate such a question with a mark ranging from one (for best) to some number [Bortz & Döring (1996), p. 163ff.].

These questions have nothing to do with the game-show itself. But as a user feedback they are very valuable to determine if the learning system as a whole is working well or if there are some weak points either in the system or the learning modules.

Usually users unwillingly fill out whole questionnaires. Normally those answers are not very accurate, too, because such forms are completed some time after what is to be rated has occurred. From this it follows that the user has to remember what he thought earlier. Therefore it seems to be a good idea to present just one of the questions now and then. This is supposed to be accepted considerably better and moreover will reflect the actual mood of the user during the game [Maurer (1997), p. 1130].

2. System Requirements
1. Overview

The VR-Friends system consists of a collection of several client and server programs. According to the philosophy of client/server the individual programs can be distributed among several computers, thus forming an open system. The communication between the modules is realized by commands that are transferred between client and server by means of the TCP/IP protocol [Postel (1981a, 1981b)].

See Figure 3 for a diagram that illustrates the main parts of the system and their communication relations. As we can easily see, the VR-Friends system can be considered as a system set up implementing a 3-tier architecture. This means the system consists of three parts or levels, each of them having its own tasks.

The Frontend is responsible for the user interaction, that is giving the user the possibility to do some input and sending this input to the next level. In addition the Frontend retrieves data from the next level and displays it.

The next level contains the Inbetween Modules. They implement the application logic, or in other words they process the inputs coming from the Frontend, decide what has to happen next, communicate with the next level, the Backend Servers, and send outputs or rather commands to the Frontend.

The Backend Servers typically are database servers which are only responsible for maintaining the databases. For this purpose they receive requests to store or retrieve some data and return the desired results to the calling Inbetween Modules.

Although the different levels and, to a certain extent, also the individual modules can run on several computers, the VR-Friends system can be divided into two parts, namely the Client and the Server Side as illustrated in Figure 3, where the client side is the user’s computer that is connected via internet or other kind of network to one or more server computers at the server side.

 



Figure 3: Overall VR-Friends system design

 

The various parts of the system that are displayed in Figure 3 are briefly explained in the following sections.

 

2. Backend Servers

For all Backend Servers, which actually are database servers, wizards are needed for editing the databases. Both wizards and clients communicate with the servers statelessly using a command language. Basically a client contacts the server (thus establishing a connection) and sends one or more commands. The server sends the appropriate answers or in case of a failure some error messages. After that the connection is terminated. These are the Backend Servers needed:

The K-module Server runs several learning databases, the so-called K-modules (or learning modules). They contain the questions and some parameters that influence the run of the corresponding game-shows.

Some additional databases are needed for the execution of a game-show. That are databases that store interrupted learning sessions, the ranking list (high-score table) and the question history (for a certain learning session).

The Reaction Server runs the database that stores all informations needed for displaying the individual VR-Friends, their reactions and their comments. The specification of this database is explained in detail in chapter 3.1.

The database VR-Friend Status stores some status-informations of VR-Friends related to given users (see chapter 3.4 for a detailed explanation).

 

3. Inbetween Modules

As an Inbetween Module the Quizmaster Server controls the proceedings of the game-show. For that reason it communicates with the frontend browser (the Game Show) as well as the backend Learning Module Server.

The second Inbetween Module is the VR-Friend Server. This module communicates with the VR-Friend Display at the client side and with the Reaction and Status Database Servers. It decides what is to be displayed at a certain client’s VR-Friend Display. To get informations about the proceedings of a certain game-show it also communicates with the Quizmaster Server and on occasion sends back the decision whether the VR-Friend answers a question correctly.

 

4. Frontend Modules

On the one side the Frontend Modules consist of the VR-Friend Display, which is entirely controlled by the VR-Friend Server. It is responsible for displaying everything what is sent by the VR-Friend Server. For that reason it communicates with the Inbetween Module VR-Friend Server. In addition it receives informations about the proceedings of the game-show from the browser (see next paragraph). The module VR-Friend Display is described in detail in chapter 3.5.

The second Frontend Module is the WWW-Browser that displays the questions and other informations provided by the module Game Show. Thus it represents the interface of the game-show that allows the user making inputs to the system.

 

3. Example of a Game-Show (Learning Session)

To illustrate the working method of the VR-Friends system this example is given. Let us first define some questions that are stored in the selected learning module called "Learning" that should teach something about learning itself.

 

No.	Question	Correct answer
1	Does a Behaviourist teacher behave like a coach.	No
2	Learning is a basic cognitive process.	Yes
3	Pawlow was one of the first supporters of Constructivism.	No
M

 

In the following scenario some abbreviations are used that are listed in the following table:

 

Abbreviation	Meaning
VRF	VR-Friend (In this example the name George was chosen for the VR-Friend)
QUM	Quizmaster (that is the learning system)
LRN	Learner (In this example Sabine is the name of the learner)

 

The following scenario could be a possible course of the conversation between a learner, the learning system and the VR-Friend (comments are printed in italics). In this example the questions are used in the above order. Actually it would be wiser to take them in a random order.

 

The game is started:
VRF	"Hello, Sabine, I am really glad we are taking part in the game together."
QUM	"This is the game-show Learning, welcome Sabine and George!"
QUM	Asking question 1.
Now there are two possibilities how the learner could react:
Possibility 1			Possibility 2
LRN	"No" (correct answer)		LRN		"Yes" (wrong answer)
QUM	"That’s right! You receive two points."	VRF		Showing a disappointed reaction, for example frowning.
VRF	Showing a happy reaction, for example a smile.
			At this point the VR-Friend may jump in and give the correct answer:
			VRF		"No" (right answer)
			QUM		"That’s right! You receive one point."
Then the learning session is continued with the next question:
QUM	Asking question 2.
Again there are two possibilities how the learner could answer:
Possibility 1			Possibility 2
LRN	"Yes" (correct answer)		LRN			"No" (wrong answer)
QUM	"Exactly! You receive two points."
VRF	Showing a big smile, calling "Go on Sabine!"		Now the VR-Friend again could jump in but perhaps is not able to give the correct answer now:
			VRF			For example shrugging its shoulders.
			QUM			"This answer is wrong! You receive no points." Explaining the right answer.
And so on ...
In case of permanently wrong answers the VR-Friend could react as follows:
VRF	(angry) "Hey Sabine, do you think you know anything about learning? I cannot answer all questions on my own!" (funeral music can be heard)
On the contrary in case of some right answers the VR-Friend could react like this:
VRF	(happy) "Hey Sabine, you’re sensational! I wouldn’t be able to answer so many questions!" (cheerful music is played)

 

 

 

2. Specification of the VR-Friends Frontend and Backend

As explained in the preface this thesis deals with those parts of the VR-Friends system, which are responsible for displaying the VR-Friends themselves, their individual reactions and behaviour. Four modules are needed for these purposes, that include the Reaction Database, the Status Database, and the modules VR-Friend Server and VR-Friend Display. The modules VR-Friend Server and VR-Friend Display need to have the possibility to communicate with each other and with the modules Quizmaster Server and GameShow. In addition the VR-Friend Server also communicates with the Reaction and Status Databases.

Besides these modules the auxiliary modules User Select and VR-Friend Select for user login and selection of a VR-Friend are explained as well.

Before the detailed specifications of all these modules and their communications are described, general informations about different kinds of database systems and related topics are given. Implementation details of the modules, the databases, and the communications are explained in chapter 4.

 

1. Database systems

For the purpose of storing data in a database system two different kinds of database models can be used, the relational and the object-oriented database model. Before that explanations of database related terms are given.

 

1. Terminology

The following table (Table 1) lists all database related terms that are used throughout the following chapters together with an explanation [definitions cf. Geppert (1997), p. 4f, or as stated otherwise]:

 

Term	Explanation
Database	A database is an integrated collection of related data [Sayles (1989), p. 8].
Database Model	The database model defines all kinds of needed data, their properties, relationships and behaviour.
Database Management System	The database management system (DBMS) is a program, that manages databases. Thus it realizes the secure, reliable and long-term management of large amounts of data that can be used by many users and enables an easy access to the data.
Database system	The term database system (DBS) stands for the combination of the DBMS and the database (including the database model).
Application programs	Application programs interact with the DBMS which itself manages and coordinates every access to the databases.

Table 1 – continued on next page –

Table 1 – continued –

Term	Explanation
Object-Oriented DBMS	The object-oriented database management-system (OODBMS) is a DBMS with an object-oriented database model [Atkinson and others (1989), cit. from Geppert (1997), p. 5].
Data Access Language	A data access language (SQL for example) is the language system users employ to communicate with the DBMS to accomplish their work [Sayles (1989), p. 8].

Table 1: Definitions of database related terms

 

2. The Relational Database Model

The relational database model dates back to the year 1970 when it was introduced by E. F. Codd [Codd (1970)]. It does not use pointers to represent relations between data as it had been used in earlier database models. In the relational model data are related to each other exclusively by its contents [Schwinn (1992), pp. 11f; cf. Cattall (1991), pp. 52f].

Sets of informations (called entities) that hold data of the same kind are arranged in the form of tables or, in other words, relations. The description of each entity corresponds to one line or row of the respective table, each of those rows being unique within the same relation. As a result the order of the table’s rows does not make any difference.

The columns of the table are called attributes. Each attribute has its own name and is always referenced by that name instead of the position. In addition all attributes have defined ranges of valid values which can be compared with datatypes in a programming language. It is a very important principle of the relational model that the values of the attributes of a row contain simple values only, or in other words all attribute values are atomic. That means that values cannot be decomposed into smaller pieces [Date and Darwen (1992), p. 44].

A key of a relation is a subset of the attributes of that relation that meets two conditions:

1. The unique identification which guarantees that a table’s row can be uniquely referenced by the value of the key.
2. The absence of redundancies meaning that no attribute of the key can be omitted without violating rule 1.

As mentioned above each row of a relation is unique and therefore there always exists a key. A trivial key that includes all attributes of the relation will always fulfill condition 1. So it remains to find the subset meeting condition 2. A relation can have more than one candidate keys. That means that there may exist two or more different subsets of attributes that fulfill both conditions 1 and 2. In this case a primary key must be chosen [Hughes (1992), p. 13-15].

The relationships between tables, that is between individual rows or entities of two tables, are generated by storing references to the other table as attribute values in the first table. By way of illustration say, there are two tables A and B, A owning primary key KeyA, B having primary key KeyB. In order to let a certain row of table B be related to perhaps some rows of table A, the primary key value (KeyB) of B’s row is stored as attribute values in the concerned rows of table A. From table A’s point of view these stored references of KeyB are called foreign key values because these values actually are key values, thus uniquely identifying B’s row, but not key values of table A. The term foreign key identifies the table’s attribute (or subset of attributes or columns respectively), where the foreign key values are stored.

There are three different types of relationships: strictly unique (1:1), one-sided unique (1:n) and complex (n:m). The values in parenthesis express how many rows of one table are related to how many rows of the other table. The values n and m represent any positive integer value.

A 1:1 relationship requires exactly one row of the first table being related to each row of the second table and vice versa.

Having a 1:n relationship indicates that each row of the first table may have any number of related rows in the second table. Describing this relationship from the second table’s point of view, each row of the second table belongs to exactly one row of the first table. Such a relationship is often called a master-detail relationship where the first table is representing the master table and the second table the detail table. So we can say that in a master-detail relationship each row of the master table can have any number (including zero) of depending rows of the corresponding detail table.

In an n:m relationship there are no restrictions in the number of rows of both tables that are related to each other.

 

A relational database model that is designed according to the above definitions may still contain ambiguities and inconsistencies that should be cleared up before implementation. This process is called normalisation [Hughes (1992), p. 22f].

The theory of normalization is based on the concept of normal forms, starting with the first normal form. A relation is in first normal form if every attribute value is an atomic, not further divisible unit of data. As this condition is already included in the definition of a relation, every relation that is created according to the above definitions is in first normal form.

Every further normal form is based on the previous normal form. That means that a relation that is in second normal form already is in first normal form or, for example, a relation in third normal form is both in first and second normal form and so on. The following normal forms have been defined in this order of increasing precision of their conditions:

• First normal form
• Second normal form
• Third normal form
• Boyce-Codd normal form
• Fourth normal form
• Fifth normal form

A relation is in second normal form if it is in first normal form and every non-key attribute is fully functional dependent on the primary key. This means that there must not exist a (non-key) attribute of which the value is implied by any other attribute beside the primary key attribute(s).

A relation is in third normal form if it is in second normal form and every non-key attribute is not transitively dependent on the primary key. In other words no non-key attribute is allowed that depends on another non-key attribute which itself depends on the primary key.

Moving from one normal form to the next is always possible by splitting up relations. Of course no information is being lost thereby because the original relation can be restored by joining the new relations.

A relational model in third normal form can still have some inconsistencies that can be eliminated by applying the remaining normal forms. However such inconsistencies rarely appear and therefore the conversion to the third normal form is sufficient for most applications. In practice a total normalisation is not always desirable because of the following reason: It is true that updating data is simplified in a higher normal form because inconsistencies are prevented, but retrieving data is complicated because more tables must be joined then. This however negatively affects the performance of a database application which requires the database developer to carefully consider normalization and performance issues [Hughes (1992), p. 30].

For the VR-Friends system the number of needed attributes is relatively small and the relationships in between are not very complex, so meeting the conditions of the third normal form will satisfy the VR-Friends system’s requirements which is why the other normal forms are not treated any further.

 

1. The Object-Oriented Database Model

Many modern programming languages offer efficient algorithmic structures for representing complex data or better data objects. The possibility to define such abstract objects together with suitable operations is very useful for applications using databases [Hughes (1992), p. 75f].

The relational database model tends to decompose objects in artificial structures which actually are easier to implement. Object-oriented databases are based on the application-oriented representation of the data. This approach enables the user to imagine the database as a collection of complex objects having relations among each other and viewing the objects in an abstraction level that is required for the application. In this way the object-oriented model helps in describing an application in a way closer to the real world.

In an object-oriented database every object is an instance of a class. All objects that belong to a class are described by the class definitions. Instead of describing individual objects the object-oriented approach focuses on describing valid conditions and behavioural patterns that are common to the whole class of objects. The state of an object is realized by attributes. However in contrast to the relational model these attributes are not restricted to atomic data types such as Integer, Real or String but can be complex objects itself.

The behaviour of an object is implemented by a set of procedures, usually called methods, which are encapsulated together with their attributes. Any number of instances of such an object coexist, having different contents and identities but equal structures and behaviour. Such objects are called objects of the same class.

 

Relational database models use attributes that can uniquely identify certain rows of a table as keys. For that the contents of the data are used instead of an address where the data can be found. Object-oriented models use a more complex way for identifying an object in a database namely by the object’s identity. This is a property of an object that sets it apart from other objects and can be used for regaining that object. This identity is implemented by three levels of independence [Hughes (1992), p. 159]:

• The location independence requires the preservation of the object’s identity regardless of the location of its storage.
• The value independence demands the preservation of the object’s identity independent of changings of the object’s values.
• Finally the structure independence ensures that the object’s identity is independent of changings of the object’s structure.

 

There are some more characteristics an object-oriented database model has to fulfill. These are original features of the object-oriented programming languages which were adopted for the object-oriented database model [Geppert (1997), p. 10-12]:

Encapsulation means that values or conditions of an object are neither accessible nor visible from "outside" (for other objects). Attributes can only be read or changed by methods which must be provided in the object’s class for that reason.

The object-oriented database model must support the so-called is_a-relationship which implements specialization and generalization. In other words a most general superclass can be defined with its attributes and methods and more specific subclasses can be derived from that superclass, which inherit all the superclass’s attributes and methods but can add any number of more specializing attributes and methods. Because this subclassing and inheriting can be continued to any level a class hierarchy can be implemented.

The object-oriented model must support the overwriting of methods in subclasses. In this case some methods can exist all having the same name within a class thus overloading the original (superclass or inherited) method. In addition it should be possible to write new (derived) classes that again overwrite inherited methods. For this reason it cannot be determined at compile-time which of all overloaded methods is the right one to call. As a consequence the OODBMS has to decide at runtime which method is the correct implementation for a certain situation and calls it then. This is known as late binding.

Finally user-defined classes must be supported and from the user’s point of view there must not exist any differences between predefined and user-defined classes.

 

1. The Entity-Relationship Model

For the design of a database several concepts had been suggested that did not gain great importance, probably because they were too complex or hard to implement. However, all of the more important models can be combined in the term Extended Entity-Relationship Model (EER). It was originally introduced by Chen [Chen (1976)] and has turned out to be sufficient for most application programs in those days. However in the last years database developers had to face increasing complexity of applications which required extended semantic concepts for data modelling. In this way Chen’s approaches have been modified and extended by many others. Because of its clear representation and simple use the EER-model obtained great importance for commercial applications as well as in research over the last years [Hughes (1992), p. 2].

In the EER-model informations are represented by means of entities, attributes and relationships, all of which have been explained in the previous chapters. Schematic representations of entity-relationship models use diagrams that describe the natural structure of the data. In such a diagram a rectangle stands for an entity and a rhombus for a relationship. Related entities are connected with lines that are supplemented by the type of the relationship (for example 1:1 or 1:n). In an complete EER-model each entity and each relationship contain lists of all included attributes.

Figure 4 shows a simple example of an EER-model [Gogolla (1994), p. 7] with the entities Person, Town and Country being related among one another. Each entity is represented by its own rectangle having the entities’ names printed in bold type and the attributes listed below the name. The two relationships in between are displayed by a named rhombus in each case. Since a town (per definition) always has exactly one person as a mayor there is a 1:1 relationship between person and town. On the other hand any number of towns can lie in a country. Therefore there is a 1:n relationship between country and town. The symbol indicating this type of relationship can be seen in the figure.

 



Figure 4: Example for an EER-model

 

For the sake of completeness it should be mentioned that in practice there often appear more complex types of relationships between the entities. The simple 1:1, 1:n and n:m relationships are not sufficient. For example relationships between the same entity could be needed, or relationships that comprise more than two entities.

 

2. SQL (Structured Query Language)

SQL is a data access language that consists of a set of facilities for defining, accessing and managing SQL-data. Originally SQL was intended to be a language for managing relational data, but up to now the standard has developed so far that it is no longer truly relational. Hence the term SQL-data is used.

The starting point of the development of SQL was the already mentioned publication of E. F. Codd in 1970 [Codd (1970)] when he introduced a new approach for database management, the so-called relational model. Starting from that paper the development of relational database technology involved the design and implementation of a variety of relational languages, one of those being worth mentioning: The Structured English Query Language, or SEQUEL in short, was a commercial success of the IBM company and therefore was developed further while changing its name to SQL. Finally IBM brought an SQL-based DBMS to market known as DB2. In the 1980s numerous other vendors also announced SQL products, such as Sybase or Ingres. Today there certainly are over 100 products on the market that support some dialect of SQL, running on machines that range from comparatively small personal computers to large mainframes. So SQL has become the de facto standard in the database world.

In addition SQL also is an official standard, since in 1986 the American National Standards Institute (ANSI) ratified a standard relational language, that essentially consisted of the IBM dialect of SQL. This ANSI standard was later accepted as an international standard by the International Organization for Standardization (ISO). That original standard version is known as SQL/86. Of course some improvements and extensions have been made since then, resulting in ISO and ANSI announcing the SQL/92-standard which is what is normally referred to as "the SQL standard" today [Date & Darwen (1996), pp. 3-6].

 

SQL is composed of a data definition language, a data manipulation language, and a data control language. Figure 5 shows the structure of SQL with its three branches, including the commands that belong to each of them. By means of these three parts of the language all kinds of relational data processing are supported [Sayles (1989), pp. 3f].

 



Figure 5: Structure and commands overview of SQL

 

The data definition part of the language allows creation, deletion and modification of needed data structures which include databases, tables and indexes.

Data manipulation is divided into three parts: retrieving, manipulating and updating of data. Retrieving data means querying the database and obtaining the desired data that is stored in the tables. Manipulating data stands for features that allow to perform statistical functions on data such as calculating averages or sums of columns or other arithmetic functions such as multiplying columns. Updating data means inserting and deleting rows on tables and changing values in columns, or – in other words – database maintenance.

Data control allows definition of a security mechanism for protecting the data in a system from unauthorized access or manipulation. It consists of granting and revoking access privileges on all data objects to all users of the database system.

 

3. Choosing the VR-Friends‘ Database model

For the design of the databases needed for the VR-Friends’ learning system the relational database model is chosen. The main reason for this decision is the structure of the data that is to be stored. As we will see in the following chapters there are only a few kinds of different data so that the powerful possibilities of an object-oriented database model seems to be too much overhead. With the relational model all the data of the VR-Friends can be stored in a few simple master-detail relationships.

Another argument for using a relational model is the variety of relational database management systems currently on market that makes the implementation possible for a great number of computer systems.

 

4. Conventions for the database model

The VR-Friends’ database model will graphically be shown by an adapted form of the entity-relationship model because of the clear and easily understandable representation of such kind of model. In contrast to the example of an EER-model given in Figure 4 on page * relationships will not be explicitly named. Instead of that tables will be directly connected by lines with or without the shown symbol indicating the type of relationship.

Beside that the definitions of the needed tables will be tabularly listed including all column (field) names, each of them together with the column’s data type and restrictions.

 

The following data types are used:

• Integer: signed integer number
• Unsigned integer: positive integer number
• Byte: signed integer number that fits in the range from –127 to +128.
• String: string of specified length
• Boolean: holding true or false as the value
• Unsigned decimal: positive decimal number
• Signed decimal: positive or negative decimal number
• Char: one character represented by an unsigned byte
• TPicture: special type for storing pictures or series of pictures or rather informations about those. Depending on the implementation a field of this type can store the picture data itself or an information where the picture can be found (that is a link to an URL for example).
• TAudio: special type for storing audio files. For this purpose type TAudio is either holding the audio data itself or the information where the audio file can be found.
• TLangID: data type used for identifying the used language.

 

The following restrictions are used:

• Not null: In general a column’s value for a given row can hold some data or no data which is pointed out by having a null-value. However, a column that is owning the "Not null"-restriction must not contain any null-values in any row, that is, every row of such a table needs to hold some value in this column. This is generally true for key values but also needed for some other fields to guarantee data consistency.
• PK (Primary Key): This column’s contents represent key values in another table (those key values of the other table are called foreign key). In this case the actual table is called master table and the other table detail table of a master-detail relationship. However, this does not require that the key column(s) of the master table has (have) to contain data. But if a row’s field contains data, the value has to be used at least once as a (foreign) key in the specified (dependent) detail table.
• FK (Foreign Key): A table including a foreign key serves as a detail table of a master-detail relationship. The contents of this column are key values that reference a primary key of another table which is the master table. Depending on the type of the relationship, the foreign key column contain unique values for a 1:1 relationship. In case of a 1:n relationship (one row of the master table is related to any number of rows of the detail table) the foreign key’s column cannot be considered to hold unique data. However, the column is implicitly required to have the "not null"-restriction. Therefore it has to contain data in each row.

A table that is serving as a master table in a master-detail relationship because of a primary key can be a detail table in another master-detail relationship at the same time by defining any column as a foreign key. In doing so a multilevel master-detail relationship can be build. In the same way a table can of course appear as a master table of more than one master-detail relationships.

For completeness it should be mentioned that there are other types of master-detail relationships, such as reflexive relationships where master and detail tables are the same. The above definitions of primary and foreign keys (above all the not-null requirements) may not apply to those kinds of relationships. However, those relationships are not needed for the definition of the VR-Friends’ data model and have therefore been disregarded in the above definitions.

 

Finally a detailed explanation to all defined columns will be given in the specifications of the database models.

 

1. Database Engine

To save data to or to retrieve data from a database an additional tool is needed, which is called database engine. This program could be seen as the backend or server part of a client/server database model. This program is responsible for running the database (that is the database files) only.

For that reason it is responsible for two tasks:

• First it accepts requests for getting data from any number of concurrently running clients. These clients are the module VR-Friend Server (see chapter 3.5), which of course could be run simultaneously on different sites with different learning sessions. After receiving and evaluating the request, the desired data is sent back to the client. The way requests and data are transferred between clients and server is defined in chapter 3.7.
• Beside the above explained read-only access to the database there also must exist a way, how data is saved to or edited in the database. That means that new VR-Friends including their reactions and all the related data need to be saved and the data of existing VR-Friends may need to be changed or enhanced. Therefore a wizard is used that communicates with the database engine like an ordinary client with the difference that data is transferred to the engine and stored in the database. As usual for a client there is no need that the wizard is run on the same machine as the database engine.

The database engine itself runs on the same machine where the database is stored. This ensures optimal performance accessing the data.

 

1. System Module Overview

The following chapters cover all those modules and communication channels of the VR-Friends learning system that are part of this thesis. Figure 6 gives a graphical overview that – for the sake of completeness – also includes the other parts of the VR-Friends system in grey color.

The databases are as usual symbolized as cylinders, modules as rectangles. For communication channels arrows are used. The direction of an arrow determines which module initiates a connection to another module.

 

The definitions of the Reaction and Status Databases are given in chapters 3.2 and 3.4. The description of the modules VR-Friend Server and VR-Friend Display follows in chapters 3.5 and 3.6. Finally the communication channels printed in bold type are covered by chapter 3.7.

 



Figure 6: Modules and Communication Channels

 

2. Reaction Database

This database holds all informations which are needed to display a VR-Friend with the module VR-Friend Display.

This includes several different data, namely:

• A standard (that is neutral) view of each available VR-Friend. This is first needed to select the desired VR-Friend before starting the learning session which for example could be done by displaying a thumbnail view of all VR-Friends or a possibility to browse through the VR-Friends.

Second this neutral view is the default visual appearance of the VR-Friend during a learning-session. It is only temporarily replaced by an animation or a reaction if that includes a picture or an animation itself.

As discussed later on it will turn out to be useful to combine the standard view with the available mental states.

• Animations that are displayed in irregular intervals that should ensure the feeling of a "living" VR-Friend. These animations are to be grouped according to the actual mental state of a VR-Friend.

Special kinds of animations are needed for welcome or farewell gestures. Another type of animations is used for the case the user needs too long to answer a question.

• Reactions are direct consequences of some actions of the user. Different reactions have to be provided that are used depending on the correctness of a user’s answer. Therefore reactions have to be classified into positive, negative and neutral ones. Which of the available reactions is finally displayed also depends on the mental state of a VR-Friend.
• Animations do always and reactions can include a picture or a series of pictures. Both animations and reactions may also include sound or text that has to be played or rather displayed. The latter two may or may not be provided in different languages.

 

It is distinguished between animations and reactions, because animations, that are displayed with more or less neutral contents now and then, and reactions to actions of users will usually not be identical. For example a VR-Friend clapping his hands will have to be considered as a reaction to a correct answer of the user but would not be suitable as an animation. Imagine the user just thinking over an answer and all of a sudden the VR-Friend is clapping his hands! In the opposite case looking at a side for a few seconds is a typical example for an animation but probably will not be sufficient for a reaction.

Nevertheless, for highest flexibility for filling the Reaction Database an action of the VR-Friend can be defined as animation and reaction at the same time.

As we can see major parts of the data stored in the Reaction Database correspond to a mental state of the VR-Friends. Therefore the concept of mental states has to be defined first.

 

1. Mental States

The mental states possible for an individual VR-Friend could be seen as a scale ranging from some negative to some positive number. The more negative the number the worse is the VR-Friend’s mood (for example caused by continuous bad answers of the user). Conversely, the more positive the mental state number the happier is the VR-Friend. Consequently a mental state of zero represents a neutral state which can be used as the "startup-mood" of a VR-Friend.

In which way the mental state changes during a learning session is the very decision of the VR-Friend Server (see chapter 3.5) which of course bases its decision on the user’s behaviour (mainly answers) in some way.

Mental states apply to reactions as well as animations. A very important thing is that the number of possible reactions or animations of a VR-Friend is as great as possible. This prevents the user from getting bored by always seeing the same reactions or animations of the VR-Friend. This is absolutely necessary because motivation and arousal are very important for learning [Holzinger & Maurer (1999)].

However, it is not useful to provide reactions and animations in an exact "mental" order. In addition, to allow random selection to a certain extent in approximately the same mental state, mental state pools are introduced. That means that there is only a relatively small number of mental states (related to all available reactions and animations) each of them forming a pool of any number of mentally equal but different reactions/animations which can be just textual or auditive or visual or some combination.

To decide the reaction of the VR-Friend the VR-Friend Server has to determine first if the mental state of the VR-Friend is to be changed. Then a random reaction could be chosen out of the currently selected pool of reactions favourably not repeating one of the last reactions displayed.

The same applies for the display of animations with the exception that the animations are randomly displayed in time. So there is no need to change the mental state before displaying it. But of course an animation of the pool currently corresponding to the VR-Friend’s mental state is to be used.

To give a simplified example, let us say, for a specific VR-Friend there is a number of 5 mental state pools ranging from –2 to +2. For each of these states any number of possible reactions and animations could be defined. For instance, state +2 could include a "clapping-hand" picture as a reaction (with or without sound), or some displayed text with gratulations, or just some audio and so on. After a visual reaction or during a reaction that has no picture or animation included the default picture of the VR-Friend is used.

An example for an animation that belongs to the neutral mental state pool zero could be looking left or right.

Chapter 0 includes some snapshots of the running system. There could also be seen some examples of animations and reactions.

 

2. Appearance of a VR-Friend

There are two possibilities how a picture of a VR-Friend can be generated. First the whole appearance of a certain VR-Friend is stored in one picture. This is the easiest way for filling the Reaction Database because pictures of VR-Friends only have to be photographed or painted without any further processing. On the other side a complete picture for each reaction has to be fetched from the database and transferred to the VR-Friend Display. The use of complete pictures is also necessary if a comic-like VR-Friend should be created that could look like something completely different compared to a human-like VR-Friend.

The other possibility is to divide the picture of a VR-Friend into some segments such as head (which could again be subdivided into eyes, nose, mouth, and so on), body, and background. To display a VR-Friend one piece of each kind has to be chosen. So, in order to fill the database, there only has to be created a number of head-displays, eye-displays, body-pictures and so on, all of them representing on the one hand smaller pictures. On the other hand there are only a few pictures to create for each kind but in total there are many combinations of all individual pictures. Another advantage is that for some reaction there is no need to transmit the complete VR-Friend but only the piece or pieces of the whole appearance which is (are) changing.

The drawback of this possibility is the difficulty to compose a homogenous picture put together by different parts. This could be managed with comic-pictures but is rather complicated with photographs of human beings because of the different shadings of the faces of different persons. Another problem is the display of animations if more than one segment of the VR-Friends is concerned. For example letting the VR-Friend shake its head needs to synchronize the display of the eye-, nose-, hair- (and other) segments.

For all these reasons the first approach is chosen, that stores complete pictures of a VR-Friend and its reactions and animations. In addition this requires no restrictions to the contents of the reactions or animations or to the visual appearance of a VR-Friend itself. Furthermore this approach can be optimized in this respect that not always the complete pictures have to be transferred but only those areas that have to be changed.

Another aspect of the appearance of VR-Friends is the number of "standard"-pictures that are available for a special VR-Friend. For example, only a single picture per VR-Friend could be provided which is usually displayed and only sometimes and temporarily replaced by an animation or visual reaction. The disadvantage of this solution is the disregard of the mental states during the time no animations or reactions are displayed. That means first that the user is not able to distinguish between a happy and an angry VR-Friend. Second it possibly could be a strange effect if the VR-Friend is for example shouting for joy as a reaction to some correct answers and then, after completing the display of this reaction, returning to his neutral appearance.

As a consequence it seems to be advisable to provide one picture of a VR-Friend for each mental state pool that exists for that VR-Friend, thus giving several "standard"-pictures, each representing one of all possible mental states.

 

3. Database Model

Four tables are needed to realize any number of mental states including textual and auditive reactions and animations for all available VR-Friends. For the definition of the tables the conventions laid down in chapter 3.1.7 are used.

As mentioned in some previous chapters a database wizard has to be implemented that is responsible for storing and maintaining the data of the Reaction Database. This wizard has to take care of what can be stored in the database. Throughout the following definitions of the tables valid ranges or contents for several table columns are specified. The wizard has to check all entered data for fulfillment of these restrictions and – if necessary – point possible correct values out to the wizard’s user.

 

1. Definition of Tables

Figure 7 illustrates the definition of the tables (and their relations) that are used in the Reaction Database. The model is shown as an EER-model as described in chapter 3.1.4 on page * with the adaptations mentioned in chapter 3.1.7. Each table is shown with its table name (printed in bold type) and all attributes. The relationships in between are pointed out by lines whereby the symbol denotes a 1:n (or master-detail) relationship. That means that the table at the single-line side of the symbol is the master table having any number (including zero) of related rows in the detail table which is on the multi-line side of the symbol. The relationship is established by corresponding row values of the table’s columns the symbol points to.

The tables TblVRFriends and TblMentalStates set up a master-detail relationship. For each available VR-Friend in the VR-Friends learning system there exists one row in TblVRFriends. In TblMentalStates, which is holding all VR-Friends’ reactions and animations that depend on the mental state, any number of rows can be stored, all related to exactly one row of TblVRFriends. Furthermore the table TblMentalStates itself is a master table in the relationship to the table TblLanguageData which is holding language dependent data for a certain reaction or animation.

In this table the language is identified by an ID. All these ID’s that are available, that is one per language, are stored in table TblLanguages. Strictly speaking the tables TblLanguages and TblLanguageData create a master-detail relationship as indicated in Figure 7 because one record of table TblLanguages can have any number of belonging records in table TblLanguageData.

 



Figure 7: Database model of the Reaction Database

 

The master-detail relationship between tables TblMentalStates and TblLanguageData is used to implement multilinguality. To retrieve the appropriate data in a certain language, of course the primary key is needed, which is column LangDataID of table TblMentalStates, and in addition the language ID of the desired language.

In the following four chapters all tables included in the database model are explained in detail.

 

2. Table TblVRFriends

The "main" table that could be seen as the "master table" of all VR-Friends is called TblVRFriends. It holds all general information for all available VR-Friends. These informations include the names of the VR-Friends as well as certain characteristics such as the range of the concentration degree.

Each record of this table references all available reactions and animations for all mental states of an VR-Friend in the dependent table TblMentalStates.

In the following Table 2 an overview of the contents of TblVRFriends is given that is followed by a detailed explanation of all used table columns.

 

Column	Data type	Restrictions	Explanation
VRFriendID	Integer	Not null, PK	Unique number (clearly identifying each available VR-Friend)
Name	String	Not null	Name of VR-Friend (should be unique)
MentalStateFactor	Unsigned Decimal	-	Factor that is multiplied with the default mental state step value
ConcDegMin	Byte	-	Lower bound of the range of the concentration degree; valid values:
ConcDegMax	Byte	-	Upper bound of the range of the concentration degree; valid values:
ConcDegChangeMax	Byte	-	Maximum change value for the concentration degree as a consequence to the current mental state.

Table 2: Definition of the table TblVRFriends of the Reaction Database

 

This is the detailed explanation of all used fields:

• VRFriendID: Each record of this table has its own unique ID, so that it is possible to reference an individual record for some actions such as deleting. In other words each VR-Friend that is available in the VR-Friends learning system is internally referenced by the ID provided by this field.

This column is the primary key related to foreign key column VRFriendID of table TblMentalStates.

• Name: Every VR-Friend of the VR-Friends system has its own name. This name should be unique to help the users distinguish between different VR-Friends. Although this is not necessary from the system’s point of view because a VR-Friend is always referenced by its ID internally. Nevertheless the wizard responsible for storing the name to the database should prohibit the usage of duplicate names.
• MentalStateFactor: For the change of a VR-Friend’s mental state a so-called step value is needed. Although this step value has a default setting, this field can be used to influence that value, therefore letting the available VR-Friends behave in different ways. Details about this step value and the mental states in general are given in chapter 3.5.1.
• ConcDegMin, ConcDegMax: By specifying these values the valid range of the degree of concentration of a VR-Friend can be determined. In this way it is possible to create a VR-Friend that for example is not very good in concentrating by setting ConcDegreeMax to perhaps 50.

As stated in chapter 3.5.2 the degree of concentration is a probability value. Therefore the ConcDegMin and ConcDegMax values must be in the range from zero to 100, letting ConcDegMax being greater than ConcDegMin.

The declaration of each of these values is optional. If one or both values are not specified (that is having a null value stored), the VR-Friend Server automatically replaces every missing value by the default values specified in chapter 3.5.2.

• ConcDegChangeMax: A VR-Friend’s concentration degree changes by adding or subtracting a step value as will be explained in chapter 3.5.2. The influence of the current mental state is reflected by a so-called change value that effects the step value. This change value is always equal or greater than zero. The default upper limit of the change value can by set by providing a value in this field.

In this way the influence of the mental state on the concentration degree can be controlled. A relatively high upper bound for the change value will, for example, let the VR-Friend change his degree of concentration more rapidly, whether it is in good or bad mood.

Note that this value is a maximum value that possibly could directly be added to or subtracted from the current concentration degree. Therefore this value should be carefully chosen to be in proportion to the above explained fields ConcDegMin and ConcDegMax.

The declaration of this value is optional. If it is not specified (that is having a null value stored), the VR-Friend Server automatically replaces the missing value by the default value specified in chapter 3.5.2.

 

1. Table TblMentalStates

The table TblMentalStates holds or references all informations that depend on a VR-Friend’s mental state. Each row of this table represents an appearance, an animation or a reaction of a given mental state for an available VR-Friend. Data that have to be supplied in different languages is stored in the dependent table TblLanguageData.

In the following Table 3 an overview of the contents of TblMentalStates is given that is followed by a detailed explanation of all used fields.

 

Field	Data type	Restrictions	Explanation
ID	Integer	Not null	Unique number (clearly identifying each record of this table)
VRFriendID	Integer	FK	ID of the selected VR-Friend.
MentalState	Signed Integer	Not null (exceptions see below)	Equal mental states of the same VRFriendID are building a mental-state-pool.
IsDefaultView	Boolean		The content of this row is the default view for this mental state.
KindOfReaction	Signed Byte		The content of this row is a positive (1), negative (-1) or neutral (0) reaction.
KindOfAnimation	Char		The content of this row is a neutral animation, an animation for saying hello or goodbye or an animation to display when the user needs too long to answer a question.
Picture	TPicture		Holds a picture or animation.
OffsetX	Unsigned Integer		Horizontal offset for partial picture.

Table 3 – continued on next page –

Table 3 – continued –

Field	Data type	Restrictions	Explanation
OffsetY	Unsigned Integer		Vertical offset for partial picture.
ShowFrames	Unsigned Integer		If a picture format supports the storage of several frames per picture, this field sets the maximal number of frames that are to display.
ShowSeconds	Unsigned Integer		Number of seconds a reaction or animation is to display.
LangDataID	Integer	PK	Corresponds to the ID in TblLanguageData for language dependent parts of the reaction or animation, that includes audio and text.

Table 3: Definition of the table TblMentalStates of the Reaction Database

 

This is the detailed explanation of all used fields:

• ID: Each record of this table has its own unique ID, so that it is possible to reference an individual record for some actions such as deleting.
• VRFriendID: This ID is the foreign key of the master-detail relationship between tables TblVRFriends and TblMentalStates. So this field identifies the VR-Friend which the mental state data belongs to. Because of course more than one record is needed to store a VR-Friend’s data, the VRFriendID is not unique in this table.
• MentalState: As discussed earlier all animations and reactions of a VR-Friend are grouped in mental state pools. This field determines which mental state pool a record belongs to. By definition, the neutral state is state zero; therefore at least for this mental state pool data have to be provided. Other mental state pools can freely be used in positive (happier mood) or negative (worse mood) direction of the mental scale. However it is not possible (and actually not meaningful) to skip some mental state pool values between the minimal and maximal mental state pools. In other words the available mental state pools have to form a continuous integer scale ranging from the minimum to the maximum mental state pool.

In this way the number of available mental state pools is not fixed neither in positive nor in negative direction and can also be different for different VR-Friends.

Usually for each reaction and/or animation a mental state should be specified. The only meaningful exception to this rule are some certain kinds of animations (see field KindOfAnimation) where providing animations for all mental states might not be useful.

However the design of the VR-Friends system allows the storage of reactions and animations (but not default views) without assigning a mental state, thus leaving this decision to the creator of a VR-Friend. A reaction or animation without a mental state is considered as appropriate for all available mental states.

• IsDefaultView: This field determines if the contents of a record identify the default view of a given mental state for a certain VR-Friend.

If this field contains the value "true", the record identifies the VR-Friend’s default view. In this case the fields KindOfReaction and KindOfAnimation are ignored. In fact these fields should not contain any values. Exactly one record for each mental state of a given VR-Friend is required and allowed to hold the default view. Besides a mental state must be provided in field MentalState.

If this field contains "false" as value, the record can either be a reaction or an animation or both. Which of these possibilities is true, depends on the fields KindOfReaction and KindOfAnimation.

• KindOfReaction: If this field contains a value, the record is a reaction (which does not exclude it from being an animation). Because there are three types of reactions, namely positive, neutral and negative reactions, this field is used to identify the type of the reaction.

The reaction can consist of a combination of pictures, a series of pictures, sound and text. The following applies to a visual reaction: Before the reaction the displayed picture of the VR-Friend is the default picture of the current mental state. During the display of the visual reaction the default picture is replaced by the reaction’s picture(s). Afterwards the default picture is shown again, which could have been changed in the meantime due to a change of the VR-Friend’s current mental state pool.

• KindOfAnimation: This field determines if the contents of a record identify an animation (which does not exclude it from being a reaction). There are four different kinds of animations that can be displayed. Each of them is identified by a character as specified in the following:

N: This is a "neutral" animation that can be displayed from time to time to give the feeling of a "living" VR-Friend.

W: This is an animation used for welcoming the user.

G: This kind of animation is used for saying goodbye to the user.

T: These animations are kind of "timeout"-animations. They are used if a user needs too long to answer a question. In this way after some time the VR-Friend could react accordingly, for example becoming impatient or urging the user to finally answer the question.

Note that for animations the declaration of a mental state is not absolutely necessary, although it is of course possible. The reason is that animations – above all the latter three kinds – are relatively seldom displayed, so there need not be too many of them. In addition animations possibly do not differ very much in dependence of the mental state.

The same rules for displaying an animation apply as for reactions with the exception that the VR-Friend’s default view will not change in the meantime.

• Picture: This field is used for storing the default view, a reaction or an animation, thus holding a picture or a series of pictures. For this purpose and depending on the implementation type TPicture is either holding the picture(s) itself or the information where the picture(s) can be found.

The size of a reaction’s or animation’s pictures could be reduced if only that part of the default picture is replaced which is really needed for displaying the reaction or animation. Blinking with the eyes, for example, does not require a picture with closed eyes showing body and head of the VR-Friend but only the area of and around the eyes.

As an exception default views must never be reduced in size but have the dimension that is defined for the VR-Friends system.

To be able to realize the usage of partial pictures, two additional numbers have to be stored together with a picture. These values are needed to display a picture at the correct position within the default picture. These numbers are called OffsetX and OffsetY (holding the distance between the origin of the default picture and the partial picture). The default values for OffsetX and OffsetY are zero thus indicating to display the picture in the default position (but not necessarily saying that it has the default size).

In connection with picture sizes it is to mention that for the whole VR-Friends system the maximum dimensions must be defined. All default views of all available VR-Friends must be dimensioned according to this size.

• OffsetX, OffsetY: Horizontal and vertical offset of a partial picture (or series of pictures). See above for details.

If these fields contain null-values and the field Picture contains data, the null-values are considered to be zero.

• ShowFrames: Special picture formats supports the storage of several frames that can be displayed one after the other, in this way creating an animation. The GIF picture format as an example for such a picture type.

In addition to the possibilities such a format offers, with this field the maximal number of frames can be defined. If the specified number of frames has been shown the animation is stopped.

This field is only used for reactions and animations and should contain null-values for a default view. If no value is specified in this field, the entry of field ShowSeconds is used.

• ShowSeconds: The duration of the display of reactions or animations must of course be limited. If the picture contains no frames and no value is specified in field ShowFrames, this field is used to display the reaction or animation for the specified duration. If this field does not contain an entry, a default duration is used.

This field is only used for reactions and animations and should contain null-values for a default view.

• LangDataID: Textual or auditive reactions (or parts of reactions or animations) may be provided in different languages. Therefore these informations have to be stored in another table called TblLanguageData. So, if there is stored some textual or auditive data, the field LangDataID contains the ID of the corresponding records of TblLanguageData each of those containing the same data in different languages (see below for a more detailed explanation of this table).

Thus, this field serves as a primary key for the master-detail relationship between TblMentalStates and TblLanguageData.

 

1. Table TblLanguageData

Textual or auditive data depends on the chosen language and can therefore not be stored in TblMentalStates because the number of available languages is not fixed. On the contrary it should be possible to add another language in an easy way, of course not having to change the structure of the database. Therefore, for the two existing parts of "information" that have to be provided in different languages a dependent table of TblMentalStates is needed. These informations are the auditive and the textual parts of a reaction or animation thus giving us table TblLanguageData.

In the following Table 4 an overview of the contents of TblLanguageData is given that is followed by a detailed explanation of all used fields.

 

Field	Data type	Restrictions	Explanation
LangDataID	Integer	FK	ID for some audio to play or text to display.
LanguageID	TLangID	FK	ID of selected language.
Audio	TAudio		Holds the audio data in appropriate language.
TypeOfAudio	Char		Specifies whether the audio is sound or spoken words.
Text	String		Text in appropriate language.

Table 4: Definition of the table TblLanguageData of the Reaction Database

 

This is the detailed explanation of all used fields:

• LangDataID: The ID’s stored in this field correspond to the LangDataID’s of a reaction or animation of TblMentalStates. For each of those values more than one record can exist in table TblLanguageData giving the same information in different languages. Therefore, to retrieve the wanted information from this table not only the LangDataID is to be used but also the ID identifying the language (field LanguageID). This is no problem since the language was chosen by the user before the VR-Friend comes into action. So the language ID can be regarded as a known constant during the whole learning session, automatically restricting the contents of TblLanguageData.

This field is the foreign key of the master-detail relationship between TblMentalStates and TblLanguageData. Therefore per definition it is not allowed to contain null-values.

• LanguageID: This ID identifies the language of this record’s data. Under certain circumstances this field may contain null-values although it actually is a foreign key (see below for details), usually it contains the values of column LanguageID of table TblLanguages

Nevertheless the combination of the fields LangDataID and LanguageID represents a unique index in table TblLanguageData. This implies that at most only one record may have a null-value in field LanguageID for a given value in LangDataID.

• Audio: This field is used for storing audio. For this purpose type TAudio is either holding the audio data itself or the information where the audio file can be found.
• TypeOfAudio: Because of the possibility to independently control the playing of music, sound effects and spoken words during a game-show, there must be a criterion that can be used to identify the contents of audio data. Since for the VR-Friend’s auditive parts of reactions or animations only sound effects and spoken words must be taken into consideration, the following two types are introduced:

S: The audio data only includes sound without any spoken words.

W: The audio data of this record includes (at least some) spoken words.

The appropriate audio type must be defined by the creator of the VR-Friend because there is of course no other possibility to dinguish between sound and spoken words.

This field must contain one of the above types for every record that does contain data in field Audio.

• Text: In this field the textual part of a reaction or animation in the given language is stored.

Because either an auditive or a textual part of a reaction or animation needs not have to be defined, one of the fields Audio and Text can have a null-value (so being empty). Having both of these fields empty for a given reaction or animation is not allowed because the corresponding field LangDataID in TblMentalStates will already be empty in this case.

For certain text messages it could be desirable to include the name of the user or the VR-Friend in the text. For this purpose the following character sequences can be used:

• "%u" inserts the user name at this position.
• "%v" insert the name of the VR-Friend at this position.

To give an example let the user name be Sabine and the name of the VR-Friend George. The text message "Hello %u! My name is %v." will then be changed to "Hello Sabine! My name is George." before the text is displayed.

 

Finally a special case must be taken into consideration: The auditive part of a reaction or an animation might be some sound without spoken words. Therefore there is no use to store the same sound-data for all languages. In this case it is allowed that the field LanguageID as described above does not contain an ID (but a null-value instead). In this case only the field Audio is meaningful to use in such a record.

If in addition some multilingual text should be needed for that reaction/animation, extra records have to be stored in this table for each language with the usual combination of fields LangDataID and LanguageID.

To realize this special case the following rules for requesting the language dependent data from the database for a reaction or animation are required:

1. First a record is searched that has the given LangDataID for the reaction/animation and the known LanguageID. If a record is found, the entries of the fields Audio and/or Text – whichever of those contains data – is to be used.
2. If such a record does not exist or the found record has no entry in field Audio, another record is searched having the same LangDataID but no LanguageID (that means LanguageID equals null). The entry of field Audio of the new found record is used now and either represents or supplements (using text-data of rule 1) the result of the first search.

Figure 8 shows the described query process in a flowchart style. There the order of the queries is apparent including their respective results in grey rectangles. The retrieved text-data of the first query is used depending if one was found. The second search (if it is needed at all) is for audio data only which then replaces the audio data of the first search. In other words the result of the whole query process is the fetched text data and the last fetched audio data.

 

Figure 8: Query process of language dependent data.

 

1. Table TblLanguages

This table contains the ID’s and names of all available languages. This table is on the one hand needed for letting the user choose the desired language out of all available. On the other hand it is useful that same languages have the same ID which also gives the wizards the possibility to fill the tables homogeneously.

In the following Table 5 an overview of the contents of TblLanguages is given that is followed by a detailed explanation of the used columns.

 

Column	Data type	Restrictions	Explanation
LanguageID	TLangID	Not null, PK	Unique ID (clearly identifying each available language)
Name	String	Not null	Name of language

Table 5: Definition of the table TblLanguages of the Reaction Database

 

This is the detailed explanation of the used fields:

• LanguageID: This is an ID that allows the unique identification of the language. The values of this column are stored in column LanguageID of table TblLanguageData.
• Name: The name of a language is stored in this field. This is only used for letting the user choose the desired language in module UserSelect.

 

1. Status Database

Before a user is able to start a learning session (that is a game-show), he or she has to choose a VR-Friend to play with. From this moment on the chosen VR-Friend always has a certain state referred to that individual user. This state consists of the VR-Friend’s mental state and concentration degree and persists until the VR-Friend will be deleted. So a VR-Friend can accompany the user during many learning sessions with a continuos behaviour between the sessions. The reason is that it is desirable that the status of the VR-Friend is the same when starting the next learning session as it was at the end of the previous session. As a consequence this status has to be saved for each VR-Friend and each user which is achieved by defining a table called TblVRFriendStatus.

Of course also the users, especially the ID’s of the users, must be stored in a way that a user can uniquely identify himself or herself by his or her name. In addition a password must be provided that a user is not able to play unauthorized as another user. These user informations are stored in table TblUser.

 

1. Definition of Tables

Figure 9 illustrates the definition of the tables (and their relations) that are used in the Status Database. The same EER-model with the mentioned adaptations is used as for the Reaction Database in chapter 3.3.3.

 



Figure 9: Database model of the Status Database

 

The tables TblVRFriendStatus and TblUser set up a master-detail relationship. For each stored user there exists one row in TblUser. In TblVRFriendStatus any number of rows can be stored that are related to a user in table TblUser.

In the following two chapters the tables included in the database model are explained in detail.

 

2. Table TblVRFriendStatus

This table holds the status-information, that is the current mental state and concentration degree, of a VR-Friend referred to a certain user. In the following Table 6 an overview of the contents of this table is given that is followed by a detailed explanation of all used fields.

 

Field	Data type	Restrictions	Explanation
UserID	TUserID	Not null	System’s user ID
VRFriendID	Integer	Not null	Unique ID of VR-Friend
MentalState	Signed Decimal	Not null	Current mental state
ConcDegree	Integer	Not null	Current degree of concentration

Table 6: Definition of table TblVRFriendStatus of the Status Database

 

These fields are used:

• UserID: This field contains an ID that uniquely identifies the user logged in. Because this highly depends on the platform the system is to be implemented, type TUserID is used as the type for this field.
• VRFriendID: This field holds the ID of that one of all available VR-Friends that the user (who is owning the current UserID) has selected before starting the game-show. Therefore this ID is the same as in TblVRFriends of the Reaction Database.
• MentalState: This field contains the actual mental state of that VR-Friend that is playing with the given user. It references one of the available mental states for this VR-Friend as described in section 3.3.1.
• ConcDegree: This field stores the actual degree of concentration. The behaviour of a VR-Friend relating to answering questions depends on this value. A detailed explanation of concentration degrees can be found in chapter 3.5.2.

 

Apparently this Status Database must be stored on a backend server establishing the same client/server model as it is to be introduced for the Reaction Database. For the Status Database the client (namely module VR-Friend Server) is the same as for the Reaction Database. The backend now includes the table explained above and needs a database engine.

Because of the similarities to the Reaction Database it seems to be more convenient to enhance the Reaction Database by the table defined above and also to use the database engine that must already be created for running the Reaction Database itself. As another advantage the same communication channel can be used for storing and retrieving the VR-Friend’s status as it is used for communicating with the Reaction Database. This is true because the client and server parts of the connection are the same now.

For the purpose of specifying the Reaction Database and Status Database it makes no difference if the specified features will be implemented in one module by enhancing its functionality or by creating two modules each of them only taking care of its functions. Therefore a justified decision to choose one of the two possibilities will be given in the implementation details.

 

Although the communication details between the client and server side regarding the Status Database are again left for chapter 3.7, it is to discuss, how often the actual status should be saved to the database. There are many possibilities ranging from the most secure but slowest approach to the most error-prone but fastest one.

The most obvious rule for storing the status is to send new status data whenever it has changed. The drawback for this method is that the status usually changes as a result of a user action. But such an action must also be transferred to the server and causes a reaction from the server, which usually will be another page to display in the browser. So it seems not advisable to produce additional traffic on the net and load on the server for transferring and storing status-information, which is not of such importance that it must be saved in the Status Database immediately.

On the other extreme the actual status-information could be saved to the database when a learning session is finished, no matter if it was cancelled by the user or terminated by the system after answering the last question. This approach requires the status-information to be sent only once thus producing least traffic and load. The drawback is the possibility of a broken connection between the VR-Friend Server and the database server, where a learning session cannot be completed in a proper way. As a consequence after restarting a learning session with the same individual VR-Friend, its mental state and concentration degree would not be the same as just before, which would be a bigger problem as the older the status-information is.

Therefore it seems to be a good solution to update the status-information at the server in some periodic intervals taking the current load of the system into account.

 

1. Table TblUser

This table holds informations about a user that is needed to verify logins to the VR-Friends system. Table 7 gives an overview of the contents of this table that is followed by a detailed explanation of all used fields.

 

Field	Data type	Restrictions	Explanation
UserID	TUserID	Not null	System’s user ID
UserName	String	Not null	Unique name of user
Password	String		Password for this user

Table 7: Definition of table TblUser of the Status Database

 

These fields are used:

• UserID: This field contains an ID that uniquely identifies the user. Because this highly depends on the platform the system is to be implemented, type TUserID is used as the type for this field.
• UserName: This field contains the name of the user. Although this table already owns a unique index, namely UserID, this field must be unique, too. This is because a user logs in to the system using his or her user name. Then, with this name the user ID is determined which is only for system-internal use.
• Password: A user has the possibility (and should of course also make use of it) to protect the use of his or her user name with a password. This is stored in this field.

Whenever passwords are used, security consideration should be kept in mind. This concerns both the storage of a password and the transmission from the client to the server side. However, since there are no important data protected be the passwords in the VR-Friends system, these considerations are not further dealed with.

 

1. VR-Friend Server

The module VR-Friend Server is responsible for controlling the behaviour of a VR-Friend or rather the module VR-Friend Display. For this purpose it has the following tasks:

• It has to manage an individual VR-Friend’s mental state and degree of concentration which is responsible for the decision whether a VR-Friend is "able" to answer a question correctly when the user is not able to.
• Second, according to the actions (mainly answers) of the user, appropriate reactions have to be retrieved from the Reaction Database, such as smiling, which have to be transferred to the module VR-Friend Display then.

 

To be able to fulfill the above tasks, the VR-Friend Server itself must be controlled, or better its actions must be triggered, by another module, namely the Quizmaster Server. For that a communication channel is needed that is explained in detail in chapter 3.7.

 

1. Maintaining the Mental State

The current mental state of a VR-Friend is influenced by the following factors in this order of importance:

1. The previous mental state.
2. The ability of the user to give correct answers.
3. The position in the ranking list.

 

As explained in a previous section all available mental states of a given VR-Friend are grouped in mental state pools ranging from a minimum mental state value to a maximum one. The actual mental state of a VR-Friend is moving in steps as defined below on a continuous scale between these values. A possibly fractional part of the mental state value is rounded by which the current state pool is yielded.

 

The mental state value is always changed as a result to an answer given by the user or the VR-Friend. The following table lists the possible changes:

 

Change of the mental state	Cause
Increment by the step value	The user answered a question correctly.
Decrement by the step value	The user was wrong but the VR-Friend answered correctly.
Decrement by the double step value	The user as well as the team gave a wrong answer.

Table 8: Change of the mental state in response to the team’s answer

Figure 10 shows these possible changes of the mental state in response to the user’s or rather the VR-Friend’s answer. The starting point always is the current mental state that is changed upwards or downwards by a step value.

Figure 10: Change of the mental state in response to the team’s answer

 

The used step value cannot be a constant value because the number of mental state pools as well as the number of questions per session is not fixed. The basic idea for the definition of this value is as follows: Starting with mental state zero at the beginning of a learning session and assuming that the session includes some number of questions and the user is able to answer all of them correctly, the mental state should be the maximum value at the end of the session.

This seems to be a practicable solution since the only way the mental state value increases is a correct answer given by the user. Even a correct answer of the VR-Friend decreases the mental state and even worse if no answer was given by the team the mental state value is decremented two times the step value. That means that the mental state of a VR-Friend changes to a bad mood easier than to a happy one. That should be compensated by the user being expected to be able to answer more questions right than wrong.

To give a formal definition for the step value let the number of questions of the current session be n and the maximum mental state value max (note that the neutral mental state is always zero). Then the step value is defined as

.

This is the default step value that is used for all VR-Friends. However – to enable creators of VR-Friends giving the VR-Friends some sort of traits – there is the possibility to alter the mental state step value for an individual VR-Friend by setting the appropriate field of the corresponding row in table TblVRFriends of the Reaction Database. If such a value is present, it is multiplied with the default step value as defined above, otherwise the default value is left unchanged. If, for instance, a value of 0.5 is stored in that table, the step value is halved, letting the VR-Friend slower change its mental state.

Figure 11 shows an example for a mental state scale ranging from –2 to +2 giving a max-value of 2. The number n of questions is assumed to be 5 for simplifying the diagram. Therefore the above formula gives a step value of . In other words when starting from mental state and mental state pool zero, answering correctly every question in a row (white arrows) will finally end up in the highest mental state as well as mental state pool (+2).

If, however, the user is not able to answer the 4^th question for instance but the VR-Friend does, the mental state is decreased as pointed out by the grey arrow. In the case that the VR-Friend should not answer the question, the mental state is decreased more as shown by the black arrow. In this case even the mental state pool is changed downwards.

 

Figure 11: An example for a mental state scale

 

As mentioned at the beginning of this chapter there is another criterion that influences the mental state, namely the position in the ranking list. There exists an entry for every game-show a team has completed. Each entry consists of the number of points the team gathered, together with date and time of the game-show. Only the position of the actual team’s most recent entry for the given game-show has an influence on the VR-Friend’s mental state.

Of course the number of entries in the high-score table varies, so absolute ranking numbers cannot be used. Instead of that a normalized value is taken for the position to obtain a factor that is multiplied by the step value defined above.

In the case that a correct answer was given by the team, the intent is to increase the step value, if the team’s ranking is good and to decrease it otherwise. So a good place in the ranking list causes the mental state to increase faster after a right answer whereas a bad place impedes the rise of the mental state.

In the opposite case of a team’s wrong answer the step value is increased if the team has a bad ranking and decreased if the ranking is better. As a result a wrong answer at a bad ranking will lower the mental state more then at a good ranking.

To give a formal definition of the influence of the current place in the ranking list on the mental state of a VR-Friend, let p be the current place and n the number of entries in the ranking list. Then gives the normalized position in the table (valid for all entries in the ranking list), independent of the number of entries, namely a number in the range of . The lower this value the better the ranking is. Because a ranking list with less than three entries is not meaningful, the step value is not changed in this case, which therefore is not further dealed with.

In the following table (Table 9) the factors are defined by which the step values are multiplied for both cases of a right or wrong answer. To illustrate the way those factors work, the possible range of the factors’ values are given together with an explanation of the factors’ effect on the step value at a good or bad ranking. Note that the step value (now multiplied with the respective factor) is added to the current mental state after a right answer but subtracted from the mental state in case of a wrong answer (see the table at the beginning of this chapter for details).

 

Team’s answer	Factor	Range of factor (good ranking to bad ranking)	Effect on step value at
			good ranking	bad ranking
Right			Increased	Decreased
Wrong			Decreased	Increased

Table 9: Effect of the ranking list on the step value for mental state changes

 

The factors are chosen in a way that the step values are modified by a maximum of 50 percent of their original values either upwards or downwards. Which value in this range of 50 to 150 percent of the step value is taken, linearly depends on the team’s position in the ranking list.

 

So far those criterions have been explained that influence the calculation of a new mental state. To illustrate how this calculation works, Figure 12 shows in a diagram the determination of a new mental state.

 

Figure 12: Calculation of new mental state

 

The last criterion that influences the mental state of a VR-Friend are the intervals between two game-shows. The idea behind it is that a user should complete a game-show frequently, yet it is of no importance which game-show is done. So when starting a game-show the date and time of the user’s last game-show must be determined, which could be done by querying the ranking list.

After that the number of days since the last game-show can be calculated. The specific effect on the VR-Friend’s mental state however depends on some factors that have to be defined for the whole VR-Friends system. That is the frequency a user is required to carry out game-shows. If the daily use of the VR-Friends system is desirable, the mental state of the VR-Friend could be decremented by the step value for each day that is beyond the required duration of one day.

An upper limit of the change of the mental state should be defined, because otherwise a longer absence from doing game-shows would result in the VR-Friend’s worst mood. This might adversely effect the motivation of a user or rather let the user want to select another VR-Friend with the default (neutral) mental state.

In the opposite case the mental state could be increased if the frequency of performed game-shows is considerably higher than required.

 

1. Maintaining the Degree of Concentration

The degree of concentration is a probability value that determines how likely the VR-Friend is able to give a correct answer if the user is not able to. By default this value ranges from 2 to 95 percent. As for the mental state these default values can also be changed by setting the appropriate fields for the lower and upper bounds in table TblVRFriends of the Reaction Database. In this way the default boundaries of the concentration degree are replaced by those stored values independently from each other.

The degree of concentration is influenced by the following factors:

• The previous degree of concentration.
• The answers of the user and the VR-Friend.
• The current mental state of the VR-Friend.

 

The actual degree of concentration of an individual VR-Friend only changes in response to answers of the user. So the basis of a new concentration degree is of course the current one that will change during a game-show session due to one of the triggers listed in the following table:

 

Change of the concentration degree	Trigger
Increment by a step value	The user answered a question correctly.
Decrement by a step value	The user as well as the team gave a wrong answer.
Stays the same	The user was wrong but the VR-Friend answered correctly.

Table 10: Change of the concentration degree in response to the team’s answer

 

Figure 13 shows these possible changes of the degree of concentration in response to the user’s or rather the VR-Friend’s answer. The starting point always is the current concentration degree that is changed upwards or downwards by a step value or is left unchanged.

Figure 13: Change of concentration degree in response to the team’s answer

 

If the user is not able to give the right answer, the VR-Friend Display will decide whether or not the VR-Friend "is able" to answer correctly. This is how the decision is made:

If the question was not asked before or the user was not able to give the correct answer at the latest try, the VR-Friend will only give the correct answer with the probability of the actual degree of concentration or else it answers incorrectly.

However, if the user did answer the question the last time, the VR-Friend might more likely be able to "remember" the correct answer. For this behaviour the mean value of the current concentration degree and 100 is calculated, thus giving a value that is more or less higher than the current concentration degree. The VR-Friend will now give the correct answer with the probability of this value.

Internally the module VR-Friend Server will of course only send the information to the Quizmaster Server whether or not its answer is right, instead of sending the answer itself. Moreover the VR-Friend Server does not need to know the contents of the question.

 

So the main reason for the change of the concentration degree is the last (correct or incorrect) answer given by the team. That decides if there is an improvement, a deterioration or no change of the degree of concentration. If there is a change, its extent (in the above table pointed out by the step value) is determined by the previous answers of the user. For this the actual step value is defined to be the number of questions, the user answered equally (right or wrong) in a series. Note that this series is only related to the answers of the user but not of the team. The reason for that is illustrated in the following example of a possible scenario:

Say, the user answers the first question right. Then the step value will be one, thus incrementing the degree of concentration by one. After answering the second question correctly the step value will be two, incrementing the concentration degree by two and so on. So if the user has a good run, the step value will grow resulting in a faster increase of the degree of concentration of the VR-Friend. However it is to expect that at some point during the game-show the user will not be able to answer a question. Then the VR-Friend can jump in answering the question right or wrong according to the probability given by its degree of concentration. But in any case the series of good answers of the user is terminated and therefore the concentration degree will not substantially change for the next few questions.

Actually, if the VR-Friend answers correctly, its concentration degree stays the same at all as it was defined in the table above. Besides the number of questions is already counted that could not be answered by the user in a row. When the VR-Friend has to jump in for such an answer, it should be able to answer it according to its degree of concentration. But if there are more questions the user fails to answer now, it will happen that there is a question the VR-Friend will not answer. In this case its concentration degree will suddenly drop fast (the step value is now the number of answers the user failed to give in that series), until the user is able to answer a question again. That stops the decrease of the concentration degree and a new series of answers begins.

As we can see, it is necessary to use the number of the user’s instead of the team’s incorrect answers because otherwise the degree of concentration will be very unlikely to decrease again after it has reached a considerably high value. For that assume a high VR-Friend’s concentration degree and let the user not being able to answer questions. The VR-Friend will answer most of the questions correctly then due to the high probability value. In the rare case of the VR-Friend failing to answer, the concentration degree will only be decremented by one because that would be the first incorrect answer of the team. Because it is very improbable that the VR-Friend as well as the user would fail several times in a row, the degree of concentration would - if at all - only decrease in very small steps.

 

The last criterion for the change of the degree of concentration is the actual mental state of a VR-Friend. The current mental state (or strictly speaking the current mental state pool) should cause the following effects on the concentration degree, whenever the concentration degree itself is going to change as a result to an answer of the team (as described at the beginning of this chapter):

• A mental state of zero has no effect on the degree of concentration.
• If the degree of concentration is to be incremented by some step value as described above, a positive (that means a "happier") mental state increases this step value resulting in a higher concentration degree.

Whereas a negative mental state (which represents bad mood) decreases this step value, possibly down to zero. However, the step value cannot become a negative value. Therefore in this case the degree of concentration is less increased or perhaps stays the same at all.

• In the case of the concentration degree being decremented the same rules as above apply, but in the opposite sense. A positive mental state lets the concentration degree decrease less by reducing the step value which again can be reduced to zero at the most.

If the mental state is negative, the step value is increased therefore giving a lower degree of concentration.

In addition the maximum extent of the change of the step value is defined to be two by default. This value can be changed by providing an entry in the appropriate field of table TblVRFriends (see chapter 3.3.3.2 for details). For the following definitions let MaxChange be the maximum change value (either the default value or that specified in TblVRFriends). So the change value ChangeValue of the step value is an integer value defined as

.

Since now the effects of the mental states and their extents are laid down, the exact calculations of the step value’s changes can be defined. For doing so in the following the step value will be named StepValue, the current mental state pool MentalState, the minimum and maximum mental state pool values MinState or MaxState respectively.

The minimum and maximum mental state pool values are necessary because the number of mental state pools is not fixed. As the extents of the concentration degree’s changes should not exceed MaxChange, these values are needed for mapping the current mental state pool to a range from 0 to 1.

 

Concentrat. degree	Mental State	Effect on deg. of concentrat.	Calculations of change and step value
Increasing	Positive	Increasing more
	Negative	Increasing less
Decreasing	Positive	Decreasing less
	Negative	Decreasing more
Increasing Decreasing	Neutral	Only dependent on step value	The step value is unchanged.

Table 11: Effect of the mental state on the concentration degree’s step value

 

Table 11 shows the calculations of the change values that are applied to the step values for all possible combinations of increasing and decreasing concentration degrees as well as mental states. Note that the change values are always equal to or greater than zero and rounded to the nearest integer value before they are added or subtracted to the step values which are integer values, too. The maximum-functions are used to prevent the step values from getting negative values.

For clearness the directions of the changes of the step value in dependence of the increasing or decreasing concentration degree and the mental state are shown in Figure 14.

 

Figure 14: Changes of concentration degree and its step value caused by the mental state

 

1. Choosing Animations

According to the four different types of animations (see chapter 3.3.3.3) there are four different situations when an animation is displayed.

First the display of a VR-Friend has not changed for some time, so an animation of the VR-Friend is displayed to give the feeling of a "living" VR-Friend. Such an animation contains some neutral motion (that is why these animations are of type N which stands for neutral) of a VR-Friend (for example blinking with the eyes or looking around).

Second the user has just selected a VR-Friend to play with. Then this VR-Friend appears and welcomes the user. Such welcome scenes are animations of type W.

The third situation occurs when a VR-Friend has to say goodbye, after a user decided to quit playing (at least with this VR-Friend). Animations of type G are used for these situations.

Finally the fourth situation takes place if a user needs too much time to answer a question. In this case an animation of type T (for timeout) is displayed that should show the VR-Friend becoming impatient.

As explained earlier all these animations may be but need not be assigned to a mental state. For the latter three situations it is obvious when they have to be displayed. For the first case the moment when such an animation is to be displayed depends on

• the time of the last input or action of the user,
• the time of the last displayed animation and
• the connection to the VR-Friend Display (this is of course also true for the fourth, the timout, situation). This point can be neglected because it is the VR-Friend Display that determines when an animation is to display and in this case requests the animation at the VR-Friend Server.

 

The duration the user did not show any reaction could on the one hand be defined as a fixed value such as one minute. On the other hand the learning module could provide an average answering time for a specific question. In any case the duration should be randomized in a certain range. The so obtained "timeout" of course also applies to the situation where an animation was displayed and the user again did not show any reaction.

After the VR-Friend Display sends a request for an animation, the VR-Friend Server has to decide which of all available animations for the current VR-Friend should be displayed. The available animations include all animations of type N or rather T of the current mental state pool or without having specified a mental state pool.

The animation to display could be a randomly selected out of one of those, favourably not repeating the animations displayed last, in order not to repeat the same animations too often. Anyway the chosen animation is transferred to the VR-Friend Display and displayed (see chapter 3.6.2 for display and chapter 3.7 for communication details). An already transferred animation could be stored at the client side for later use, so that the VR-Friend Server does not need to send the animation’s data more than once. Only the command to display the respective animation is sent to the VR-Friend Display in this case.

The retrieving of animations from the Reaction Database and sending to the VR-Friend Display could also be done in advance thus locally storing and keeping animations for later use at the client side. For this the same considerations can be taken as already described in chapter 3.4. That means that in those times, when the communication channels between the frontend and the backend servers are not used (for example when the user is thinking over an answer), animations could be fetched from the Reaction Database for local (client-side) caching. By doing this each animation is – of course – only fetched once from the database.

 

1. Choosing Reactions

The choosing of reactions works in a similar way as with animations. However reactions are always displayed in response to a user or rather the team answering a question. Such a question could be answered right by the user, wrong by the user but right by the VR-Friend, or wrong by the team. The VR-Friend Server gets these informations by the Quizmaster Server which is also running as an Inbetween Module.

According to one of these possibilities the VR-Friend Server determines the new mental state as previously described in chapter 3.5.1.

Then a request for a reaction is sent to the Reaction Database with the information about the VR-Friend, the correctness of the answer, and the current mental state included. A reaction is chosen by the Reaction Database and transferred back to the VR-Friend Display in the same way as an animation. Caching of the reactions is possible as well as for animations.

 

1. VR-Friend Display

The module VR-Friend Display is responsible for the display of a VR-Friend. It is completely controlled by the module VR-Friend Server and has the following tasks:

• It has to retrieve and store default views, animations and reactions that are sent by the VR-Friend Server.
• It has to display the VR-Friend itself (that is a default view) and all the reactions and animations that has previously been received from the VR-Friend Server, if the VR-Friend Server sends a respective command.
• It has to decide when an animation is to be displayed from time to time. In that case the VR-Friend Display has to request an animation at the VR-Friend Server which will respond by sending an appropriate one.
• Finally it has to inform the VR-Friend Server that the VR-Friend Display is going to close because, for example, the user wants to select another VR-Friend. This notification is needed for displaying a goodbye-animation and properly closing the connection between VR-Friend Display and Server.

 

1. Displaying the VR-Friend

For each mental state of all available VR-Friends a default view exists which can be displayed on the screen in various ways depending on the implementation of the VR-Friends system.

In any case these default views must be retrieved from the VR-Friend Server. When the user selects a VR-Friend, the view for the current mental state (as saved in the Status Database) is needed immediately. If the mental state changed just before the display of the reaction, the default view for the new mental state will also have to be transferred from the Backend. So, because of performance issues it will be highly advisable that all previously received default views will be kept in memory locally, particularly since there usually are only a few mental states. In this way, after a change to some mental state previously held, the corresponding new view of the VR-Friend can immediately be displayed. Of course animations and reactions could be cached as well.

Again caching of all default views during the first time of the game-show would be a good idea to improve the response time of the VR-Friend Display.

 

2. Playing Animations and Reactions

Because animations and reactions can include pictures or rather series of pictures as well as sound or text, in the following "playing" is meant in the sense of displaying pictures, playing sound or showing text.

If an animation or reaction include at least a picture, the display of the current mental state’s default view of the VR-Friend is temporarily suspended. Then the contents of the animations are played. Afterwards the default view is restored if the mental state did not change in the meantime. Otherwise the default view for the new mental state is displayed.

A picture or series of pictures does not need to have the same size as the default view. This would reduce the amount of data stored in the Reaction database and to be transferred to the VR-Friend Display by omitting parts of the display that are not touched by the animation or reaction. In this way the speed of displaying the picture(s) can be improved considerably. To realize this optimization, offset values are provided in the Reaction Database (see chapter 3.3.3.3).

 

1. Auxiliary Modules

As mentioned previously two auxiliary modules are needed, namely the User Select and the VR-Friend Select.

 

The User Select provides the possibility for the user to login to the VR-Friends system by entering a user name and a password. For that a user must have been created previously. Finally the User Select should allow the renewal of the password.

If a user wants to login for the first time (or wants to create another user), the user name and the password must be entered. It is advisable to let the user retype the password to avoid typing errors. A new user ID is created and stored in table TblUser of the Status Database together with the user name and password.

When a user wants to login the next time he only has to enter the user name and password. These informations are verified with the data stored in the Status Database. On this occasion the user ID is retrieved from the database that is needed for the system later on.

For the case of a password change the user has to enter the user name, the old password and the new one. Again it is advisable to let the user retype the new password. Before the new password is stored, the user name and old password is verified with Status Database’s data.

After the module User Selects terminates the user ID of the logged in user is known. The second output of this module is the language, the system should use. For this purpose the User Select has to provide some sort of choice possibilities where all available languages are listed. This selection can be filled using table TblLanguages of the Reaction Database.

 

The second auxiliary module is the VR-Friend Select that lets the user choose the VR-Friend he or she wants to play with. For that reason all available VR-Friends must be presented in some way, for example by listing their names or displaying default views. Whichever kind of selection is realized in the implementation, the output of this module always is the ID of the selected VR-Friend.

 

2. Communication Channels

As illustrated in Figure 3 on page * and Figure 6 on page * the VR-Friends system consists of four modules and some backend servers. Only those communication channels that concern modules which are covered by this thesis are explained, that are channels where the modules VR-Friend Server, VR-Friend Display and the Reaction and Status Database Servers take part.

Table 12 lists those channels together with the information wich of the concerned modules opens or rather accepts a connection.

 

Channel	Opened by	Accepted by
CommQuizmaster	Quizmaster Server	VR-Friend Server
CommVRFDisplay	VR-Friend Display	VR-Friend Server
CommGameShow	GameShow	VR-Friend Display
CommReactionDB	VR-Friend Server	Reaction Database Server
CommStatusDB	VR-Friend Server	Reaction Database Server
CommUserSelect	User Select	VR-Friend Server
CommVRFSelect	VR-Friend Select	VR-Friend Server

Table 12: Communication channels overview

 

1. Channel CommQuizmaster

This communication channel is used for communication between the Quizmaster Server and the VR-Friend Server. By means of this channel, that is always opened by the Quizmaster Server, the Quizmaster Server informs the VR-Friend Server about what is going on in the game-show. In this way the Quizmaster Server controls the behaviour of the VR-Friend Server.

 

Event	Explanation	Data sent with the event:
		Quizmaster Server sends	VR-Friend Server replies
ShowStart	Start of game-show	VR-Friend ID	–
		User ID
		Language ID
		User name
		Total number of questions per session
		Team’s position in the ranking list
		Total number of entries in the ranking list
		Times this module was played before by the respective user
ShowQuit	Game-show terminating	–	–
QuestNew	New question displayed	K-Module or User Feeling question	–
		K-Module question was asked/answered before
QuestRight	Question answered right by user	–	–

Table 13 – continued on next page –

Table 13 – continued –

Event	Explanation	Data sent with the event:
		Quizmaster Server sends	VR-Friend Server replies
QuestWrong	Question answered wrong by user	Number of answers of the last series of correct or incorrect answers	Information if VR-Friend answers the question right or wrong
QuestUser	User Feeling question answered	Index of answer	–
		Number of possible answers

Table 13: Events of the CommQuizmaster communication channel

 

There are several events that occur during a game-show, that are forwarded to the VR-Friend Server, possibly passing on some information to the VR-Friend Server. The reply of the VR-Friend Server can eventually include some data that is sent back to the Quizmaster Server. Table 13 lists all these events.

 

In the following explanations to all these events are given:

• ShowStart: This event indicates that a user has started a game-show. All necessary initialization data is sent to the VR-Friend Server, that is the VR-Friend ID, the User ID, the user name and the Language ID. The total number of questions of the learning session is needed for some calculations.

In addition the team’s position in the ranking list and the total number of entries in the ranking list are provided. Although these informations are only needed, whenever a user is not able to answer a question correctly, they are sent to the VR-Friend Server at the start of a game-show because the position in the ranking list is not changed during the game-show.

Finally the number of times the current learning module was played before by the respective user is included in this event.

• ShowQuit: This event notifies the VR-Friend Server that the game-show is going to terminate, whether because of all questions being answered or because of the user cancelling the game-show.

If a goodbye-animation for the current mental state exists that has not been sent to the VR-Friend Display yet, it is now sent immediately without including the command to display it (see channel CommVRFDisplay for details). After that the actual VR-Friend’s status, that is the mental state and concentration degree, is stored in the Status Database and the communication channel is closed.

• QuestNew: This event is the notification of the VR-Friend Server that a new question was sent to the game-show.

As a parameter to this event an information is provided, if the displayed question is a question that is part of the learning module or if the question is one of those that are used to find out the user feeling.

An additional parameter informs the VR-Friend Server if the user was able to answer the question before. This is needed when recalculating the concentration degree after the question is answered.

• QuestRight: This event informs about a user correctly answering a question. This results in recalculating the mental state and concentration degree of the VR-Friend and the sending of a reaction to the VR-Friend Display.
• QuestWrong: This event occurs if the user was not able to answer a question right. Now the VR-Friend Server has to decide, if the VR-Friend can give the correct answer. To be able to make this decision, the VR-Friend Server first needs to know, if the user previously was able to answer the corresponding question. This information was sent as a parameter to the QuestNew event.

Beside that the VR-Friend Server needs the team’s current position and the total number of entries in the ranking list. These informations were already provided by the event ShowStart.

• QuestUser: This event tells the VR-Friend Server that a User Feeling question was answered. Such questions provide a scale from one to some number where the user can give his opinion. The lower the chosen number on the scale the more positive is the user’s assessment. On the contrary a high number indicates a dissatisfaction of the user.

As parameters to this event the range of that scale, that is the maximal index an answer can have, and the position of the user’s answer are provided. An answer of zero means that the user skipped the question.

 

1. Channel CommVRFDisplay

The module VR-Friend Display can be seen as a module that is only responsible for output, namely displaying or rather playing the VR-Friend default view, reactions or animations. All these kinds of output are either triggered by the Quizmaster Server that "tells" the VR-Friend Server that something has happened, which again sends the appropriate data to the VR-Friend Display. Or the only other possible trigger is a timeout that occurs at the VR-Friend Display. Then the VR-Friend Display requests and gets an animation from the VR-Friend Server.

So a communication channel between the VR-Friend Display and its "controller", the VR-Friend Server, is needed which is called CommVRFDisplay. As the VR-Friend Display runs on the client side, it has to open the communication channel while the VR-Friend Server is listening to this channel and is accepting connections.

This channel is opened as soon as the user selects a VR-Friend to play with and remains open until the user selects another VR-Friend or leaves the VR-Friend URL. Therefore the channel CommQuizmaster is opened only after this channel has been opened and is closed before this channel is closed. In fact, the channel CommQuizmaster could be opened several times one after the other, while the channel CommVRFDisplay stays open. This happens when a user plays several game-shows with the same VR-Friend.

In this communication channel there is no dependence between the notifications that are sent by the VR-Friend Display and the data and/or commands that are received. The available notifications are listed in Table 14.

 

Notification	Explanation	Data sent by the VR-Friend Display
VRFShow	VR-Friend Display is created	VR-Friend ID
		User ID
		Language ID
VRFHide	The VR-Friend Display is closed	–
ReqAnim	An animation is needed	–

Table 14: Notifications of the CommVRFDisplay communication channel

 

In the following explanations to all these notifications are given:

• VRFShow: This notification always is the first notification sent to the VR-Friend Server, informing it, which VR-Friend is to display, which language is to use and which user is playing. These informations are necessary to fetch the VR-Friend’s status data from the Status Database and – after that – the default views, reactions and animations for the VR-Friend.
• VRFHide: This notification is sent to the VR-Friend Server to inform it that the VR-Friend Display is being closed for some reason, for example because the user wants to select another VR-Friend.
• ReqAnim: From time to time the VR-Friend Display has to display a neutral animation. For that a timeout is used after its expiration this notification is sent to the VR-Friend Server. As a reaction to that an appropriate animation is transferred to the client side.

 

Using this communication channel the VR-Friend Server sends a VR-Friend’s data to the VR-Friend Display whenever this is necessary. The data includes the VR-Friend’s default views, reactions and animations or, in other words, all what is stored in tables TblMentalStates and TblLanguageData of the Reaction Database.

Which data must be sent in a certain situation depends on the informations the VR-Friend Server gets from the Quizmaster Server or the VR-Friend Display (the choosing of animations and reactions is covered in chapters 3.5.3 and 3.5.4).

Beside the data commands are also sent to the VR-Friend Display telling it what it has to display at the moment. This is because it is the VR-Friend Server that actually is controlling the display of a VR-Friend. Therefore the VR-Friend Server instructs the VR-Friend Display to change the default view or display some reaction or animation.

If caching of the mental state data at the client side is used, the separation of sending data and command is especially useful and effective. In this way data only need to be transferred the first time it is needed (this means the data and the command to display is sent). The next time, when these data are to display, the command causing the display will suffice. Caching of mental state data, that could be used later, could also be done by just sending the data at any time prior to use. This is especially useful for precaching default views or a goodbye- animation, for example.

 

1. Channel CommGameShow

This communication channel is used by the GameShow to notify the VR-Friend Display of the proceedings of the game-show. As explained with channel CommQuizmaster most of these events or at least similar ones also take place on the server side between the modules QuizmasterServer and VR-Friend Server which is the cause for the VR-Friend Server sending appropriate data to the VR-Friend Display.

Therefore this channel is an additional way for such notifications. The existence of this channel is also important in case of a lost connection between the VR-Friend Display and the VR-Friend Server, so that the VR-Friend Display can itself close its connection to the GameShow.

In this communication channel events that may have parameters are sent by the GameShow. On occasion a response of the VR-Friend Display is sent back to the GameShow. The available events are listed in Table 15.

 

Event	Explanation	Data sent with the event:
		GameShow sends	VR-Friend Display replies
QuestNew	A new question is completely displayed	–	–
QuestRight	The user answered a question right	–	–
QuestWrong	The user answered a question wrong	–	–
QuestUser	The user answered a user feeling question	–	–
VRFHide	The VR-Friend Display will be closed due to user selection	–	Information if goodbye-animation is available
SetOptions	Options for the VR-Friends system has been changed	Play sound effects	–
		Play speech	–

Table 15: Events of the CommGameShow communication channel

 

The events QuestNew, QuestRight, QuestWrong and QuestUser occur under the same circumstances as the events of the channel CommQuizmaster that have the same names. See there for more detailed explanations for these events.

In the following explanations to the other events are given:

• VRFHide: This event informs the VR-Friend Display that it is going to be closed, for example because the user wants to select another VR-Friend. In this case a goodbye-animation should be displayed, if one is available for the VR-Friend and the current mental state.

As explained previously the connection between the Quizmaster Server and the VR-Friend Server is closed after a game-show has been terminated. On this occasion the VR-Friend Server sends an appropriate goodbye-animation (if one exists) to the VR-Friend Display that caches it for later use. So this animation is transferred while the user is reading the results of the game-show just played and the high-score table. In the case the user plays another game-show with the same VR-Friend, the goodbye-animation could possibly be used after this game-show.

In the other case that the user wants to select another VR-Friend, the goodbye-animation for the current VR-Friend is already available at the client and can be displayed immediately. After that or if no goodbye-animation is available at the VR-Friend Display (that means there is no goodbye-animation defined for the VR-Friend in the Reaction Database) the VR-Friend Display terminates the CommVRFDisplay and CommGameShow communication channels and closes itself.

As a response to the VRFHide event the VR-Friend Display informs the GameShow if a goodbye-animation is available or not. Depending on the implementation this response can be used by the GameShow to decide, if a Continue-button should be displayed. With it the user could first watch the animation and then continue with the VR-Friend selection. Or the GameShow waits for the response that has to occurs after the animation is finished then. In this case the content of the response is ignored and the VR-Friend selection is displayed without further user interaction.

• SetOptions: This event tells the VR-Friend Display that some options concerning the execution of the game-show has been altered by the user.

As described at the Reaction Database’s table TblLanguageData sound that is played by the VR-Friend Display has one of two available types: sound and spoken words. The playing of either types can independently switched on and off by the user. The GameShow informs the VR-Friend Display by means of this event if at least one of these settings has changed. The new settings are passed as parameters to the event.

 

1. Channel CommReactionDB

This communication channel is used to retrieve VR-Friend data (pictures, sound or text) from the Reaction Database. How this communication works, highly depends on the kind of database that is being implemented. In any case the VR-Friend Server always opens the communication and gets the answer back from the database including the desired data.

There are several different types of requests that could be sent to the database, which mainly are requests to retrieve either a default view or a reaction or an animation. These requests are listed in Table 16 together with the needed data that has to be sent to the database in order to get the desired data back.

 

Request	Explanation of request	Data sent to the	Data returned by the
		Reaction Database
ReqGenData	General data	VR-Friend ID	General data from TblVRFriends
ReqDefView	Default view	VR-Friend ID	VR-Friend’s default view
		Mental State
ReqReact	Reaction	VR-Friend ID	VR-Friend’s reaction
		Mental State
		Language ID
		Kind of reaction
ReqAnim	Animation	VR-Friend ID	VR-Friend’s animation
		Mental State
		Language ID
		Kind of animation

Table 16 – continued on next page –

Table 16 – continued –

Request	Explanation of request	Data sent to the	Data returned by the
		Reaction Database
ReqLang	List of languages	–	List of languages from TblLanguages

Table 16: Requests of the CommReactionDB communication channel

 

In the following explanations to all these kinds of requests are given:

• ReqGenData: General information for the VR-Friend is required that is stored in table TblVRFriends. This includes among other things the name of the VR-Friend and certain settings that influence the behaviour of this VR-Friend. Only the VR-Friend ID is needed to retrieve such data.
• ReqDefView: The default view picture of a VR-Friend should be retrieved. Beside the VR-Friend ID the mental state of the desired default view is needed for this request.
• ReqReact: This request retrieves a certain reaction of a VR-Friend from the database. In addition to the VR-Friend ID and the mental state also the Language ID and the kind of the reaction is required. The Language ID is used for selecting the appropriate language dependent data (text and/or audio). As explained in chapter 3.3.3.3 there are positive, neutral and negative reactions which is determined by the kind of the reaction.
• ReqAnim: This request retrieves a certain animation of a VR-Friend from the database. The same applies for this type of request as for request ReqReact.
• ReqLang: This request retrieves a list of all available languages that are stored in table TblLanguages. This is needed for letting the user select the desired language in module UserSelect. Therefore this request is indirectly invoked by that module.

 

1. Channel CommStatusDB

This communication channel works analogous to channel CommReactionDB. The difference is that this channel is used for communication between the VR-Friend Server and the Status Database. Again the VR-Friend Server opens the connection and sends or receives data.

This communication channel is used for two purposes only, namely either storing or retrieving the VR-Friend’s status-information which includes its mental state and concentration degree. The other job of this channel is to maintain table TblUser holding user data. Table 17 lists the communication events of this channel.

 

Communication event	Purpose	Data sent to the	Data returned by the
		Status Database
StoreStatus	Storing status-information	User ID	–
		VR-Friend ID
		Mental state
		Concentration degree

Table 17 – continued on next page –

Table 17 – continued –

Communication event	Purpose	Data sent to the	Data returned by the
		Status Database
RetrieveStatus	Retrieving status-information	User ID	Mental state
		VR-Friend ID	Concentration degree
StoreUser	Store user data	User name	User ID
		Password
RetrieveUser	Retrieve user data	User name	User ID
			Password
ChangePwd	Change password of a user	User ID	–
		Password

Table 17: Communication events of the CommStatusDB communication channel

 

In the following explanations to these kinds of communications are given:

• StoreStatus: This communication event stores the current mental state and degree of concentration of the VR-Friend to the Status Database. For doing so the user ID and the VR-Friend ID must be provided. As explained in chapter 3.4 the status-information could be stored regularly immediately after it has changed or only once at the end of the game-show or sometime in between. When it is actually stored is the decision of the VR-Friend Server.
• RetrieveStatus: On behalf of this communication event a VR-Friend’s mental state and concentration degree is fetched from the Status Database. For that the user ID and the VR-Friend ID are needed. This information will only be retrieved once, namely immediately after the user selects a VR-Friend.
• StoreUser: This communication event stores new user data to table TblUser of the Status Database. For that the user name and password must be provided and the ID of the newly created user is returned.
• RetrieveUser: With this communication event the VR-Friend Server fetches the user ID and password for a given user name from the Status Database. This is needed to compare the password a user entered with that one stored in the database to test if the user is allowed to login with this user name.
• ChangePwd: When a user wants to change the password for the user name, this communication event is needed, which updates the password for a given user ID.

 

1. Channel CommUserSelect

When a user logs into the VR-Friends system, he must be identified by his user name and password. Furthermore it must be possible to create a new user or to change the password of an existing user. In addition a user can choose a language during the login. The available languages are retrieved from TblLanguages.

For all these tasks this communication channel is used by the module UserSelect. It first connects to the VR-Friend Server sending the request and parameters. After performing some database operations the VR-Friend Server then returns the desired data and the communication channel is closed. Table 18 shows the requests of this communication channel.

 

Requests	Purpose	Data sent to the	Data returned by the
		VR-Friend Server
UserLogin	Check login of a user	User name	User ID or error
		Password
UserLoginKill	Check login of a user and kill existing connection	User name	User ID or error
		Password
UserLoginNew	Create new user	User name	New user ID or error
		Password
UserChangePwd	Change a user’s password	User name	Ok or error
		Old password
		New password
LangSelect	Retrieve list of available languages	–	List of languages

Table 18: Requests of the CommUserSelect communication channel

 

In the following explanations to all these requests are given:

• UserLogin: A user wants to login to the system using the provided user name and password. If this combination is not correct, a login error is sent back to the UserSelect.

If user name and password are valid, all currently opened client connections are to check by the VR-Friend Server. The reason is that a given user ID is allowed to be logged in only once. So if there is another client connection open, using the same user ID, an appropriate error message is sent back to the UserSelect, which itself explains this situation to the user.

Otherwise – in case of a successful login – the ID of the user is returned.

• UserLoginKill: This request does the same as the request UserLogin with an exception for the case of an already existing connection for the given user.

That client connection is closed in such a way as if the connection would have been lost. Therefore an interrupted session would possibly be stored that can be continued later on. After that the ID of the user is returned to the UserSelect.

• UserLoginNew: A new user wants to login to system. The VR-Friend Server first has to check if the desired user name does already exist. In this case an appropriate error message is returned to the UserSelect.

In the other case the new user name is stored together with the password to the table TblUser creating a new user ID. This ID is sent back to the UserSelect.

• UserChangePwd: This request is needed for letting a user change his password. First the combination of the user name and the old password is checked. If they are valid, the new password is stored in the table TblUser and an acknowledgement is sent back to the UserSelect. Otherwise an appropriate error message is returned.
• LangSelect: This request is used for retrieving all available languages which is needed for letting the user select one of them. The names and abbreviations (which actually are the ID’s) are stored in table TblLanguages and are sent back to the UserSelect for each language.

 

1. Channel CommVRFSelect

After a user has logged into the system, he gets the possibility to select a VR-Friend. This is done in the auxiliary module VR-Friend Select either by providing a list of the names of all available VR-Friends or for example by displaying a thumbnail view of all VR-Friends.

For that purpose the VR-Friend Select has to connect to the VR-Friend Server to retrieve at least the names of all available VR-Friends. Table 19 shows the request of this communication channel.

 

Request	Explanation	Data sent by the VR-Friend Server
VRFSelect	List of VR-Friends is needed	All VR-Friend data needed to display the VR-Friend selection

Table 19: Request of the CommVRFSelect communication channel

 

All the data the VR-Friend Select will display is immediately returned by the VR-Friend Server. After that the communication is closed.

 

1. Course of Execution of the VR-Friends System

Since all parts of the VR-Friends learning system has been described so far, the overall course of execution of the whole system can be explained now. See Figure 15 for a diagram showing the order of events.

The system starts when the user opens the VR-Friends URL in his or her browser. The system appears with a possibility to enter a username which could of course be password-protected. In addition the creation of a new user must be possible.

After login the system checks if there is an interrupted session left for that user. Such a session is created, if, for example, the connection is lost during a running game-show. If such a session is available the user has to decide whether or not he or she wants to continue that session. Continuing immediately starts the game-show. Refusing the continuation or if no interrupted session exists, starts the VR-Friend selection.

There the user has the possibility to choose his or her companion for the game-show among all available VR-Friends. After that the module VR-Friend Display is started which now connects to its backend, the VR-Friend Server and the desired VR-Friend appears.

In the next step the user can select one of all available learning modules. After that the game-show is started and the module GameShow connects to the VR-Friend Display and to its backend, the Quizmaster Server which itself connects to the VR-Friend Server. The game-show is now ready to run.

 



Figure 15: Execution flowchart of the VR-Friends system

 

After it is finished a selection is presented which gives the user the possibilities to either select another VR-Friend, another game-show or repeat the same game-show with the same VR-Friend.

 

2. Time Sequences of the VR-Friends System

In this chapter it should be illustrated in which order the several communication events that have been described in chapter 3.7 can take place. Mostly, the reason for some activities of the system is the user doing some inputs or a timeout that expires. Therefore for all of these possible triggers diagrams are provided that show time sequences of the successive communication events of all involved modules.

Such an event is represented by a labelled arrow that points from one rectangle to another, thus indicating which module is talking to whom. An arrow’s label printed in italics means an event, a kind of request or name of communication as explained previously. Note that in these diagrams all communication events are chronologically ordered from top, which occurs first, to bottom, which occurs last. An event can occur only after the event(s) arranged above is (are) completed, events shown side by side may run parallel.

As mentioned in some previous chapters it is possible for the VR-Friend Server to send picture data to the VR-Friend Display in advance thus locally storing and keeping the pictures for later use at the client side. This client-side caching is not taken into consideration in the following diagrams. So with caching, if there is a default view, reaction or animation request to the backend shown, actually an access to the cache is meant. The actual fetch from the database and transfer from the VR-Friend Server could have been occurred at any other time, the VR-Friend Server had considered as convenient. Of course this is only true for data already stored in the cache. If this is not the case the request to the Reaction Database must be done as shown in the diagrams.

 

1. User Login

When starting the VR-Friends learning system, that is opening the VR-Friends URL, the user has to login to the system. For that a login screen is displayed that provides input fields for the user name and password and for the language selection. This selection is to fill with all available languages which are stored in table TblLanguages of the Reaction Database.

The time sequence for the initialization of this user login screen is illustrated in Figure 16. The User Select sends the LangSelect request of the CommUserSelect communication channel to the VR-Friend Server, which itself uses the ReqLang request of channel CommReactionDB to fetch a list of available languages from the Reaction Database. This list is passed on to the User Select then.

 



Figure 16: Time sequence for initialization of user login screen

 

After that the user has several possibilities: He can login specifying an existing user name and password, he can login as a new user specifying a new user name and a password and he can change the password of an existing user specifying the user name and old password for this user name.

 



Figure 17: Time sequence for login of existing user name

Figure 17 shows the login of a user with an existing user name. The User Select sends the UserLogin request of channel CommUserSelect to the VR-Friend Server, which fetches the corresponding user data from the Status Database using event RetrieveUser of the CommReactionStatus communication channel. Then the VR-Friend Server checks the password entered by the user and returns the user ID to the User Select, if the login is ok, or an error otherwise.

 

When a login of an existing user takes place, there is the possibility that this user is currently logged in to the system. This situation must be avoided because two clients using the same user ID are "using" the same VR-Friend status record in the Status Database. This, of course, is highly undesirable.

If a user is logged in, a connection between the VR-Friend Display and VR-Friend Server exists. If the same user again logs in, this situation can be detected by the VR-Friend Server by searching for the user ID, that is about to login, in all its client connections. This is done directly after successfully verifying the password in the Status Database. If a client connection for this user exists, an error is reported to the User Select which has to inform the user about this situation. At the same time it gives the user the possibility to close the existing connection. The VR-Friend Server then terminates this client connection and the corresponding Quizmaster connection if one exists. After that the current VR-Friend status-information will be saved to the Status Database.

 

Figure 18 shows the time sequence that occurs when a new user logs in. For this the User Select sends request UserLoginNew of the CommUserSelect communication channel to the VR-Friend Server, which tries to store the new user name with password in the Status Database (using event StoreUser of the CommStatusDB communication channel). If the user name does not already exist the ID of the newly created user is returned. In case of an already existing user name the storage attempt results in an error. In either case the result, which is the new user ID or the error is sent back to the User Select.

 



Figure 18: Time sequence for login with new user name

 

Finally Figure 19 illustrates the time sequence when a user wants to change the password. Request UserChangePwd of communication channel CommUserSelect is sent to the VR-Friend Server which tries to change the password via the ChangePwd event of the CommStatusDB channel. The result indicates success or error and is passed on to the User Select.

 



Figure 19: Time sequence for changing the password

 

2. Selecting a VR-Friend

After a successful login and before the start of a game-show a user has to select the VR-Friend, he or she wants to play with. The time sequence for this event is illustrated in Figure 20.

 



Figure 20: Time sequence for selecting a VR-Friend

 

First module VR-Friend Select sends the VRFSelect request of the CommVRFSelect communication channel to the VR-Friend Server. This module retrieves general data for all available VR-Friends from the Reaction Database using the ReqGenData request of channel CommReactionDB. Depending on the implementation of course not only the names could be fetched at this point but also, for example, the default views or even animations of the VR-Friends. Accordingly the requests ReqDefView or rather ReqAnim must be added to Figure 20.

In any case the VR-Friend Server sends the fetched data to the VR-Friend Select that displays it in this way providing the possibility for the user to select the desired VR-Friend.

 

3. Starting the VR-Friend Display

After the user has selected a VR-Friend, the module VR-Friend Display is started and must be initialized. Figure 21 shows the time-sequence for all communications that follows this event.

After the VR-Friend Display is started and initialized with the user data it connects to the VR-Friend Server sending the VRFShow notification of communication channel CommVRFDisplay.

The VR-Friend Server then has to retrieve general informations about the VR-Friend out of the Reaction Database. Already at the same time the VR-Friend’s status-information can be fetched from the Status Database. Only after that the first default view of the VR-Friend can be retrieved from the Reaction Database because the current mental state must be known for that. In addition a welcome animation is also fetched from the Reaction Database if one is available. All such data is now sent to the VR-Friend Display that displays the VR-Friend.

As explained earlier there is the possibility of an interrupted session left for a user. In this case the selection of a VR-Friend is skipped after login of a user, which is why the module VR-Friend Select must be replaced by GameShow in Figure 21 then.

 



Figure 21: Time sequence for starting the VR-Friend Display

 

4. Starting a Game-Show

Figure 22 shows the events that occur after a game-show is started. The GameShow tells its backend, the Quizmaster Server, that the game-show has started.

 



Figure 22: Time sequence for starting a game-show

 

This module sends the event ShowStart of the CommQuizmaster communication channel to the VR-Friend Server providing all data needed for initializing the VR-Friend Server as parameters.

 

5. Displaying a New Question

Immediately after a game-show has started and always after the user has answered a question a new question must be displayed by module GameShow. Figure 23 shows the time sequence for this event.

 



Figure 23: Time sequence for displaying a new question

 

First the GameShow notifies the Quizmaster Server of the new question that is to display. The Quizmaster Server sends the question that is now displayed by the GameShow. After the display is complete and ready for the user to think over an answer, the notification QuestNew of channel CommGameShow is sent to the VR-Friend Display. This starts a timeout that will be used for animations later. In the meantime – strictly speaking at the same time when the Quizmaster Server sends the new question to the GameShow – the Quizmaster Server also sends the event QuestNew of channel CommQuizmaster to the VR-Friend Server. As parameters informations about the new question are included.

If the user does not show any reaction for a certain time, the previously started timeout expires in module VR-Friend Display. In this case the notification ReqAnim of channel CommVRFDisplay is sent to the VR-Friend Server which chooses and retrieves an appropriate animation from the database using request ReqAnim of channel CommReactionDB. This animation is passed on to the VR-Friend Display using channel CommVRFDisplay and displayed at the client.

 

6. Answering a Question Right

Whenever a user answers a question correctly, the actions illustrated in Figure 24 take place.

First the GameShow notifies both the VR-Friend Display (with notification QuestRight of channel CommGameShow) and the Quizmaster Server that the user answered the question. The Quizmaster Server passes this information on to the VR-Friend Server using event QuestRight of communication channel CommQuizmaster. The VR-Friend Server now has to recalculate the VR-Friend’s current mental state and concentration degree. According to these new values an appropriate reaction is chosen and fetched from the Reaction Database via request ReqReact of channel CommReactionDB. This reaction is passed on to the VR-Friend Display using the CommVRFDisplay communication channel and displayed at the client.

It is possible that the current mental state pool has changed because of the recalculation of the mental state. In this case the new default view also must be loaded from the Reaction Database with the request ReqDefView of the CommReactionDB communication channel and sent to the VR-Friend Display where it is displayed after the just shown reaction is finished. Because the transferring of a new default view is not always necessary after a question was answered right, in Figure 24 this is indicated by dotted arrows.

 



Figure 24: Time sequence for answering a question right

 

7. Answering a Question Wrong

The contrary event to the previous time sequence Figure 25 shows the course of events that happen if the user incorrectly answers a question or does not give any answer at all.

Again the GameShow notifies both the VR-Friend Display (with notification QuestWrong of channel CommGameShow) and the Quizmaster Server that no correct answer was given. The Quizmaster Server then informs the VR-Friend Server about that using event QuestRight of channel CommQuizmaster. Now the VR-Friend Server decides whether or not the VR-Friend answers the question and sends an appropriate response back to the Quizmaster.

After that the VR-Friend Server has to recalculate the VR-Friend’s current mental state and concentration degree. With these new values an appropriate reaction is chosen and retrieved from the Reaction Database using request ReqReact of channel CommReactionDB. After that this reaction is sent via the CommVRFDisplay communication channel to the VR-Friend Display that displays the reaction at the client.

In the possible case that the current mental state pool has changed during the recalculation of the mental state, the new default view also must be retrieved from the Reaction Database using request ReqDefView of channel CommReactionDB. This default view is then sent to the VR-Friend Display where it is displayed after the just shown reaction is over. Because the transferring of a new default view is not always necessary after a wrong answer has been given, in Figure 25 this is indicated by dotted arrows.

 



Figure 25: Time sequence for answering a question wrong

 

8. Quitting a Game-Show

Finally there is one trigger left that causes several events to occur. That is quitting the game-show either because all question were answered or because the user stopped it. Figure 26 shows the time sequence belonging to that incident.

First the GameShow notifies the Quizmaster Server that the game-show is over. Then the Quizmaster Server sends the ShowQuit event of the CommQuizmaster communication channel to the VR-Friend Server. This fetches an appropriate goodbye-animation from the Reaction Database (using request ReqAnim of the CommReactionDB communication channel) and – if one exists – passes it on the VR-Friend Display (using the CommVRFDisplay communication channel), where it is cached for the time being and displayed in case that the VR-Friend Display is closed because the user wants to choose another VR-Friend.

In the meantime the VR-Friend Server also saves the VR-Friend’s current status (that is the mental state and concentration degree) to the Status Database using the StoreStatus communication event of channel CommStatusDB.

 



Figure 26: Time sequence for quitting a game-show

 

At this point it should be mentioned that the VR-Friend Server can save the status-information more often during a game-show as it was explained in chapter 3.4. It is not explicitly dealed with that special case in this chapter.

 

9. Reselecting the VR-Friend

After a game-show has been terminated, the user has the possibility to choose another VR-Friend he or she wants to play with. In this case the current VR-Friend, that is the module VR-Friend Display, must be closed. Figure 27 shows corresponding the time sequence.

 

Figure 27: Time sequence for reselecting the VR-Friend

 

First the VR-Friend Display is informed that it will have to close. Then the VR-Friend Display displays a goodbye-animation, if it has an appropriate one cached in memory. This goodbye-animation is automatically sent by the VR-Friend Server after the game-show was finished (see section 3.10.8), of course only if such an animation for the current mental state exists in the Reaction Database.

After that the VR-Friend Display informs the GameShow that the display of the animation has been completely performed (indicated with the arrow labelled "ready" in the figure). The GameShow closes the VR-Friend Display then.

 

 

 

This concludes the detailed explanation of the specification of those parts of the VR-Friends learning system that are covered by this thesis. The next chapter deals with the implementation of those parts and all related topics.

 

 

 

1. Implementation of the VR-Friends Frontend and Backend

So far this thesis covered the basis of the VR-Friends learning system and a complete specification of several parts of the system. This specification described in detail how all contained modules work and interact.

The second part of the thesis comprises the implementation of these modules and communication channels. First general objectives of the implementation are explained. Then the implementation details of the communication channels and finally of all modules itself are covered.

 

1. General Implementation Objectives

The VR-Friends learning system is intended to be a software that implements the method of incidental learning. Because only few knowledge is available about the advantages or disadvantages of this kind of learning, this software should present the possibility for scientific examinations of this topic. So special attention was given to the fact that the user should have the best possible illusion of playing in a game-show together with his or her virtual companion [Holzinger & Maurer (1999)].

The decision was made that this software should run over the internet in form of calling WWW pages (thus using the HTTP-protocol [Berner-Lee and others (1996)]), so that a user can start a learning session with his or her browser from wherever the user wants. Because of that the programming language for the client side software was also prescribed by it. Since WWW-browsers run on several different platforms, Java is the only choice for such a system.

The frontends – therefore implemented as Java applets – communicate with their backend servers. Of course the backends usually run on the same machine "forever". So a programming language could have been chosen that is appropriate for a special platform. To avoid this platform-dependency also the backend servers are implemented with Java, now creating console applications. In this way the machine(s) and operating system(s) of the backend servers can easily be replaced.

 

2. Communication Channels

This chapter describes the details of the protocols of the communication channels between the various modules. The communication takes place by sending ASCII text commands, binary data or by directly calling routines of the concerned modules.

 

1. Quizmaster – VR-Friend Server (CommQuizmaster)

This channel implements the communication between the backend modules Quizmaster and VR-Friend Server. The communication is always initiated by the Quizmaster whenever a new game-show is going to start. Therefore the VR-Friend Server listens to the predefined port, designated for the communication between these modules. The communication remains open until the game-show is over.

All communication is done by sending carriage return and line feed delimited commands, optionally followed by space separated parameters which again are terminated by a carriage return and line feed sequence. In this way always complete lines of text are sent.

Each communication event starts with the Quizmaster sending a command line (which could also be seen as a notification message) as the first line of the event. Depending on the command additional lines can be sent including parameters. The commands are case insensitive. The VR-Friend Server always responds with one line of text. This can include a special response as described later, or the text "Ok", indicating the acceptance of the received command, or an error message. This case means that the VR-Friend Server is not able to perform the required action. An error response always starts with the text "Error:" followed by an error message in plain text. In any case the response is terminated by a carriage return and line feed sequence.

In the following all possible communication events including their parameters, the parameters’ data types and the possible responses are explained in detail. The names of the parameters are only used for description reasons and do not appear in the actual communication.

 

1. Event ShowStart

This event is the first event that is sent after establishing the communication. It is only sent once during a game-show and is needed for initializing the VR-Friend Server for the started game-show session.

 

Event	ShowStart¿
Parameters	VRFriendID UserID LangID NoOfQuestions PosInHighScoreTable NoOfEntries NoOfTimesPlayed¿ UserName¿
Explanation of parameters	VRFriendID	Integer	ID of selected VR-Friend
	UserID	Integer	ID of user
	LangID	String	ID of selected language
	NoOfQuestions	Integer	number of questions the game-show includes
	PosInHighScoreTable	Integer	team’s position in the high-score table at the beginning of the game-show
	NoOfEntries	Integer	number of entries in the high-score table at the beginning of the game-show
	NoOfTimesPlayed	Integer	number of times this module was played before by the given user
	UserName	String	name of user
Response	Ok¿ or
	Error: ... ¿
	There are two possible causes for replying an error message: There is no open connection to a client with the given UserID. The arguments have an invalid format or are incomplete.

 

1. Event ShowQuit

This event is the last event that is sent before the communication is terminated. It is used for notifying the VR-Friend Server that the communication channel is being closed. Note that there is no response message that is sent back to the Quizmaster before closing the connection.

 

Event	ShowQuit¿
Parameters	-
Response	-

 

2. Event QuestNew

This event is sent whenever a new question is transferred to the frontend.

 

Event	QuestNew¿
Parameters	KindOfQuestion HowAnsweredBefore¿
Explanation of parameters	KindOfQuestion	K or U	A single character indicating that the question belongs to the learning (knowledge) module or the question is a user feeling question.
	HowAnsweredBefore	N or W or C	A single character indicating if or how the question was answered before by the user. The characters have the following meanings: N: The user has never seen the question before. W: The user did not answered the question right in the last try. C: The user answered the question right in the last try.
Response	Ok¿

 

3. Event QuestRight

This event is sent whenever the user answered a question right. With the information provided the VR-Friend Server recalculates the VR-Friend’s current mental state and concentration degree and initiates the display of a reaction at the frontend.

 

Event	QuestRight¿
Parameters	QuestId¿
Explanation of parameters	QuestId	Integer	An ID that uniquely identifies the answered question within the current learning module.
Response	Ok¿

 

4. Event QuestWrong

This event is sent whenever the user was not able to answer a question right. With the information provided the VR-Friend Server recalculates the VR-Friend’s current mental state and concentration degree and initiates the display of a reaction at the frontend.

 

Event	QuestWrong¿
Parameters	QuestId¿
Explanation of parameters	QuestId	Integer	An ID that uniquely identifies the answered question within the current learning module.
Response	Y¿ or N¿
	The response informs the Quizmaster if the VR-Friend "is able" to answer the question ("Y") or cannot answer it ("N").

 

5. Event QuestUser

This event is sent whenever a user answered a user-feeling question. Such a question does not influence the VR-Friend’s actual mental state or concentration degree, but could cause a reaction which can either be negative, neutral or positive depending on the user’s answer.

 

Event	QuestUser¿
Parameters	IndexOfAnswer NoOfAnswers¿
Explanation of parameters	IndexOfAnswer	Integer	Index of the answer on the scale of possible answers for this question. Zero means that the question was not answered.
	NoOfAnswers	Integer	All possible answers are on a scale beginning from 1 (most positive answer) to NoOfAnswers (most negative answer)
Response	Ok¿

 

1. VR-Friend Display – VR-Friend Server (CommVRFDisplay)

Both directions of this communication channel work independent from each other. The VR-Friend Display sends notification and requests to the VR-Friend Server. The VR-Friend Server mainly sends data to the VR-Friend Display that may be data the VR-Friend Display requested but can also be data the VR-Friend Server decides to display or to cache at the client side.

All the communication from the VR-Friend Display to the VR-Friend Server works similar to the communication between the Quizmaster Server and the VR-Friend Server. That means notifications are sent with carriage return and line feed delimited commands, optionally followed by space separated parameters which again are terminated by a carriage return and line feed sequence. For more details see chapter  4.2.1.

In the following three sections the possible notifications sent by the VR-Friend Display including their parameters and the parameters’ data types are explained in detail. The names of the parameters are only used for description reasons and do not appear in the actual communication. Afterwards the communication from the VR-Friend Server to the Display is explained.

 

1. Notification VRFShow

This notification is the first notification that is always sent directly after the communication channel is opened. It informs the VR-Friend Server which VR-Friend and language is used by which user, so that the VR-Friend Server can send appropriate data to the client.

 

Notification	VRFShow¿
Parameters	VRFriendID UserID LangID¿
Explanation of parameters	VRFriendID	Integer	The ID of the VR-Friend that is displayed at this client.
	UserID	Integer	The ID of the user that is playing at this client.
	LangID	String	The ID of the language the user selected.

 

2. Notification VRFHide

This notification is sent by the VR-Friend Display when it is going to close the communication channel. This happens for example after a goodbye-animation has been shown and before the VR-Friend Selection is displayed in the user’s browser.

 

Notification	VRFHide¿
Parameters	-

 

3. Notification ReqAnim

From time to time the VR-Friend Display shows neutral animations. Whenever such a timeout expires this notification is sent to the VR-Friend Server, which decides which animation is to display. If the animation is already cached at the client it sends a command to display it. Otherwise it sends the animation data and the display command. Note that no animation is displayed until the VR-Friend Server responds.

 

Notification	ReqAnim¿
Parameters	-

 

4. Communication from the VR-Friend Server to the VR-Friend Display

This direction of the CommVRFDisplay communication channel is both used for sending commands and data. The data that is to transfer mainly consists of binary data, such as pictures or sound, but also text messages and several numbers.

On principle there a two quite different possibilities to send data. First lines of text can be transferred as it is used with the CommQuizmaster communication channel. Second the data can be sent as raw data (that means as a byte array or stream) providing the possibility for the recipient to detect the end of the data.

The fact that binary data is to transfer would already require to use the latter possibility. But Java and the Internet offer another variant for data transfer. Providing a URL pointing to some picture or audio data enables Java to independently load that data over the internet. In this way only the URL, which in case is a string, needs to be sent to the client. Therefore still both above possibilities could be used for the VR-Friends system.

During the development it was not laid down from the beginning, if the actual data should be transferred from the server to the client by means of this communication channel or if the transfer should be left over to Java. Later on the decision to use this feature of Java has been made.

Nevertheless and because not only strings but also numbers are to send, the binary transfer was preferred. That means that for numbers the byte or bytes representing the number are sent without conversion. To be able to send text messages or strings, the length of the string must be transferred as an integer number directly before string itself.

To sum it up it can be said that the VR-Friend Server creates a byte stream according to the rules laid down in the following enabling the VR-Friend Display to extract the received data. These rules determine how many bytes are to be read at a certain position and how to interpret this data.

The writing and reading of this byte stream immediately starts after opening the connection. Whenever the VR-Friend Server wants to send a command or some data it starts writing data to the stream until all the data is sent. After that it returns to the initial state, that means it waits and some time later on it again sends a command or some data and so on. In the meantime the VR-Friend Display reads the stream and interprets it. Because of the stream’s format the VR-Friend Display recognises the end of a command or data record, even if the VR-Friend Server sends new commands or data directly after another (without pausing).

Therefore the byte stream of the connection can be seen as being composed of a number of blocks, each containing exactly one command or data record. A block starts with one of two different keywords, each having a length of four bytes as it could be seen in Table 20.

 

Bytes	Contents	Explanation
4	QUIT	The client connection is to close for some reason.
	DATA	Data or the command to display data is sent in this block.

Table 20: Header of byte stream block sent from the VR-Friend Server to the Display

 

After sending the Quit-keyword the connection is closed by the VR-Friend Server. This is needed when the UserSelect detects a situation where a client connection of the VR-Friend Server is still open for a user and this particular user is just logging in again. Then the user can decide to kill the already existing session (see chapter 3.10.1 for details).

In case of the Data-keyword the byte stream is continued with the data header as shown in Table 21. Two bytes define, if data is sent in the block and/or the command that data is to display.

At this point it should be noted that instead of the number of bytes the data type of a variable could be specified in the following tables. This is because Java supports the direct write of variables, for example of types Integer, Byte or Boolean, to a DataOutputStream, which is used for this connection. So the actual byte count for these data types is of no importance and left over to Java.

 

Bytes	Contents	Explanation
1	D or space	If the D is present, this block will contain data.
1	S or space	If the S is present, the indicated must immediately displayed by the VR-Friend Display
Integer	ID of data record	The ID that uniquely identifies the data record that is to display or is sent.

Table 21: Data header of byte stream block sent from the VR-Friend Server to the Display

 

Note that the D or the S or both or none can be present in the data header. For clearness Table 22 lists the meanings of these four possibilities.

 

Keys present		Meaning (for this byte stream block)
D	S	Data is sent that also is to display immediately.
D	–	Only data is sent, but it must not be displayed. This is used for caching reactions or animations, for example.
–	S	The command to display some data immediately is sent. The data is not included because it already has been sent previously. However, the ID of the data record follows, so that the VR-Friend Display knows what it should display.
–	–	This is a special case of the above possibility. It is used to inform the VR-Friend Display that another default view is to use that already has been sent previously (that is why a D is not appropriate). But it must not be displayed immediately because a reaction or animation is just shown. After finishing that the VR-Friend Display does not restore the previous default view but that having the ID that follows data header.

Table 22: Meanings of the possible contents of the byte stream block’s data header

 

If the D was not present in the data header, that means no data will be sent, this byte stream block terminates at this point. In the other case it is continued with the transferred data as listed in Table 23. The contents of those data can be a default view, a reaction, or an animation.

 

Bytes	Contents	Explanation
Integer	MentalState	The mental state of the record.
Boolean	IsDefView	The record is a default view.
Byte	KindOfReaction	The kind of the reaction (possible values: -1, 0, 1, 100).
Byte	KindOfAnimation	The kind of the animation (possible values: 0, 1, 2, 3, 4).
Integer	OffsetX	The horizontal and vertical offset values for pictures that are not to display in the origin.
Integer	OffsetY
Integer	Length of picture filename	The length of the filename of the picture file (that is the URL for the picture). If this integer value equals zero, there is no picture and the next line (Picture filename) can be omitted.
³ 0	Picture filename	The filename (URL) of the picture. The length of this entry was defined by the integer value above.
Integer	ShowFrames	The number of frames that are to display for an animated GIF.
Integer	ShowSeconds	The number of seconds a reaction or animation is to display.
Byte	TypeOfAudio	The type of the audio (sound or speech; possible values: 1, 2). If there is no audio data, a value of zero is used for this field.
Integer	Length of audio filename	The length of the filename of the audio file (that is the URL for the sound). If this integer value equals zero, there is no audio file and the next line (Audio filename) can be omitted.
³ 0	Audio filename	The filename (URL) of the sound. The length of this entry was defined by the integer value above.
Integer	Length of text message	The length of the text message. If this integer value equals zero, there is no text message and the next line (Text message) can be omitted.
³ 0	Text message	The text message. The length of this entry was defined by the integer value above.

Table 23: Data part of byte stream block sent from the VR-Friend Server to the Display

 

This is the end of a byte stream block. The VR-Friend Server can now continue to send another command or new data, perhaps some time later on or immediately. Because of the data format the VR-Friend Display knows that the next byte after the last one of this block will belong to the next block.

 

2. GameShow Display – VR-Friend Display (CommGameShow)

The communication between the modules GameShow and VR-Friend Display is only used for notifying the VR-Friend Display of the proceedings of the game-show. No information is transferred back to the GameShow.

Because these modules both run in a browser on a client machine in form of Java applets, the communication is realised by directly calling methods of the module VR-Friend Display. These methods are qualified by the VR-Friend Display Java applet. They may receive parameters or return a value. They are listed in Table 24.

 

Method	Explanation	Parameters	Returns
cmdQuestNew	A new question is completely displayed in the browser and the user can start answering it.	–	–
cmdQuestRight	The user gave a correct answer.	–	–
cmdQuestWrong	The user gave an incorrect answer.	–	–
cmdQuestUser	The user answered a user feeling question.	–	–
cmdVRFHide	The VR-Friend Display will be closed due to user selection.	–	boolean: goodbye-animation displayed
cmdSetOptions	The VR-Friend Display will be closed due to user selection.	boolean: sound	–
		boolean: words

Table 24: Java methods implementing the communication between GameShow and VR-Friend Display

 

The method cmdVRFHide displays a goodbye-animation if an appropriate one is available. After that it returns true, if an animation has completely been displayed, or false otherwise.

The method cmdSetOptions takes as first parameter the new setting for the playing of sound and as second parameter the new setting for the playing of spoken words. A value of true enables the playing, a value of false disables it.

 

The initialisation of the VR-Friend Display takes place when the Java object is created by passing parameters to the object’s constructor. However, as the VR-Friend Display is started immediately after choosing the VR-Friend, it is already active when the game-show is started. Nevertheless the needed parameters for the initialisation are laid down in the next Table 25 (compare to event ShowStart of the communication channel between Quizmaster and VR-Friend Display).

 

Parameter	Datatype	Explanation
VRFriendID	Integer	ID of selected VR-Friend
UserID	Integer	ID of user
UserName	String	Name of user
LangID	Integer	ID of selected language

Table 25: Constructor for module VR-Friend Display

 

Note that only those variables are included in the table that are essential in the context of communication channels.

 

3. User Select – VR-Friend Server (CommUserSelect)

After this communication channel is opened the User Select sends a command, optionally followed by parameters, to the VR-Friend Server which sends a response. After that the channel is closed.

All the communication from the User Select to the VR-Friend Server works similar to the communication between the Quizmaster Server and the VR-Friend Server. That means that requests are sent with carriage return and line feed delimited commands, optionally followed by parameters which again are terminated by a carriage return and line feed sequence. For more details see chapter  4.2.1.

The responses of the VR-Friend Server work in the same way as for the CommVRFDisplay communication channel, that means in form of a byte stream (see chapter 4.2.2.4). So for each response the length in bytes of all parts of the response is provided.

In the following four sections the possible requests sent by the User Select including their parameters and the parameters’ data types are explained in detail. The names of the parameters are only used for description reasons and do not appear in the actual communication.

 

1. Request UserLogin

This request is sent to verify the user name and password given by a user in order to login to the system. The VR-Friend Server returns the user ID belonging to them or an error in case of an incorrect login.

 

Request	UserLogin¿
Parameters	UserName¿ Password¿
Explanation of parameters	UserName	String	The user name and password as specified by the user.
	Password	String
Response	Integer	> 0	The user ID (only if login was ok).
		= 0	Login was incorrect because of wrong user name or password.
		= –1	The given user name is already logged in. Request UserLoginKill must be used to kill the existing connection and login then.

 

2. Request UserLoginKill

This request is sent to kill an existing connection for the specified user in order to verify the user name and password and to login then.

 

Request	UserLoginKill¿
Parameters	UserName¿ Password¿
Explanation of parameters	UserName	String	The user name and password as specified by the user.
	Password	String
Response	Integer	> 0	The user ID (only if login was ok).
		= 0	Login was incorrect because of wrong user name or password.

 

3. Request UserLoginNew

This request is sent to create a new user with the specified user name and password. The ID of the newly created user is returned in case of success (expecting that the user name did not already exist).

 

Request	UserLoginNew¿
Parameters	UserName¿ Password¿
Explanation of parameters	UserName	String	The user name and password as specified by the user.
	Password	String
Response	Integer	> 0	The user ID (only if creation of user was successful).
		= 0	The creation of the user failed because the user name already exists.

 

4. Request UserChangePwd

This request is sent to create a new user with the specified user name and password. The ID of the newly created user is returned in case of success (expecting that the user name did not already exist).

 

Request	UserChangePwd¿
Parameters	UserName¿ OldPassword¿ NewPassword¿
Explanation of parameters	UserName	String	The user name of which the old password is to change to the new one
	OldPassword	String
	NewPassword	String
Response	Integer	= 1	The password was changed successfully.
		= 0	The change of the password failed.

 

5. Request LangSelect

This request is sent to get a list of all available languages. No parameters are needed for that. As the response the language IDs and names of all languages are returned.

 

Request	LangSelect¿
Parameters	–
Response	Integer	Length of language ID	The length of the language ID. A value of zero marks the end of the list of languages. The connection is immediately closed in this case.
	> 0	Language ID	The language ID. The length of this entry was defined by the integer value above.
	Integer	Length of language name	The length of the language name. This must be a positive value because of the definition of TblLanguages (see chapter 3.3.3.5).
	> 0	Language name	The language name. The length of this entry was defined by the integer value above.
	This is the response’s data block for exactly one language. For each available language such a block is sent, one directly following the other.

 

4. VR-Friend Select – VR-Friend Server (CommVRFSelect)

After this communication channel is opened the VR-Friend Select sends the only command of this channel without any parameters to the VR-Friend Server which sends a list of all available VR-Friends including their default view of mental state zero as a response. After that the channel is closed. Per definition of the Reaction Database this default view must always exist for a VR-Friend.

The request from the VR-Friend Select to the VR-Friend Server is sent with a carriage return and line feed terminated command.

The response of the VR-Friend Server work in the same way as for the CommVRFDisplay communication channel, that means in form of a byte stream (see section 4.2.2.4). So for each response the length in bytes of all parts of the response is provided.

 

1. Implementation Details of Modules

This chapter deals with the implementation of the modules VR-Friend Server, VR-Friend Display, User Select and VR-Friend Select.

The implementation was done using the Microsoft Visual J++ 6.0 development environment. The written Java source code is fully compatible to and compileable with the Java Development Kit of version 1.1.7b.

As usual for Java all global classes are written in their own files, so that the Java class name and file name are identical. Some of all implemented classes are used for several modules. This fact is mentioned in the class descriptions below.

 

1. Backend

The only backend module is the VR-Friend Server which is implemented as a console application. The main class file that also includes the main()-method is called VRFServer. This main()-method is automatically called when the application is started. There are two possibilities for running the application:

• At a command line the Java Runtime Interpreter is to call with the above class file (including the ".class"-suffix) as a parameter.
• For Microsoft Windows only the Microsoft Visual J++ also offers the possibility to create an executable that could be run directly without manually calling the Java Interpreter.

 

There are several pieces of information the VR-Friend Server needs to be able to run, for example the communication ports or database connection settings. All these settings are stored in a configuration file (see section 4.3.1.2). The name of this file can be passed on to the application as a command line parameter. If this parameter is omitted, the file name "VRFSERVER.INI" is used as the default name. Specifying a question mark ("?") as a parameter instead of a file name displays a help screen that informs about all possible entries in the configuration file including the used default values for all entries that are not included in the file.

If invalid values are found for entries in the configuration file an appropriate error message is displayed and the VR-Friend Server is terminated.

 

Before further explanations of all classes and their interactions Table 26 gives an alphabetically sorted overview of the used classes together with a short explanation of their purposes.

 

Java Class	Brief explanation of purpose
ClientConnection	Implements the connection to a client, which can be the VR-Friend Display, User Select or VR-Friend Select.
DBConnection	Implements the connection to the Reaction Database.
DBConnectionStatus	Implements the connection to the Status Database.
Language	Holds all data of a language.
LanguageData	Holds all language dependent data for a certain language and mental state record.
Logging	Does all the logging.
MentalState	Holds all data that belong to one record of TblMentalState including the language dependent data.
MentalStateVRFS	Same as class MentalState but specialised for the VR-Friend Server.
QuizmasterConnection	Implements a connection to the Quizmaster.
QuizmasterListener	Listens to the Quizmaster connection port and starts a QuizmasterConnection thread for each accepted connection.
ReactionRecord	Stores the ID and kind of a reaction or animation.
ReactionSelector	Is responsible for selecting the next reaction or animation.
Settings	Reads from the initialisation file and holds all these settings of the VR-Friends system.
StoreReactionIDs	Stores the ID’s of all reactions or animations that belong to a mental state.
StoreSent	Keeps track of the mental state records that have already been sent to the VR-Friend Display.
User	Holds all data of a user.
VRFriend	Holds all data of a VR-Friend.
VRFriendSelect	Holds special data of a VR-Friend needed for communication with module VR-Friend Select.
VRFServer	Class holding the main method that starts and initialises the VR-Friend Server.
VRFStatus	Holds current status-information of a VR-Friend.
WaitingConnections	Stores all VR-Friend Display connections that are waiting for a corresponding Quizmaster connection.

Table 26: Java classes for the VR-Friend Server

 

1. VR-Friend Server’s Course of Execution

Figure 28 shows the main concept of the VR-Friend Server’s course of execution. After the application is started with class VRFServer, the configuration file is read (see below and class Settings) and the application correspondingly initialised. After that an instance of class QuizmasterListener is created that implements a thread and runs parallel to the main thread of class VRFServer.

From now on thread QuizmasterListener is listening for incoming Quizmaster connections at the Quizmaster connection port. For each such connection an instance of class QuizmasterConnection is started (realising the CommQuizmaster communication channel) that also implements a thread running parallel to all other threads. This thread or rather this connection is living while a game-show is running and is terminated after the game-show has been quitted.

Directly after starting the QuizmasterListener thread the VRFServer starts listening for incoming client connections at the client connection port. This port is used by modules VR-Friend Display, User Select and VR-Friend Select. Analogous to the QuizmasterListener thread an instance of class ClientConnection is started for each such connection (thus realizing the CommVRFDisplay, CommUserSelect and CommVRFSelect communication channels). Obviously this class again implements a thread, thus running parallel to all other threads. The living duration of this kind of thread depends on the connecting module.

Therefore the only tasks of class VRFServer is to initialize the system and start threads for incoming connections until the VR-Friend Server is shut down.

 



Figure 28: Execution flowchart of the VR-Friend Server

 

2. The VR-Friend Server Configuration File

As mentioned above the VR-Friend Server configuration file includes all settings needed for running the VR-Friend Server. The declaration of some of the possible values is compulsory whereas other settings have default values that may but need not be defined or altered.

The configuration file is a plain text file. A line that includes a setting must start with one of the keywords listed below, directly followed by an equal sign ("="). All following characters till the end of the line are treated to belong to the value of the setting. A line that starts with anything else than one of the available keywords is skipped and interpreted as a comment. For example, a semicolon (";") at the beginning of a line could mark a comment line for better readability, although the semicolon is not explicitly required. Keywords are case insensitive and can be defined in any order.

Table 27 alphabetically lists all available keywords together with a possibly existing default value and an explanation of the keyword. If no default value is specified, a value for this keyword must be declared in the configuration file.

 

Keyword	Default value	Explanation
ClientPort	(no default)	The connection port for client connections.
LogConnectMessages	1	Connection messages are logged.
LogDataRecMessages	0	Received data messages are logged.
LogDataSendMessages	0	Sent data messages are logged.
LogErrorMessages	1	Error messages are logged.
LogFile	VRFServer.log	Path and file name of log-file. If the path is omitted, the file is created in the same directory as the application.
LogInfoMessages	0	Information messages are logged.
LogToStandardIO	1	All the logging that is done to the log-file is also printed to the standard output.
QuizMasterPort	(no default)	The connection port for Quizmaster connections.
SqlDataSourceStatus	jdbc:odbc:VRFStatus	The ODBC datasource name for the Status Database.
SqlDataSourcReaction	jdbc:odbc:VRFReaction	The ODBC datasource name for the Reaction Database.
SqlDriver	com.ms.jdbc.odbc.JdbcOdbcDriver	The Java driver for the database connection.
SqlPassWord	(none)	The password for accessing the database.
SqlUserName	sa	The user name for accessing the database.

Table 27: Keywords of the VR-Friend Server configuration file

 

3. VR-Friend Server Class Descriptions

In this section the purposes and working methods of all classes of the VR-Friend Server are explained as far as it is necessary to understand the interactions and usage of them.

 

Class ClientConnection implements a client's connection, that is a connection from the VR-Friend Display (or User Select or VR-Friend Select) that is running on some client machine to the VR-Friend Server. This class extends the class Thread meaning that each client connection is running its own thread at the VR-Friend Server.

A client connection to the VR-Friend Display is the first connection that is established to the VR-Friend Server. It stays active before and after a game-show is actually running until the user selects another VR-Friend or leaves the VR-Friend URL.

This class contains all methods that are used to send data to the client. Those methods are also called by the corresponding instance of class QuizmasterConnection. Besides some of the variables are used by that class, too.

To improve the readability of the log-messages, where messages from all parallel running threads are appended alternately, each connection thread gets an ID-value during creation. This is an integer value starting with one for the first connection and being incremented for each further accepted connection.

 

Class DBConnection implements all connections to the Reaction Database. These are the requests of the CommReactionDB communication channel and used for fetching VR-Friend and mental state data from the database. For the database connection the settings of class Settings are used.

 

Class DBConnectionStatus implements the storing and retrieving of the VR-Friend status-information (current mental state and concentration degree) to and from the Status Database. The events of the CommStatusDB communication channel are used for that. The status-information is always saved for a VR-Friend - user combination.

In addition this class is responsible for maintaining the user data and needed by module User Select during the login of a user.

 

Class Language is only used to hold the ID and name of a language.

 

Class LanguageData is used for storing the language dependent data of a mental state record. In addition constants for possible types audio are defined.

 

Class Logging is responsible for all logging done in all threads by the individual classes. Only one instance of this class is needed, so all methods are declared static to be accessible by all classes. Because the methods are called by several threads concurrently, they are declared synchronized.

Which one of all logging messages are actually logged and the destination of the logging is decided by the settings stored in class Settings (also see section 4.3.1.2 for corresponding configuration file entries). A timestamp is inserted at the beginning of each line. After that and if applicable the ID and kind of communication thread is inserted before the actual message. "QM" means a Quizmaster Connection and "Cl" a client connection (see classes ClientConnection and QuizmasterConnection).

 

Class MentalState is an abstract base class holding constants and general variables for a mental state data record. This class is extended to build special classes as needed by the VR-Friend Server and Display.

 

Class MentalStateVRFS extends class MentalState enhancing it with the variables needed for the VR-Friend Server.

 

Class QuizmasterConnection is implemented as a thread and listens to a Quizmaster connection (connection to module Quizmaster) that corresponds to a certain game-show that has been started by the GameShow.

The creation of this connection requires the existence of an instance of class ClientConnection for the given user. If such an instance does not exist, the thread will immediately terminate (also see class WaitingConnections). This class calls several methods of class ClientConnection to send data to the VR-Friend Display. Also several variables of that class are used.

To improve the readability of the log-messages, where messages from all parallel running threads are appended alternately, each connection thread gets an ID-value during creation. This is an integer value starting with one for the first connection and being incremented for each further accepted connection.

 

Class QuizmasterListener listens to the Quizmaster connection port and starts a QuizmasterConnection thread for each opened connection.

 

Class ReactionRecord is only used for holding the ID of a reaction or rather animation together with its kind.

 

Class ReactionSelector is responsible for selecting reactions or animations out of all available ones. Usually there are several reactions or animations for a given VR-Friend and mental state that can be shown at a given situation. This class has to pay attention to using all suitable reactions or animations and not to repeat those that have just been displayed.

Another fact to consider is that reactions or animations need not have a mental state value assigned. Such records are treated to be suitable for all mental states.

The class ReactionSelector works as follows: Two vectors are defined, one for reactions, the other for animations. Both work in the same way:

For each available mental state an instance of the class StoreReactionIDs is added to the vector when the mental state is first used (this is done in the init()-method). An instance of class StoreReactionIDs stores the mental state assigned to it and itself owns a vector that lists all reactions/animations suitable for that mental state. Instances of class ReactionRecord are used in this vector.

The order of the vector’s items of the instance of StoreReactionIDs is randomized, so that the selection of reactions/animations can step from one item to the next on subsequent requests. After all items have been selected once, the items are randomly reordered. When requesting the next reaction/animation the type of the reaction/animation must also be taken into consideration.

Note that the same mental state ID's can occur more than once in the StoreReactionIDs vectors if such mental state records are reactions and animations at the same time or do not have a mental state assigned.

 

Class Settings implements the reading and holding of the configuration file’s settings. Because only one instance of this class is needed for the VR-Friend Server and to be accessible by all other classes, all variables and the initialization method were made static. The initialization is triggered by class VRFServer after start-up of the application.

 

Class StoreReactionIDs is used by class ReactionSelector. It stores the ID’s of all reactions or rather animations of a certain mental state including their kinds in a vector. In addition the index of the last used reaction- or animation-ID is also stored in this class.

 

Class StoreSent keeps track of all mental state data that have been sent to the VR-Friend Display. In this way already transferred data (inclusive of default views, reactions and animations including their language dependent data) need not be sent twice.

Before sending some of the above data to the client, the instance of this class is asked, that only and always exists in every ClientConnection thread, whether the data has already been sent to the client. In this case only the display command but no data is sent. Otherwise the data is sent and this class stores the information of the transfer.

 

Class VRFriend is only used for holding all of a VR-Friend’s data that are stored in the Reaction Database in table TblVRFriends.

 

Class VRFriendSelect is extending class VRFriend and is only used for holding special VR-Friend’s data that is needed by the module VR-Friend Select. These data are the VR-Friend’s default view.

 

Class User is only used for holding a user’s data. It is needed when communicating with module User Select.

 

Class VRFServer is the main class of the VR-Friend Server application. It can take the configuration file name as a parameter on the command line. Program execution begins with the main()-method (see above for a more detailed explanation of this class).

 

Class VRFStatus is only used for holding a VR-Friend’s current status which includes the mental state and concentration degree.

 

Class WaitingConnections maintains a vector of client connections that wait for the corresponding Quizmaster connection. Two connections "corresponds", if they have the same user ID. So if a Quizmaster connection (see class QuizmasterConnection) is accepted, its user ID is searched in the waiting client connections (also see class ClientConnections). If it is found the client and Quizmaster connections are "connected" and the corresponding entry is deleted from the list of waiting connections. If it is not found, the QuizmasterConnection thread terminates with an error.

Since only one list of waiting connections is useful for the VR-Friend Server, the vector storing the connections and all methods of this class are made static in order to be accessible by all other classes.

 

1. Frontend

The frontend consists of three modules: the VR-Friend Display, the User Select and the VR-Friend Select. These modules are realized by the classes VRFDisplay, UserSelect or VRFSelect respectively. They partly make use of the same classes and of classes the VR-Friend Server also needs.

 

1. Frontend’s Course of Execution

The complete frontend of the VR-Friends learning system not only consists of the modules mentioned above but also of module GameShow which is responsible for displaying questions and providing capabilities for the user to answer. All these modules are displayed partly one after the other or at the same time (compare chapter 3.9).

The best way for handling this was to create a container applet, named VRFControl, that is responsible for the overall execution control of the frontend. This means that this applet displays or hides the other modules as needed.

A user starts the VR-Friends system by entering the VR-Friends URL to his or her browser. That implies that from this address a HTML-page will be loaded which is the VR-Friends home page and is called VRFControl.html. This page contains an applet-tag that specifies the VRFControl class. The default width and height of this applet are 640 times 480 pixels. Two parameters must be given informing the modules about the ports to use to the backend servers. This is the used applet tag:

 

<APPLET code="VRFControl.class" width=640 height=480>

<Param name="VRFServerPort" value="9990">

<Param name="GameShowPort" value="9991">

</Applet>

 

In addition the VRFControl also is responsible for holding certain variables, such as user settings or the language or VR-Friend to use. These variables can be read or set by the various modules

After the applet has been started it loads and starts the UserSelect applet which performs the user login. Then this module stores the user and language data in the control applet and terminates, causing the control applet to load and display the VRFSelect applet. This handles the selection of the VR-Friend. After this module has stored the ID of the chosen VR-Friend in the control applet and after it has terminated, the VRFControl loads and starts the VR-Friend Display and the GameShow applet which takes over the control of the frontend now. The GameShow applet terminates after a game-show has been finished and the user has decided to choose another VR-Friend. In this case the control is again passed on to the VRFControl which runs the appropriate module again.

In case of an error of one of the modules, perhaps because of communication troubles, the control applet is informed and immediately jumps to an error page (called Errorpage.html). In this way all running applets and tasks of the initial HTML-page are aborted automatically by the browser. The error page provides the possibility to jump to the VR-Friends home page for restarting the VR-Friends learning system.

 

2. Classes for the VR-Friend Display

Before further explanations of all classes needed for the VR-Friend Display Table 28 gives an alphabetically sorted overview of the used classes together with a short explanation of their purposes. Several of these classes are also used by the VR-Friend Server. See there for explanation of such classes.

 

Java class	Brief explanation
Language	Same as for VRFServer.
LanguageData	Same as for VRFServer.
MentalState	Same as for VRFServer.
MentalStateVRFD	Same as class MentalState but specialized for the VR-Friend Display.
VRFDisplay	Implements module VR-Friend Display that is responsible for displaying the VR-Friend.
VRFriend	Same as for VRFServer.
VRFriendSelect	Same as for VRFServer.

Table 28: Java classes for the VR-Friend Display

 

All classes that have not been explained with the VR-Friend Server’s classes are described in the following:

 

Class MentalStateVRFD extends class MentalState enhancing it with the variables needed for the VR-Friend Display.

 

Class VRFDisplay implements the module VR-Friend Display. Thus it is responsible for receiving and (dis-)playing all VR-Friend related data which are default views, reactions, animations, text messages and sound.

The VR-Friend Display needs to be visible in the user’s browser together with the GameShow applet. This works best if class VRFDisplay is derived from the Java class Canvas. The reason is that a canvas can be declared to have a fixed pixel size that is not influenced by a layout manager that must be used by the GameShow to arrange the VR-Friend Display and the elements of the game-show.

 

3. Classes for module User Select

Table 29 lists all classes that are needed for the User Select. Class Language is also used by the VR-Friend Server and explained there.

 

Java Class	Brief explanation
Language	Same as for VRFServer.
UserSelect	Applet that implements the user selection.

Table 29: Java classes for module User Select

 

Class UserSelect implements the user selection and the selection of the language. For that it has to connect to the VR-Friend Server for verifying user data and retrieving available languages. The display of labels changes according to the selected language.

 

4. Classes for module VR-Friend Select

Table 30 lists all classes that are needed for the VR-Friend Select. Some of the classes are also used by the VR-Friend Server and are explained there.

 

Java Class	Brief explanation
VRFPanel	Panel for display of a VR-Friend for use in a thumbnail selection of VR-Friends.
VRFSelect	Applet that implements the VR-Friend selection.
VRFriend	Same as for VRFServer.
VRFriendSelect	Same as for VRFServer.

Table 30: Java classes for module VR-Friend Select

 

Class VRFPanel extends the Java class Panel. It is used for displaying the default view of a VR-Friend together with its name. To create a thumbnail view of the available VR-Friends for user selection, these panels are used, one per VR-Friend.

This class also realizes the selection per mouse. When the user drags the mouse over a panel it is highlighted by a frame. If the user clicks on a VR-Friend, this selection is passed on to the VRFSelect applet.

 

Class VRFSelect is responsible for the selection of the VR-Friend. For that it has to connect to the VR-Friend Server to retrieve the default views and names of all available VR-Friends. For each of them an instance of class VRFPanel is created showing a VR-Friend. These panels are arranged in form and size to fit inside the control applet.

If a VRFPanel informs the VRFSelect that it has received a mouse click (meaning the selection of its VR-Friend), the VRFSelect stores the ID of this VR-Friend in the control applet and terminates itself.

 

2. Java Class Tree

Because of the object-oriented design of the Java language, each declared class is either derived from a specified class or from the built-in class java.awt.Object by default. This class is the direct or indirect base class of all Java classes.

 

Figure 29: Java class tree

 

Figure 29 shows the class tree of the VR-Friends Java implementation starting from the class Object. A few other classes defined by the Java language are also extended by classes implemented for the VR-Friends system. All classes belonging to the Java language are indicated by a grey rectangle whereas classes of the VR-Friends system are displayed by white rectangles.

The classes are shown disregarding the module(s) they belong to. To simplify the figure, only those classes are included that either are not directly derived from the class Object or that are extended by other classes. In other words every class not shown in this figure is independently extending class java.awt.Object.

All classes in white rectangles have been explained in detail in the previous chapters.

 

1. Implementation of the Databases

Several possibilities exist how the databases, needed for the VR-Friends learning system, could be implemented. This starts with implementing an own file structure and program that serves as an engine and reaches to using commercial database products or even client/server database engines.

Of course the best solution always is the usage of existing and well-tried and therefore reliable software instead of reinventing the wheel. The disadvantage of such a software only is the cost factor, which can be considerably high for professional software.

Fortunately we had a Microsoft SQLServer available at the IICM. This one of those few client/server database management systems that hold a considerably share of the market of DBMS for personal computers. In addition the SQLServer fully supports the SQL standards. For these reasons all databases are run by the SQLServer database engine of version 6.5.

As already recommended in the specification of the databases, for this implementation there is no need to set up two databases where the individual tables are distributed. This is first because no binary data itself is stored in the database but only the URL to such data, which requires very few space compared to the amount of the binary data. Second the SQLServer would also be capable of managing very large amount of data without any problem.

Nevertheless, for clearness and consistency with the specification part of this thesis the separation between Reaction and Status Databases will be continued. This is useful from a logical point of view, even if the tables belonging to each of the databases are physically stored in the same database in this implementation.

 

1. Table Definitions of the Reaction Database

This section includes the table definitions for all tables of the Reaction Database for the SQLServer. The definitions are SQL-DDL commands that are used to create the respective tables. All field names are listed together with their data types that have been used for the SQLServer. For each field the SQLServer requires the specification of a NULL or NOT NULL keyword, indicating whether a field may contain null-values or not. Dependency constraints of the individual tables are also included.

 

The table TblVRFriends stores all VR-Friends data. The field VRFriendID forms the primary key PK_VRFriends in the relationship to the table TblMentalStates.

 

CREATE TABLE dbo.TblVRFriends (

VRFriendID int NOT NULL ,

Name varchar (30) NOT NULL ,

MentalStateFactor decimal(10, 2) NULL ,

ConcDegMin smallint NULL ,

ConcDegMax smallint NULL ,

ConcDegChangeMax smallint NULL ,

NeutralAnimMin int NULL ,

NeutralAnimMax int NULL ,

TimeoutAnimMin int NULL ,

TimeoutAnimMax int NULL ,

CONSTRAINT PK_VRFriends PRIMARY KEY CLUSTERED

(

VRFriendID

)

)

 

The table TblMentalStates stores all mental state dependent data. The field VRFriendID builds the foreign key FK_MentStat related to the field VRFriendID of TblVRFriends. Because of limitations of the SQLServer no primary key can be defined for the relationship to the table TblLanguageData because the primary key column LangDataID must be allowed to hold null-values. In addition another key is defined for column ID that only works as an index.

 

CREATE TABLE dbo.TblMentalStates (

ID int NOT NULL ,

VRFriendID int NOT NULL ,

MentalState smallint NULL ,

IsDefaultView smallint NULL ,

KindOfReaction smallint NULL ,

KindOfAnimation char (1) NULL ,

Picture varchar (100) NULL ,

OffsetX smallint NULL ,

OffsetY smallint NULL ,

ShowFrames int NULL ,

ShowSeconds int NULL ,

LangDataID int NULL ,

CONSTRAINT PK_MentStat PRIMARY KEY CLUSTERED

(

ID

),

CONSTRAINT FK_MentStat FOREIGN KEY

(

VRFriendID

) REFERENCES dbo.TblVRFriends (

VRFriendID

)

)

 

The table TblLanguageData stores all language dependent data. Actually the combination of the fields LangDataID and LanguageID forms a unique key which acts as the foreign key to the field LangDataID of table TblMentalStates. Unfortunately the SQLServer does not allow a combination of columns that can contain null-values for a key, no matter if primary or foreign key, although this would be logically correct. For this reason the definition of the foreign key must have been omitted, which is no big problem because the VR-Friend Server only has read access to the concerned tables. The only other module that accesses these tables is the wizard for filling the Reaction Database. It must be left to this wizard to guarantee the uniqueness of the combination of fields LangDataID and LanguageID for all rows of the table as stated in the specification.

The same is true for the relationship between the table TblLanguageData and TblLanguages. No primary key – foreign key combination can be declared with the SQLServer.

 

CREATE TABLE dbo.TblLanguageData (

LangDataID int NOT NULL ,

LanguageID varchar (10) NULL ,

Audio varchar (100) NULL ,

TypeOfAudio char (1) NULL ,

Text varchar (200) NULL

)

 

The table TblLanguages stores all available languages. The field LanguageID corresponds to field of the same name in TblLanguageData.

 

CREATE TABLE dbo.TblLanguages (

LanguageID varchar (10) NOT NULL ,

Name varchar (30) NOT NULL

)

 

2. Table Definitions of the Status Database

This section includes the table definitions for all tables of the Status Database for the SQLServer. The same facts apply as for the Reaction Database.

 

The table TblVRFriendStatus stores the VR-Friends’ status-information for all already played combinations of VR-Friend and user. As an index for this table the combined and therefore unique index PK_Status is defined using columns VRFriendID and UserID.

 

CREATE TABLE dbo.TblVRFriendStatus (

VRFriendID int NOT NULL ,

UserID int NOT NULL ,

MentalState float NOT NULL ,

ConcDegree smallint NOT NULL ,

CONSTRAINT PK_Status PRIMARY KEY CLUSTERED

(

VRFriendID,

UserID

)

)

 

The table TblUser stores login information about all users. The unique index PK_User for this table is built using column ID.

 

CREATE TABLE dbo.TblUser (

ID int NOT NULL ,

UserName varchar (30) NOT NULL ,

Password varchar (15) NULL ,

CONSTRAINT PK_User PRIMARY KEY NONCLUSTERED

(

Id

)

)

 

3. Security Issues and User Management

From a security point of view, there are no dangerous data stored in the Reaction and Status Databases. Nevertheless the access to these databases should be restricted to authorized use in any case, particularly since the SQLServer offers a user management. This lets control the access to all database objects.

Two user accounts are needed for the Reaction and the Status Databases. First for the module VR-Friend Server that accesses both databases and second for the wizard that only accesses the Reaction Database. Both accounts are protected by a password.

 

For the VR-Friend Server a user account named VRFServer is set up. This account has read access (meaning that the SQL-DML-command SELECT is allowed) to all tables of the Reaction Database and read/write access (only including the SQL-DML-commands SELECT, INSERT and UPDATE) to the Status Database. No DDL- or DCL-commands are allowed for this user account.

 

For the wizard a user account named VRFWizard is created. This account has only access to all tables of the Reaction Database. All read/write operations are allowed, that means all SQL-DML-commands, such as SELECT, INSERT, UPDATE and DELETE. Also for this account no DDL- or DCL-commands are permitted.

 

4. Connecting via the JDBC/ODBC Bridge

For connection to the SQLServer the VR-Friend Server uses the Java JDBC/ODBC bridge by default. The Java class name when using the Microsoft Visual J++ development environment is com.ms.jdbc.odbc.JdbcOdbcDriver.

This kind of database connection allows Java to directly access any ODBC datasources that are available to the system. Of course these datasources must be set up in the ODBC configuration to establish the connection to the SQLServer.

For each datasource the engine’s hostname must be specified which will be the local machine for this implementation of the VR-Friends system. In addition the login user name and password is to be defined as well as the SQLServer’s database that should be linked to the datasource.

 

 

This concludes the implementation details of those parts of the VR-Friends learning system that are covered by this thesis. The next chapter allows a view on the results of the implementation.

 

 

 

1. A View on the Running System

Chapters 4 and 5 included the specification and implementation details of the VR-Friends system. Beside the detailed descriptions one part of this thesis was the realization into a running system as it was explained in the last chapter. To give an impression how the implemented system looks like, this chapter contains some screenshots that show what the user sees in certain situations during the game-show or the login process.

 

The first pictures illustrate the login process starting from the user login screen till the start of a game-show.

 

Figure 30: Login screen

 

Figure 30 shows the login screen where the user can login to the VR-Friends system by providing his username and password. In addition a new user could announce his or her username and password.

 

Figure 31: VR-Friend selection

 

After successful login the user has to select a VR-Friend to play with. Figure 31 illustrates the VR-Friend selection. If the mouse cursor is moved over one of the available VR-Friends a red frame is drawn around this VR-Friend. A mouse click selects the highlighted VR-Friend and the next screen is displayed.

 

Figure 32: Selection of game-show

 

After the user has selected a VR-Friend, the VR-Friend appears welcoming the user as illustrated in Figure 32. At the same time a list of all available game-shows is displayed letting the user choose the desired one.

 

Then the game-show starts. The following screenshot shows the VR-Friend in a situation during a game-show.

 

Figure 33: Negative reaction of the VR-Friend

 

Figure 33 illustrates the case when the user could not answer the question and also the VR-Friend does not know the answer. The display of the game-show indicates this and the VR-Friend shows a negative reaction.

 

 

 

 

 

 

Appendix A: Bibliography

 

Part I: Citations

• Anderson, John R. (1985): "Cognitive Psychology and Its Implications"; 2^nd Edition; New York: Freeman.
• Atkinson, M; Bancilhon, F.; DeWitt, D.J.; Dittrich, K.R.; Maier, D.; Zdonik, S.B. (1989): "The Object-Oriented Database System Manifesto (a Political Pamphlet)"; In: Proc. 1. Intl. Conf. on Deductive and Object-Oriented Databases; Kyoto (Japan).
• Berners-Lee, Tim; Fielding, Roy T.; Frystyk, Henrik (1996): "Hypertext Transfer Protocol – HTTP/1.0"; In: Request for Comments: 1945; Cambridge (MA): MIT; Irvine (CA): University of California.
• Cattell, Roderic Geoffrey Galton (1991): "Object Data Management: Object-Oriented and Extended Relational Database Systems"; Reading (MA): Addison-Wesley.
• Chen, P.P.S (1976): "The Entity Relationship Model: Towards a Unified View of Data"; In: ACM Trans. on Database Systems; Vol. 1; No. 1, pp. 9-36; Association for Computing Machinery.
• Codd, E. F. (1970): "A Relational Model of Data for Large Shared Data Banks"; In: Communications of the ACM; Vol. 13; No. 1; pp. 377-387; Baltimore (MD): Association for Computing Machinery.
• Date, C. J.; Darwen, Hugh (1992): "Relational Database Writings 1989-1991"; Healdsburg (CA): Addison-Wesley.
• Date, C. J.; Darwen, Hugh (1996): "A Guide to the SQL Standard"; 4^th Edition; Healdsburg (CA): Addison-Wesley Longman.
• Eisenmenger, Richard (1997): "Mein Liebling Tamagotchi"; Haar bei München: Markt & Technik Buch- und Software-Verlag.
• Geppert, Andreas (1997): "Objektorientierte Datenbanksysteme – Ein Praktikum"; Heidelberg: dpunkt – Verlag für digitale Technologie.
• Gogolla, Martin (1994): "An Extended Entity-Relationship Model – Fundamentals and Pragmatics"; Berlin, Heidelberg: Springer-Verlag.
• Holzinger, Andreas (1997a): "Lernen im computerunterstützten Mathematikunterricht"; Diplomarbeit an der Universität Graz.
• Holzinger, Andreas (1997b): "Entwicklung des Problemlöseverhaltens im Physikunterricht: Eine Quer- und Längsschnittuntersuchung aus dem Bereich der Wärmelehre."; Dissertation an der Universität Graz.
• Holzinger, Andreas (1997c): "Computer-Aided Mathematics Instruction with Mathematica 3.0"; In: Mathematica in Education and Research; Vol. 6; No. 4; pp. 37-40; Santa Clara (CA): Telos-Springer.
• Holzinger, Andreas (1998): "Multimedia und Internet im Mathematikunterricht: Die Neuen Medien im alltäglichen Einsatz in der Schule"; In: Computer und Unterricht 31/1998: Informationsgesellschaft; pp. 48-49; Velber: Erhard Friedrich Verlag & Klett.
• Holzinger, Andreas; Maurer, Hermann (1999): "Incidental Learning, Motivation and the Tamagotchi Effect‘: VR-Friends, chances for new ways of learning with computers"; Oral presentation at CAL99: Virtuality in Education, London, UK, 28^th-31^st of March 1999 at the Institute of Education, University of London.

Online at http://www-ang.kfunigraz.ac.at/~holzinge/vrfriends/vrf_at_cal_99/index.htm

• Hughes, John G. (1992): "Objektorientierte Datenbanken"; München, Wien: Carl Hanser Verlag; London: Prentice-Hall International.

(English original edition: Hughes, John G. (1991): "Object-oriented databases"; Hertfordshire (England): Prentice-Hall International.)

• Maurer, Hermann (1997): "On Two Aspects of Improving Web-Based Training"; In: Journal of Universal Computer Science; Vol. 3; No. 10; pp. 1126-1132; New York: Springer.

Online at http://www.iicm.edu/jucs_3_10/on_two_aspects_of/paper.html

• Paul, Bert (1997): "Das Original Tamagotchi Fan-Buch"; Königswinter (Germany): Tandem Verlag.
• Postel, Jonathan B. (1981a): "Internet Protocol - DARPA Internet Program Protocol Specification"; In: Request for Comments: 791; Marina del Rey (CA): University of Southern California / Information Sciences.
• Postel, Jonathan B. (1981b): "Transmission Control Protocol - DARPA Internet Program Protocol Specification"; In: Request for Comments: 793; Marina del Rey (CA): University of Southern California / Information Sciences.
• Sayles, Jonathan (1989): "SQL Spoken Here"; Wellesley (MA): QED Information Sciences.
• Schwinn, Hans (1992): "Relationale Datenbanksysteme"; München, Wien: Hanser Verlag.

 

Part II: General References

• Doberenz, Walter (1996): "Java"; In: Hanser Programmier Praxis; München (Germany): Carl Hanser Verlag München Wien.
• Eisenreich, Günther; Sube, Ralf (1987): "Dictionary of Mathematics: English-German"; Thun; Frankfurt/Main: Harri Deutsch.
• Hobbs, Ashton (1997): "Database Programming with JDBC"; 1^st Edition; Indianapolis (IN): Sams.net Publishing.
• Joshi, Daniel I.; Runolfson, Rodney; Chandak, Ramesh (1997): "JDBC SQL API"; 1^st Edition; North Carolina: Ventana Communications Group.
• Maurer, Hermann (1996): "Hyperwave™ - The Next Generation Web Solution"; Edinburgh Gate (England): Addison Wesley Longman.
• Niemeyer, Patrick; Peck, Joshua (1996): "Exploring Java"; 1^st Edition; Sebastopol (CA): O’Reilly & Associates.
• Vanderburg, Glenn L. (1996): "Tricks of the Java programming gurus"; 1^st Edition; Indianapolis (IN): Sams.net Publishing.

 

Appendix B: Index