[home]

This page: http://www.pms.informatik.uni-muenchen.de/publikationen/projektarbeiten/Holger.Wagner/projectThesis.shtml
Holger T. Wagner, March, 2002

Lehr- und Forschungseinheit für Programmier- und Modellierungssprachen,
Institut für Informatik der Ludwigs- Maximilians-Universität München

Tracking the Navigation Behavior of Web Communities

Holger T. Wagner

Abstract

Keywords

Web usage, collaboration, navigation behavior

Table of Contents

1 Introduction

The World Wide Web is one of the most used services of the Internet. It is used by a variety of different communities to meet all kinds of information needs: Students, for example, use the Web to find out more about their university, certain departments, or particular lectures and projects. Another example in the academic context are teams involved in research that use the Web to find out about related work. Companies employ the Web to find out more about their competitors; some individuals utilize it to keep up to date with the latest news, according to their interests.

In recent years, the Internet has also become a major marketplace, and these days not only information can be found, but also all kinds of products can be purchased. Individuals look for books, CDs, DVDs and other kinds of consumer goods, because it is convenient and fast. Departments of producing companies find the cheapest provider of components they need to build their own products.

However, little is known about how all these communities actually navigate the Web to meet their various goals. In fact, even the individual members of the communities are often not aware of their own browsing behavior or even the pages they have visited. In GVU's Tenth WWW User Survey [GTRC1998] , 27.6% of the respondents found organizing their gathered information to be the biggest problem; 30.0%, finding known info; and 16.6%, returning to pages they had previously visited. From these figures about individuals, it is easily deduced that there is little to no awareness within a community, e.g. a software development team, about the project-related information the other community members have collected. While privacy issues are a major concern that needs to be dealt with by a system that increases this kind of awareness, teams can improve their collaborative efforts by sharing the relevant pages they have found, filtering out irrelevant pages or analyzing the success of their searching-techniques. It is quite helpful to find out that the paper or API documentation one has just discovered has already been read by a colleague. Instead of working through the document, that colleague is asked about it and the required information is gathered in a personal or mediated conversation about the document. This is an enhancement of the workflow concerning information retrieval, which should not only increase the productivity of a team, but also help building a team spirit - or in a more general scenario, a community spirit.

Furthermore, recommendations to Web pages by someone are already an important reason for people to visit new pages ( [Tauscher] ), and making such recommendations both easier to give and easier to use would help making browsing the Web a more rewarding experience.

From the technological perspective, providing means to collect information on browsing behavior of communities in order to share their efforts is not much different from returning such information to the people who create the content. The information must be well protected and the feedback may only occur in the case of user's consent - however, in certain environments this can be another interesting use case for a software system in this field. Note that the possibilities of a system that tracks all of a user's behavior go much further than the usual server-side log file analysis. In particular, the path that lead a user to the own pages as well as towards where a user left and what he did then may be quite interesting.

One such environment is teaching at universities, in particular the lecture related content that is offered to the students by the teachers. With the gathered information on its usage, the authors cannot only improve the structure and content of the pages - but also obtain feedback on how the offer is used and possibly even change the style of the lecture, if necessary. For example, if many students reading lecture notes look for related information on a particular section of those lecture notes, this may indicate that this part lacks details required for understanding. If this lack is intentional, the feedback shows that the objective has been met - if not, the lecture notes may be improved by adding the relevant details and the subject may be raised again during a following lecture. If used wisely, the lecturing quality can be significantly improved and modern techniques can be interwoven smoothly with traditional forms of teaching.

The purpose of this paper is laying out the foundation for a project that deals with these aspects of Web usage and includes the implementation of a system that can be used to support communities' collaborative and individuals' Web usage. In the following section, relevant terms that need clarification are defined. While some new definitions are given, there are also terms taken from related work and put into the context of the present work. Then, an overview of related work that deals with different aspects of the current project is given and it is put into context with the present work. As privacy and security are major concerns in a project like this, these issues are discussed in the subsequent section. Finally, an overview of the features of a prototype that will be implemented as subsequent work is given.

2 Terminology

An effort to clarify the terminology for the broad area of the World-Wide Web has been summarized in [W3CWCA] . For the present work, the following terms may be relevant: resource [http://www.w3.org/1999/05/WCA-terms/#Resource], link [http://www.w3.org/1999/05/WCA-terms/#Link], anchor [http://www.w3.org/1999/05/WCA-terms/#Anchor], client [http://www.w3.org/1999/05/WCA-terms/#Client], server [http://www.w3.org/1999/05/WCA-terms/#Server], proxy [http://www.w3.org/1999/05/WCA-terms/#Proxy], user [http://www.w3.org/1999/05/WCA-terms/#User], publisher [http://www.w3.org/1999/05/WCA-terms/#Publisher], Web core [http://www.w3.org/1999/05/WCA-terms/#Core], Web resource [http://www.w3.org/1999/05/WCA-terms/#Resource2], Web client [http://www.w3.org/1999/05/WCA-terms/#Client1], user session [http://www.w3.org/1999/05/WCA-terms/#User1], episode [http://www.w3.org/1999/05/WCA-terms/#Episode], server session [http://www.w3.org/1999/05/WCA-terms/#Server1], cookie [http://www.w3.org/1999/05/WCA-terms/#Cookie], Web page [http://www.w3.org/1999/05/WCA-terms/#page], page view [http://www.w3.org/1999/05/WCA-terms/#Page], host page [http://www.w3.org/1999/05/WCA-terms/#Home], Web site [http://www.w3.org/1999/05/WCA-terms/#site], independent Web page [http://www.w3.org/1999/05/WCA-terms/#Independen], Web site publisher [http://www.w3.org/1999/05/WCA-terms/#site1], subsite [http://www.w3.org/1999/05/WCA-terms/#Subsite], Web collection [http://www.w3.org/1999/05/WCA-terms/#Collection].

A Web user community is a group of Web users that either have "something" in common or explicitely are members of a particular group. A more in-depth discussion of this term and what exactly such community members may have in common is subject of subsequent work. Note that in the related work introduced in section 3.4 , a Web Community is defined as a set of related Web pages and has no direct relation to the users who view those pages. In the present work, the term Web content community is used for the latter type of Web communities to draw a clear line between the social user communities and the more technical view on communities resulting from considering Web pages.

Navigation behavior is how particular users or a group of users navigate through the Web. Navigation behavior is an artefact of individuals' or communities' Web usage over time. Two studies that try to analyze individual user's navigation behavior are introduced in section 3.1 . A basic concept of navigation behavior can also be found in section 5.2 , where the terms navigation events and browsing state are defined. Consecutive work will discuss navigation behavior more in-depth.

The Web graph is the directed graph consisting of Web pages (nodes) and links between them (directed edges) [Kleinberg99] . The traversal of this graph performed by users is called browsing . While users are browsing the Web, they leave behind individual trails that can be accumulated "globally" or for a specific group of users to (community) paths . The Web graph can also be seen as some sort of space populated by users, which may be a useful metaphor to support collaborative browsing. Note that individual trails and (community) paths can be represented as subgraphs of the Web graph, but other representations (e.g. sequences) are also possible.

[Cheung] defines a Web tool as a software tool that helps users to retrieve, locate and manage Web documents. They classify Web Tools into five levels:

The objective of this work is laying out the foundation for a collaborative Web tool , which is at least a level 3 Web tool that additionally supports the collaboration between its users. Note that most of the examples given in section 3.3 are in some ways collaborative as they use matching of different user's profiles for their recommendations. The distinction between such recommender systems and collaborative Web tools is that the former use collaboration implicitely without necessarily letting the user even notice it. The latter, however, should provide means for users with similar interests to explicitely collaborate, e.g. by sharing the information they find on a particular search task or making annotations to a particular document available to others.

In the features of the prototype discussed in section 5 , the attempt to infer the user's profile is not made - instead the system only tracks the user behavior and he must explicitely mark his interests by bookmarking pages. While the concepts are quite interesting to discuss, the actual implementation is beyond the scope of this work. However, a design goal of the prototype is easy extensibility with features like this and subsequent work may integrate the implementation of level 3 Web tool features.

[Twidale] introduces a few interesting terms concerning (collaborative) browsing behavior:

Tactics for searching information include consulting , which is described as asking a colleague for help but may also be used when strangers are asked for assistance. This has the advantage that the references are already filtered according to the taste of the consulted person. To bibble , is to use other searchers results, e.g. published in the form of a bibliography for one's own search. However,

“The results of most searches are not published as bibliographies but are private, local and temporary and consequently, from the perpective of future users, the information is lost. This means that the great majority of searches that are conducted fail to bibble properly; they fail to take advantage of previous results because there is no mechanism to support the sharing of this information.” ([Twidale])

In an informal study conducted at the Lancaster University Library (referred to by [Twidale] ) the following collaborative interactions have been observed (which are considered relevant to Web searching): Joint search: small (2-4) groups of students working on a single terminal, involving frequent pointing at the terminal screen. Coordinated search: a group where each participant works on his own terminal, sometimes competing to find the information and sometimes clustering around terminals like in joint searches. Chance contact occurs when people happen to use the same resource and thus get in contact.

“ Group searching takes place when two or more people share a common aim, and choose to coordinate their searching efforts.” ([Twidale])

Differentiated group searching expresses that the group members work in the same area, but their specific searching aims are different.

Serendipitous altruism is used to describe the fact that

“colleagues in a community may be willing to help each other's information searching even if they are not directly involved in the project.” ([Twidale])

“If your colleagues know what you are working on and happen by accident, in the process of undertaking their own searches, to come across something that may be of interest, they may altruistically pass the information to you.” ([Twidale])

As the cost for such help must be minimal for the help to be given, a tool for collaborative searching should support serendipitous altruism sufficiently.

A distinction is made between product-related and progress-related information exchange between people. In product-related information exchange , the search results are discussed, while progress-related information exchange deals with the process of searching, e.g. how to find certain types of information.

3 Existing Approaches: Web Analysis and Browsing Helper Systems

There is a large body of literature that deals with different aspects of the Web which are relevant to the present work. The following sections are an attempt to classify that literature and provide the background to this project.

3.1 Analysis of Browsing Behavior

In this section, some approaches of tracking and analyzing the navigation behavior of Web users are introduced and their results outlined. While section 3.2 presents approaches that analyze server logfiles, this section is dedicated to client-side tracking of Web usage and the analysis of the collected data. The title Analysis of Browsing Behavior may seem applicable to both client- and server-side based approaches. However, server-side based approaches have several limitations so that the general term browsing behavior is reserved for client-side tracking in this work, while Web usage mining as a more special term is used for server-side tracking. The body of existing work introduced in this section is very small compared to the large field of Web usage mining - in fact, only two studies have been found that analyze and elaborate upon the data collected on the navigation behavior by tracking users at the clients.

However, there has been research on user strategies and usability of closed hypermedia systems preceding the WWW, which is beyond the scope of the current work, but provided a basis for [Catledge] . In their work, a modified version of NCSA's XMosaic Web browser is used to capture all user interface level events of 107 users in an experiment lasting three weeks. While there are many other types of user events also included in the study (some specific to XMosaic, e.g. Reload Configuration Files, or Delay Image Loading On/Off), the most important navigation-related user events are following a hyperlink (52%) and the back command (41%). Much less often used are opening a manually entered URL , using the hotlist ( hotlist is XMosaic's name for bookmarks) and the forward command (2% each). Opening local files (0.7%), going to the home document (0.5%) and using the history list (0.1%) are found to be the least used features. One possible explanation given for the minimal usage of history and bookmarks is the design of the interfaces to these functions.

While XMosaic does provide a bookmark feature in its interface, many users tended to also use “home pages as indexes to interesting places” ([Catledge]) , which provide a similar functionality as bookmarks but better layout control and customization.

A finding on a more abstract level is that users tend to navigate within a small area of particular sites, the individual trails resembling a spoke and hub structure (when using a graph structure where using the back command results in going back to the previous node instead of moving to a new node within a sequence).

Directions for the design of Web sites concluded from the results are that the most important information must be accessible within two to three hyperlinks of the initial home page. Different types of users are identified ("Serendipitous Browser", "General Purpose Browser" and "Searcher", taken from [Cove] ) and offering different views of the pages for these different types of users is suggested.

An approach more focussed on users' revisitation patterns and their implications on the design of revisitation tools in browsers has been taken by [Tauscher] . The design of current browsers' history mechanisms is explicitely criticized and an objective of the work is to motivate improved interface designs revisitation tools (within browsers or external, see section 3.5 for examples).

As in [Catledge] , a modified version of XMosaic is used to track the usage data. In fact, the modifications of the earlier study are used as a basis for the latter one, but a smaller set of actions are captured. A distinction is made between navigation and non-navigation actions, where hotlist management (add/delete/edit hotlist entry) belongs to the latter category. There is only one action open URL (making up 50% of the actions executed in the experiments) for the following methods of opening a new page:

• Anchor: using a hyperlink (82,7% of open URL)
• Keyboard: typing a URL into the URL field (6,8% of open URL)
• Hotlist: selecting a page from the hotlist (5,0% of open URL)
• Dialog: using the Open URL dialog (2,0% of open URL)
• History: selecting a page from the window history dialog (1,3% of open URL)
• Other: less frequent methods such as causing a page to display with an external application (2,2% of open URL)

Just as in the previous study, the back command is used frequently (30%), and other actions are used seldom (e.g. home (5%), forward (0.8%), new window (0.8%) and open local (0.2%)). It is not clear, why home, back and forward are not included in open URL, as they all result in displaying a new URL. Possibly, this is done because the URL is not selected but taken from stored data the user can not directly access, as in the other actions subsumed under open URL.

A very interesting finding of [Tauscher] is that the same pages are revisited very often, with a recurrence rate of 58%. The recurrence rate is defined as “the probability that any URL visited is a repeat of a previous visit” ([Tauscher]) . With the data of [Catledge] that has been reanalyzed in the study of [Tauscher] , the rate was even higher at 61%. The conclusion from that fact is that browser interfaces should help users revisiting pages - a few approaches are introduced in section 3.5 .

Even though the recurrence rate is very high, many pages are visited only once (60%) or twice (19%), and many of the visited pages are entirely new (40%). Furthermore, while the major contribution to the high recurrence rate are the last few pages visited (by using the back command), 15% of recurrences are not within a list of the last 10 URLs visited.

Finally, the little acceptance of current history facilities is explained with limitations of the interfaces. In particular, the effort for managing hotlists is considered a problem. Furthermore, histories are usually not easy enough to access and should be integrated better into the browser's user interface.

While studies analyzing browsing behavior as and end in itself are rare, the browsing behavior of users is used e.g. in recommender systems as introduced in section 3.3 . As an example, [Goecks] has been chosen. The basic idea is that a user's interests may be deduced from certain aspects of his browsing behavior, which allows agents giving the user recommendations of potentially interesting pages based on his usage profile (see section 3.3 for further information on recommender systems). An innovation of [Goecks] is that mouse and scrolling activity are added as parameters of the user's navigation behavior.

To obtain this information, an agent using Microsoft Internet Explorer 4.0 has been implemented. The agent captures information like the number of hyperlinks clicked on a page, the amount of scrolling the user performed, and whether the user bookmarked the page. No results comparable to those in the previously reviewed studies are available, as the objective of the work was not finding out about the navigation behavior, but using the navigation behavior as input for algorithms analyzing the user's interests.

For the current project, the architectures used for collecting information on users' navigation behavior are quite interesting. Furthermore, the results of the studies concerning the usage of single-user browsers indicate which functions may be necessary for systems supporting collaborative Web usage. While improvements for existing revisitation functions are suggested here, examples are subject of section 3.5 .

3.2 Web Usage Mining

The term Web usage mining has been suggested by [Cooley97] , as opposed to Web content mining , as a specific variation of Web mining . There, it is defined as “the automatic discovery of user access patterns from Web servers” ([Cooley97]) , and in fact all of the work in this category deals with data from Web server logfiles - alternative architectures for capturing the usage data are only theoretically discussed in the survey of [Srivastava] , but according to the author's knowledge not used in practice in this field.

The major objective of the work introduced in this section is to provide data for content providers so that they better understand their customer's use of their content. In that, the restriction to server logfiles - which prohibits logging the complete path of a user over multiple websites or gathering specific information about the navigation behavior (see section 3.1 ) - does not play a major role.

However, [Srivastava] , a recent survey of the existing work in that area, broadens the definition of Web usage mining to include any Web data, allowing proxy and client level data collection as well. This makes sense as many of the techniques proposed in the given papers could easily be applied to client level logfiles even if that application may not have been considered by the authors. On the other hand, a large part of the complexity of Web usage mining is based on the challenge of extracting individual user's trails from logfiles of servers of the stateless HTTP, a problem that does not arise when the data is captured at the client.

In [Pitkow] , some of the problems with server-side tracking are discussed and a terminology is suggested: An unidentified user is defined as a user about whom no information is available. This can be the case when Web proxies operate between server and client. The default type of visitor on the World Wide Web is called session visitor : an identifier can be inferred using heuristics based on the information available in server-logfiles, the Web site topology etc., or an identifier is explicitely created using cookies. To a certain extend, the former techniques can be used to even identify users behind firewalls, as it was done in [Pirolli] . A tracked visitor is defined as “a visitor who is uniquely and reliably identifiable across multiple visits to a site.” ([Pitkow]) This seemingly can be achieved with long-term cookies. However, it should be added that this does not work when visitors use different browsers on the same machine / user account or different machines / user accounts. Finally, an identified visitor extends the tracked visitor with additional information. To a certain extend, such information can be automatically gathered from other sources - however, the common way is asking the users for that information. Either way the reliability of such information is very questionable unless the user profits of giving valid information.

A major problem with server-side logfiles are the various levels of caching because it distorts the data significantly. If proxies and browsers cooperate, this can be circumvented by a method called cache-busting - this is tried by using HTTP headers indicating that the page should not be cached. If browsers or proxies ignore the relevant headers, cache-busting via HTTP headers fails. A more brutal approach that always works is adding a random dummy parameter to the URL, which causes the browser's or proxy's URL matching to fail and thus inhibits caching. Such techniques are questionable, however, as they interfere significantly with how the Web is supposed to work! After all, there are good reasons for caching and inhibiting this just to get better usage data (which raises privacy concerns in itself) calls for criticism.

For the present paper, Web usage mining is interesting because it provides some discussion and formal models for the individual trails users leave behind while browsing the Web as well as some discussion on how usage data can be gathered. Even though most papers of that field deal explicitely with server logfiles, many of the techniques could be adapted to client-side logging, usually in a simplified manner as some of the issues complicating the extraction of valid usage data from server-side logfiles inherently do not exist when using client-side logging.

3.3 Recommender Systems

Recommender systems are tools that recommend Web pages to a user that shall be interesting to that user. While [Terveen] includes recommendation support systems in their broad survey, where the recommendation process is not automated but instead users who want to share recommendations are supported, this section only includes systems that automatically compute the recommendations. Recommendation support systems are instead subject of section 3.6 . The data used to compute recommendations can either be of a single user only, or of a community of users. While the latter implies some sort of collaboration, the focus is on the recommendations, and how those recommendations are computed is usually not visible to the user of the system - this draws another line between recommender systems and collaborative Web usage as described in section 3.6 .

[Terveen] presents a general framework for understanding recommender systems, including what is termed collaborative Web usage in this section. They define content-based systems as using only the preferences of the seeker and attempting to give recommendations based on similarity to items previously liked by that seeker. Content-based systems focus on learning the user's preferences and filtering new items according to those preferences. Examples of content-based systems are [Armstrong1997] , [Cheung] , [Goecks] .

Systems that apply collaborative filtering on the other hand, employ the ratings of other users and try to match those new items that other users with similar preferences have liked. Thus, the recommendation process is completely content-independent. Such systems focus on algorithms that discover similarities between user preferences to match people for gathering the recommendations. Examples of systems using collaborative filtering include [Pazzani] , [Rafter] , [Resnick1994] , and [Wasfi] .

Collaborative filtering has been extended significantly by [Chalmers98] , by introducing the path model . To capture the context in which a particular information item is used, instead of using only single items, the paths of users (e.g. trails of users on the World Wide Web) are used to build both user profiles and recommendations based on these profiles.

[Claypool] introduces a few problems with pure collaborative filtering: The early rater problem occurs with new items, that haven't been rated by any users. The same applies to new users, that have no profile which can be matched. The worst case of the early rater problem are new systems, where neither users, nor items have any ratings to compute recommendations from.

The sparsity problem plays a role in information domains where the number of items exceeds what individuals can absorb and rate. As this results in sparse matrices containing the ratings of all items for all users, recommendations are hard to compute from these sparse matrices.

Finally, gray sheep are people who do not consistently agree or disagree with any group of people. Gray sheep do not benefit from pure collaborative filtering systems as the system cannot judge their interests appropriately.

Pure content-based systems are criticized as having “difficulty in distinguishing between high-quality and low-quality information that is on the same topic.” ([Claypool]) With an increased number of items in general and for specific topics, this problem gets even worse and the quality of content-based recommendations is reduced.

To solve these problems of pure collaborative and content-based filtering systems, a combination of both is suggested and an extensible architecture introduced. [Pazzani99] further extends this idea by including demographic information into the filtering process, and shows that the quality of recommendations is actually improved by using the combination.

Other work on recommender systems includes [Liebermann] and [Maglio1997] . Both try to obtain a model of how the user searches the Web and give suggestions based on this model.

For the ongoing project, an integration of automatic recommender system technology is a promising idea. While the main objective is helping people collaborate explicitely and provide an increased awareness of other people, the collected data can be used as input for any combination of the introduced techniques of automatically recommending interesting pages. Ideally, the recommendations are explained to the user, as suggested in [Herlocker] . This can further enhance the awareness of the community one browses the Web with.

Another very interesting aspect of recommender systems in respect to the current work is that they usually recognize communities based on the various types of user profiles. While the pure recommender systems need those communities to base their recommendations upon, the communities can also be used to make people with similar interests meet each other. This idea is discussed by [Terveen] , including some of the privacy issues involved therein. Furthmore, such explicit communities based on user profiles may even be used to evaluate the quality of the community by asking its members whether they feel the community shares their interests or not. The privacy issues of such a system must be carefully weighted against the potential benefit for the users, ideally in a way that puts the freedom of choice to the user himself.

3.4 Hyperlink and Content Analysis

The body of work introduced in this section deals with analyzing the static structure of the Web defined by hyperlinks and/or content to find out relationships between pages and group pages into clusters, called Web (content) communities. Notice that content analysis is only introduced in connection with hyperlink analysis here. While content analysis surely is a very large field as well, it has been left out for the sake of brevity and may be included in subsequent work. Hyperlink analysis is usually a static approach that does not take into account user behavior. A recent survey of the work in this field and some terminology is given by [Efe] . The simplest and most obvious form of page A implicitely endorsing page B is a direct link from A to B. When a page A links to two other pages B and C that is called co-citation and it is assumed that B and C have some relevance to each other as well as to A ( [Efe] ). Another measure, also taken from bibliometrics (see below), is bibliographic coupling : the more links two pages A and B have in common, the higher their bibliographic coupling and thus, a higher similarity or relevance to each other is assumed ( [Kleinberg] ).

One finding of hyperlink analysis is that Web pages can be categorized into authorities and hubs: authorities are considered the best sources of information on a particular topic and hubs are collections of links to those locations (e.g. [Chakrabarti] , [Kleinberg] , [Gibson1998b] ). Discovering these pages is not a trivial task, and much of the work tries to find algorithms that efficiently handle this task. For examples, see [Dean] , [Flake] , [Gibson] , [Kleinberg] .

Another interesting link topology is that of a web ring : a set of related pages that link to each other one after the other. Each page n links to a previous page n-1 which in turn links to n, and a subsequent page n+1 which links back to n. Web rings are discovered e.g. by the method of [Flake] .

According to [Gibson] , "link structures have been studied in hypertext research that predates the www", for example in [Botafogo] . A related field are bibliometrics , in which the patterns of citation among scientific papers is studied. A review can be found in [White] . Some of the connections between bibliometrics and hyperlink analysis are studied in [Larson] . A few important differences between scientific citations and Web links are ( [Efe] ):

• In scientific citations relevance, objectivity and information quality can be expected. Web links are usually more subjective and noisy.
• Web links also serve navigational purposes, while scientific citations always have (at least some) relevance to the content.
• Web links are dynamic, scientific citations are static. In particular, Web pages often mutually link each other - a phenomenon very rare to scientific work.

For an example where content and hyperlink analysis is combined, see [Davison2000] . While other approaches only include the topology of the links, here the text in, and around the links is used - assuming that it somehow describes the pages linked to. In the experiment it is shown that the text within the anchors often represents at least a part of the target page.

[Pirolli] attempts to improve Web navigation and assimilation by integrating hyperlink topology, page meta-information (like file size and URL), usage frequency and usage paths as well as text similarity between the pages. They have also defined a set of types of Web pages according to their roles:

• head page : best starting point for a set of pages. There are two subclasses of head page: personal home pages and organizational home pages .
• index page : basically the same as hubs, can often be identified by words like "index", "table of contents" or "toc" as part of their URL or title.
• source index page : entry points and indices into a related information space. Quite similar to index page, but these are also head pages.
• reference page s: pages that are used repeatedly to explain a concept or contains actual references. The subclass destination page is used for pages that do not point anywhere else, e.g. expanded acronyms, copyright notices, and bibliographic references.
• content page s serve no navigational purposes but only deliver information.

While previous work concentrated mostly on the communities in themselves, [Toyoda] is also concerned with the relationships between those communities and a way of navigating between related communities. To that end, they have developed a technique for creating a community chart, which is a graph of which the nodes are communities and the the edges relationships between those communities. The edges are weighted, the weight representing the strength of the relationship.

The major objective of this approach is improving the way the Web can be searched, organized and visualized. Another application of the results of this work is more specific targeting of advertisements. If the communities of which the visitors may be interested in certain products are known, the most authoritative pages can be used for effective advertising ( [Efe] ). Last but not least, finding out about the social and/or intellectual structure of the Web is an end in itself.

In the context of this paper, the results of research dealing with hyperlink and/or content analysis may be valuable to define groups of documents that people look for information at. A user may then communicate with users currently visiting pages from the same group (a Web user community based on a Web content community) which may make it much easier to find the most interesting information by simply asking others. Hyperlink analysis may be extended by using the links actually followed by users instead of all links, and possibly even using navigation behavior information like how much time is spent with a page to improve the quality and relevance of the clusters. Intuitively, a page that a user returns to many times and from which he then visits other pages may be a good hub for the topic the user is currently interested in (see section 3.1 ). A page visited from such a hub that the user spends a lot of time with, possibly bookmarks it or saves it locally is probably a good authority.

3.5 Revisitation and Annotation Tools

Work dealing with the creation and integration of user interfaces for revisitation and annotations tools includes [Barret] , [Cockburn99a] , [Cockburn99b] , [Hascoet1999] , [Hascoet] , [Kaasten] , [Koch] , [Laurent] , [Li] , and [Tauscher] . In [Hascoet1999] , an attempt is made to integrate a short term history, a personal best of list, a list of unclassified documents to be read later, and an overview of an organized collection of bookmarks into a unified user interface. The model used for this integration, termed "document as user interface" by the authors, can also be used for navigation. While most browsers show bookmarks in a simple tree, BookMap uses a fisheye view that allows zooming in to and out of areas of interest, trading details for context. Another improvement to the handling of bookmarks is filtering - a technique also used by [Kaasten] and [Li] . While the keyword filter is quite simple, a special approach has been developed for filtering by date: instead of entering the dates manually, a slider is used that consumes minimal screen estate (see below). The length of the cursor represents the length of the period, and the position of the cursor represents the period itself.

[Kaasten] deals with an integrated model for "back", history and bookmarks, based on a recency-ordered history list, in order to improve the usability. The "back-button" in current Web browsers is usually implemented as a stack , leading to problems as going back and then branching to another page destroys the old branch. A recency-based history , on the other hand, simply records the pages visited in the time-based sequence they are visited. A recency-based list not only avoids this problem but is also considered more intuitive to the users. While conventional bookmarks provide useful means of structuring the collection, this is considered "heavyweight" in the paper and a solution where bookmarks are replaced with "dogears" on the list of visited pages is proposed. Like in [Hascoet] , pages are represented via thumbnails, as this has been proven to be more effective than Web page titles or the URLs of the pages ( [Cockburn99a] , [Cockburn99b] ). Implicit bookmarks are somewhat similar to the best of list in the above work: by visualizing the page visit frequency a user can easily distinguish between pages that have been visited more or less often. By filtering, best of and bookmarks only lists can easily be created, as well as a simple form of content-based filtering, using the page's title or showing only pages from particular domains.

PowerBookmarks introduced by [Li] is an information organization, sharing, and management tool. It supports advanced query, classification, and navigation functionalities on bookmark collections and also uses users' access patterns for features like automated bookmarking, document refreshing, and bookmark expiration. For example, when a user visits a Web page frequently, it can automatically be bookmarked.

A major problem with revisitation tools is the "screen real estate" ( [Cockburn99b] ): as the Web pages the user actually wants to see usually require a lot of space on screen, revisitation tools compete with that space. Thus, the more space the tool requires, the more useful it must be for the user so that he does not hide it somewhere and thus stops using it. Therefore, “[r]evisitation tools must [...] maximise the value of the information they present, and do so using minimal screen real estate.” ([Cockburn99a])

[Cockburn99a] also discusses various approaches to the structural organization of page display:

• Hub-and-spoke dynamic trees capture the user's navigation behavior well, by adding new pages, with edges to the pages they are linked from.
• With Spatial Layouts the user can arrange the pages according to his taste, making it easier to remember a page's content by where it has been placed. This approach has a major disadvantage, namely that placing the pages is a heavy burden to the user.
• Site maps contain the complete contents of a particular site. These can be statically arranged which may make navigation easier - however, many included pages have usually not been visited before. Thus, instead of using this for revisitation it is probably more useful for finding new pages...
• Temporal organisation is another example of a natural way of representation and facilitates finding previously visited pages by exploiting the user's memory of when he visited a certain page.

There have been various approaches to annotating the WWW, some of which shall be introduced here. One major design issue with annotation systems is how the annotations are gathered, stored and presented. There are generally two classes of systems: systems that require software installation or configuration changes on the client-side (e.g. [Kahan] , [Laurent] , [Marais] ), and systems that use standard internet technology like javascript to embed the functionality in standard Web browsers (e.g. [Koch] ). The latter, however, usually requires changes on the Web server or the documents it provides, restricting annotations to pages that are prepared for taking annotations. An alternative is installing and using a proxy-server or similar architecture, where the original pages are rewritten to include the annotations. This approach has been used, but this is not covered here, see [Laurent] instead.

[Koch] discusses the use of an annotation tool in academic courses. In such an environment, the need of enhancing the documents is not a problem as most relevant documents are usually accessible and can be easily changed.

Yawas, the prototype introduced in [Laurent] is a Java and JavaScript based annotation tool that is implemented as a client-side proxy. It works with any Web browser due to its architecture and allows annotating both remote and local documents. Specific texts within Web pages can be highlighted and annotated and those annotations are stored locally, which circumvents privacy concerns. Sharing annotations is possible via import and export functions.

A very promising project is Annotea ( [Kahan] ), a W3C LEAD project for enhancing the W3C collaboration environment with annotations. For editing and viewing the annotations, which are stored on special purpose servers, an own Web client is available (Amaya). However, there are also add-ons for existing browsers including Internet Explorer and Mozilla.

A major goal of Annotea is to re-use as much existing W3C technology as possible - consequently, open standards like RDF, XPointer, XLink and HTTP are used extensively. This simplifies extending Annotea and interoperating with other annotation systems. Another interesting aspect of Annotea is that annotations are typed with types defined by the users, allowing classification of annotations into classes like comment, erratum etc.

While other approaches have a particular user interface included, Annotea is user interface independent. Clients can be implemented based on the standard protocols defined by the Annotea project.

Finally, privacy and scalability concerns are circumvented by using multiple decentral annotation servers instead of a single server. This both allows collaboration among multiple users or even user groups (unlike client-side storage) and at the same time assures that the groups using a server can keep their information private.

While annotation tools often are targetted at collaborative work, revisitation tools are usually single-user oriented. Privacy issues pose a major challenge when such information is used for collaboration, but especially small, limited groups where all participants know each other profit heavily from an integrative and collaborative approach to revisitation and annotation in the Web context. While challenging, finding a well-integrated solution for providing such services to a community may significantly change the way the Web is used. Obviously, such an approach should be based on and extend the models used for single-user revisitation and annotation tools.

3.6 Collaborative Web Usage

A good starting point to find out why a tool for collaborative Web usage is needed is [Twidale] . It draws from some findings on how conventional libraries are used by students - namely, often in a collaborative manner - and these findings can be transferred to World-Wide Web usage. One interesting idea in this work is that not only information, but also people are considered an important thing one can search for:

“We believe that browsing for people, their electronic representations or representations of their activities, is a neglected and important area.” ([])

For a digital library that allows collaboration, the authors propose the following communication aspects:

• direct information exchange between individuals: "do you know?"
• direct information exchange between an individual and a group: "does anyone know?"
• Searching for experts: "who might know?". This can be implemented by profile matching (see the coherence with section 3.3 ).
• Coordinated searching, which involves a lot of communication between participating users during the actual search.
• Making contacts. "For example, if a record is kept of books borrowed, or electronic documents inspected, then two users with overlapping 'browsing habits' might be put in contact. Alternatively, if the system maintains 'interest profiles' for many of its users, these could be grouped together using a clustering algorithm." (As proposed in section 3.3 .)
• Personal recommendation: In this case, individuals are notified of the search results. Group recommendation is basically the same except that a whole group is notified. In similar searches, this activity is done automatically, by discovering people who may be interested in the information via heuristics like their browsing behavior and passing the information on to them (see serendipitous altruism... [#Def_SerendipitousAltruism]).

[Marais] define cooperative surfing as activity of a community of users who cooperatively and asynchronously build up knowledge structures relevant to their group. They discuss design options and describe their own approach Vistabar that supports this activity. The options given include custom browser, browser plug-in, applet, parasite and proxy. A parasite is defined as “an application that attaches itself to another executing application and is able to monitor and control it through a published API.” ([Marais]) These are analyzed according to the following criteria: control over browser, monitoring, persistent presence, own UI, UI integration and extensibility. In their discussion, the two most promising options were proxy and parasite, but as proxies lacked some features they required (lack of control because of caching, browser display cannot be driven etc.), the parasite approach was chosen. Their tool, for which they have coined the term browserware which stands for software components that are both aware of the browser and the user, supports features like a searchable index on all visited pages (based on the NI2 library which is also used by AltaVista), finding similar (related) pages, classifying pages, finding referring pages and associations to real world items via barcodes.

A feature that may be particularly interesting for determining which sections of a larger document a user is interested in is also explained: zipping. This is done by determining the sections and subsections via their tags (H1, H2, etc.) and then allowing the user to collapse or expand those sections.

For cooperative surfing in the context of the paper, bookmarks and annotations are supported. An interesting feature concerning bookmarks is that it is possible to store unclassified bookmarks which are automatically classified by the system. Other users may then change the classification if it doesn't fit well.

A more recent, proxy-based approach is discussed in [Cabri] . In that work, synchronous browsing, which includes chat facilities and the like is the main center of attention. An architecture for a proxy-system that supports synchronous browsing is explained after a discussion on the different options: server-, client- or proxy-side. In fact, what is used is a combination of a proxy that also changes the documents it serves and applets that provide the client-side functionality (these are embedded into the original pages by the proxy). The features of the implemented system include user-management, caching pages, modifying pages, informing users which pages other users have retrieved and changing the colors of links that have been followed or that returned errors. An additional feature that may be interesting especially in an academic context is master-slave browsing, which allows one user to have all other users see the pages he selects. This may be also interesting for teams that want to watch each other's sessions simultanuously (of course, it would be a different feature as in this case, one would talk of joining into a session). Finally, images can be wrapped into applets so that they become sort of a shared blackboard, where users can point to areas within the image as well as painting into the image. The performance of the system is shown to be no hindrance to Web browsing.

A broad overview on collaborative Web usage is also given by [Greenberg] . One very interesting finding reported therein is that voice communication is very important for real time collaboration, but has not been implemented by most systems.

[Greenberg] then introduces GroupWeb. GroupWeb is implemented as an own Web browser, which allows some more features at the expense of forcing users to use another browser instead of the browser they are used to. In GroupWeb, master-slave browsing is also supported but here it is even possible to synchronize the scrolling of the page. Furthermore, telepointers allow participants to point to interesting parts of the pages currently displayed. Like in other approaches, group annotations are supported.

In [Dieberger2000] , CoWeb, a collaborative Web space is introduced. It allows people to change the content and create new pages easily. Furthermore, discussions are supported. An interesting feature is that access history is visualized so that users can easily find out when other users have been visiting a page. This increases the community awareness. On the other hand, the architecture - a single Web server - limits the scope of the system significantly.

As the objective of the present work is building an innovative tool for collaborative Web usage, the other approaches must be carefully examined and existing ideas must be integrated with approaches that have not previously been considered for collaborative Web usage. An important question to ask is "what is missing in those approaches?" The objective of finding a solution that integrates approaches - generalizing them - may lead to a system that either cannot be implemented or cannot be used, due to its complexity. Thus, a way must be found so that the integration simplifies instead of making things more complicated.

4 Privacy and Security Issues

As the system is intended to capture a lot of information of and about its users, privacy is a major concern. While a maximum protection of privacy may be an important criterion for many users (see [Pitkow] ), this conflicts with the intention of making the Web more personal and support collaborative Web usage. Thus, the challenge in this issue is balancing the protection of privacy with the display of personal information. One dimension of this is how much data is available about each user to which other users. [Terveen] suggested letting the users progressively reveal more about themselves, while they get to know the fellow users better (this is common practice with dating services).

Another, but more fundamental, dimension is the architecture of the system, which has a major effect on the applicability of privacy concerns. If data is captured and stored on the clients alone, private data stays on private computers and as long as no one gets access to the computer, no privacy problem arises. This approach has been followed e.g. by [Laurent] . However, collaboration can only take place if users exchange their data via other media, e.g. eMail or Web pages - which is cumbersome in this approach. In fact, such a system does not support collaboration by itself, at all.

A better solution is capturing and storing the data at some place that is only accessible by the team involved in collaboration. That way, the team members can only access other team members' data. Of course, the team members must have a trusty relationship. In this scenario, privacy issues do arise - however, it is an environment which is relatively easy to control and find consensus in, about measures against misuse of the data. A disadvantage is that team members can only use the system within the given boundaries.

The most challenging architecture is a system that can be accessed from anywhere on the Internet. This does have some advantages: teams may connect from all over the world, people can use their accounts from all over the world - a lot more people use the system and thus a lot more information is available. Some possible features (e.g. collaborative filtering or synchronous communication with people on the same Web page) only make sense or even only are possible with a very large user base, which can only be attained in such an environment. However, privacy issues become a major concern with that architecture. Not only must it be secured well against hackers which may steal and misuse the data (which is much harder when the system resides behind a firewall). The intended usage is also problematic, as most users will not know anything about the other users.

In [Bellotti] , a very useful design framework is given, which is based on control and feedback . Users should be able to control what information about them becomes available to which other users and when information is being captured, the users should be provided with feedback on this. A system for collaborative Web usage must definitely implement mechanisms that allow its users to control all data that becomes available about them. To a certain extent, forcing a user to explicitely grant other users access rights already provides him with feedback about what others can find out about him. Further feedback (e.g. if someone actually views the available information) is probably not needed unless users forget about their own settings after some time.

5 Features of the teamXweb Prototype

In this section, the features of a system called teamXweb are described. teamXweb is a prototype that is used in experiments to find out about the usability and usage of a system to support collaborative usage of the Web. The architecture and implementation of that prototype is subject of future work.

The original name of the prototype was TeamWeb, but a search for the term TeamWeb on Google [http://www.google.de] returns about 5,000 pages. With the keywords TeamWeb, Web, usage and cooperative respectively collaborative still 7 respectively 12 hits are returned, three of the latter sample are pages of the original project's Website, though. The other hits link to organization's Web teams that are responsible for the organization's Web presence, independent Web design companies, Web sites about Web design. NetObjects has an architecture called NetObjects TeamWeb™ which is used to support collaborative creation of Web sites [NetObjects] .

Thus, the name has been changed to teamXweb, the X indicating that this is meant as the cross-product of team and Web. The new name seems to be unique - at least a search on Google [http://www.google.de] returns no results, which is a very reliable indicator that the term is not used at all, anywhere on the Web. The pronounciation remains the same, however - the X is silent...

5.1 Communities

The key concept for teamXweb are communities. The term community has been chosen instead of group to point out the broader sense in which the term can be used. A more in depth discussion of communities will be part of further work. In the prototype, communities are implemented as simple groups of people, and thus the term is used interchangeably in this section.

Communities are sets of people, e.g. a team working on a particular project. Such groups can be created by users, and other users can join or leave the group at any time. For enhanced security and comfort secret groups are added, which can only be joined if their name is known and are not displayed in the community overview. Furthermore, it is possible to close groups , i.e. make it impossible to join or leave the group for all users. However, the community may still be visible to others. Finally, subgroups are only visible to members of their parent groups. This allows a sort of hierarchy, and in conjunction with the closed groups, a certain flexibility to partition the user base, which is also a useful feature for the experiment.

Another type of community in the prototype are Web site respectively Web page related communities. Such communities exist for each Web site and Web page, and users automatically join and leave these communities when they enter or leave the Web site or Web page in question. Using communities for this allows using all the features available to communities for Web sites and Web pages - in particular communication [#Communication] and community statistics (i.e. who is currently a member, who was a member before).

To support collaboration between community members while at the same time providing a high security for each user's privacy, users can give permissions to each community. This gives them control as it has been discussed in section 4 . As default, none of these permissions are set. It may be useful to allow users changing this default, or - if the rights management gets more complex - choose among different presets for different security levels.

In the prototype, there are user profiles where users can give information about themselves. While users can choose login names that are completely unrelated to their real name and thus have a certain level of anonymity, the atmosphere can be made more personal by using those profiles. However, whether other community members may see that profile or not is the first permission that must explicitely set for each community the user is a member of.

The second permission is whether or not other members may see the user's bookmarks that are described in more detail in section 5.3 . In the bookmarks window, the user can select each community he is a member of, and all bookmarks of all community members that have given that permission will be merged. It is also possible to view the bookmarks of an individual member of a community that has given that permission. In the first prototype, this applies to all bookmarks. However, this is considered a major limitation and in future versions, it should be possible to assign this permission per bookmark category. Thus, users can make their bookmarks available to different communities according to the communities' interests and according to the user's feeling of which bookmarks he wants to keep private and which he wants to be public.

The same applies to the user sessions in the session history (explained in section 5.2 ), which is the third permission that can be set. As all of the navigation behavior of a user is captured in his session history, this is the most sensitive information. Only allowing users to set this permission for all user sessions, or none is an even more severe limitation than with bookmarks. However, the prototype shall be as simple as possible and in the testbed of the experiment, the user base will be small enough and users will be aware enough that this issue can be accepted. Furthermore, it could be worked around by using different users for different browsing tasks.

5.2 Session History

The session history is the list of all user session [#Def_UserSession]s, ordered as a sequence in time. Each user session consists of a list of navigation events and browsing states for each window that has been opened during the session. The browsing states are usually equivalent to the URLs of the viewed pages. However, with framesets this simple approach is insufficient: in that case, a browsing state refers to the URLs in all the frames of the window, and if a single URL (i.e. document) changes, it is a new browsing state. Navigation events are the events with which each browsing state is entered and left. The following navigation events will be available in and captured by the system:

A useful feature could be management of the individual user sessions: each session could have a name, description and attributes like keywords to facilitate finding previous user sessions. A hierarchial categorization of the user sessions may also be useful. This feature becomes particularly interesting in the context of communities, as described in section 5.1 , because a categorization may facilitate offering some user sessions to other community members, while others are kept private or open to another community.

5.3 Bookmarks

This section describes the bookmarks in teamXweb and there are two important relations to mention between bookmarks and the session history: first, user sessions are obviously captured passively while the user browses, unlike bookmarks which must explicitely be set by the user. Second, bookmarks are also browsing states. This latter relation is important because it justifies that the Browsing States of the session history need not be editable in any way, as this can be done by adding them as bookmarks and then editing the bookmark.

This implies that bookmarks can not only be set from the current page, as in most browsers, but also from the session history view. While browsing the session history, users may find certain entries especially interesting and put those entries to the bookmarks. Or he may feel the need to extract a certain Browser State from the session to add additional information - which is only possibly with bookmarks.

Which is the major difference between a bookmark and a Browser State: bookmarks are editable. Just like user sessions, bookmarks can have names, descriptions further attributes, like keywords and be put into a hierarchy of categories. Thus, while bookmarks point to the same information in the Web as Browser States, they are more closely related to user sessions in terms of how the user can archive them. Bookmarks are the smallest editable piece of information in the bookmarks section and user sessions are the smallest editable piece of information in the session history.

Finally, as bookmarks are equivalent to browsing states, they can also capture different states of the same frameset. This distinguishes the bookmarks of teamXweb significantly from the bookmarks in most Web browsers, as they usually only store the starting page as bookmark. This feature is particularly useful with documentations like the Java API which rely on a frameset for comfortable navigation.

5.4 Communication

While sharing bookmarks and history are key components to a system that shall support collaborative Web usage, they need to be complemented by support of the most important aspect of collaboration: communication. One objective of the project is to create a well-integrated platform for collaborative Web usage, and thus the system will also provide communication features.

There are two dimensions of communication in the context of teamXweb: asynchronous vs. synchronous communication, and the target of communication. While synchronous communication (e.g. chat) will be a very interesting feature when the system is used heavily and frequently by a large user base, synchronous communication is beyond the scope of this work. Thus, only asynchronous communication is implemented.

Users can send other users private notes, similar to eMail. The advantage of providing an alternative to eMail is mainly that the whole system is more integrated that way. However, in the long run it will make sense to integrate teamXweb's messaging system with eMail so that users can choose which system to use without a break in the user interface.

Communities are another target of communication, which makes communities a sort of message-board at the same time. The same discussion as before with eMail applies here with the relevant well-established communication services for communities: mailing-lists and newsgroups. However, the tight integration into teamXweb is even more important here than in the user to user communication, and the integration of community communication justifies the integration of user to user communication even better.

A long-term goal may be providing a proprietary interface to these services that is well-integrated into teamXweb, but using the open, well-known and well-accepted standards below the surface.

Last but most important, notes can be left on Web sites and pages. This way, pages can be annotated and at the same time, a discussion about the content of the page can be held. When a user leaves a note on Web site or page, he can choose to whom this note is visible: either it is a private note that only the user can see, or the note is visible to one of the communities he is a member of, or it is a public note that is visible to all users of the system.

5.5 Statistical Information

The same choice of scope - private, per community and public - is also available for the statistical information, which is displayed for each visited page, below the actual page. There are several types of statistical data which will be outlined in this section.

For each page and site, the number of visits is shown as well as the number of visitors. As mentioned before, this can be referring to the user himself ("how often have I been on this page before?"), one of his communities or all teamXweb users.

The same applies to the followed links: Whenever a user clicks a link on a page, a counter is increased and the most popular links are displayed in the statistics. For many people, however, it may be more interesting to see which links have lead to the page - and this information is also available. Thus, one can easily follow the most popular path towards a page backwards. As pointed out in [Gibson1998b] , this can also make finding good hub pages easy.

6 Outlook

An overview of relevant terms, related work and privacy issues has been given and the features of a prototype have been sketched. This provides only an introduction to a more complex and promising matter. It has been shown that there already exists a large body of related work and approaches. However, it seems that no attempt has been made to integrate those approaches into a system that helps both individual users and communities to collaboratively use the Web.

A first step into that task is developing the ideas outlined in this paper into solid concepts. In particular, models for communities and navigation behavior are discussed. Furthermore, the architecture of the prototype is elaborated.

After that, the prototype is implemented as described in section 5 . With the implementation, the prototype is to be tested for acceptance, reliability and missing features. For this experiment, a community of users willing to use the Web in a new way, with the limitations of a prototype and the outlook on a more social and collaborative Web is looked for. Ideally, there are natural groups, e.g. teams, within that testing community, who will experiment with the provided features and give feedback on the usability and benefit of those features, which will in turn help moving Towards an Integrated Approach to Collaborative Web Usage.

A Indices

A.1 Terms

The following list gives an overview of all the terms that are used throughout this paper including links to the actual definitions as well as the sections where they have been given:

A.2 Tables

The following table gives an overview of all the tables that are included in this paper with a link to the table, its caption and a summary describing the contents of the table:

B References

Lehr- und Forschungseinheit Institut Universität

Last modified: Friday April 12 2002 by Holger (David) Wagner