by Holger (David) Wagner
This page provides the complete history of TeamWeb in reverse order - so the uppermost items are actually news ;-) If this is the first time you visit this page, and you really wanna know everything, it may be a good idea to read this backwards... !hguoht, laretil oot taht t'nod
It's been quite a while... the diploma thesis is done, I (david) have done all my final exams and that means I have graduated. Finally, I got around getting a new server that can host an application server - and so now I have JBoss running.
It took me about a day to get teamXweb running on JBoss with Jetty - mostly because this was the first application I put to JBoss and I've also moved from JBuilder to Eclipse - simply because I can't afford some $ 4.000,00 for a license...
Anyways, after a long downtime (probably half a year), teamXweb is up and running again. Possible, I might soon make it a real J2EE application (as I had planned quite a while ago) and get rid of the XML persistence.
Thanks to Sacha Berger for pointing me to http://groups.google.com/groups?dq=&hl=en&lr=&ie=UTF-8&oe=UTF-8&th=71a0e44787a7c294&seekm=ahgfbr%247f71%40ripley.netscape.com&frame=off, where a new version of the Flash Plugin that supports full scripting (aka LiveConnect) is announced.
This makes the limitations mentioned in the 2002-03-31 historical - as long as you download and install the new plugin, available @ http://www.macromedia.com/go/getflashplayerbutton/.
We welcome all Netscape and Mozilla users to the new experience!!!
As the much better looking ALPHA 4 style should now work
on most browsers, it's also the default style from now on.
Just some minor issues that should improve the user experience while browsing these pages: At the top of the entry page, there is now a quick-links-navigation-bar with the most important links right at your fingertips. The structure of that entry page has also been improved to make it more logical and intuitive, and the instructions of how to add a link to my site has been put on an own page. Finally, some of the pages now include teamXweb in the title for better bookmarking and search-engine handling.
Somehow the data become corrupt at 2002-06-08 between
11:53 and 13:53.
That caused the storage thread to not work properly anymore - resulting
in the loss of all data between 2002-06-08 11:53 and
2002-06-13 17:53.
Please forgive any inconveniences caused through this bug - in particular,
steffi, AnnaE and Susanne Hainke,
who have lost their accounts due to that problem!
However, the bug has been discovered and fixed - so it hopefully won't ever happen again!
The (still incomplete) changelog:
ALPHA 5: This is the biggest update so far:
Have a lot of fun with the new release! And let us know how we can further improve teamXweb!
Introduced teamXweb to the participants of the seminar Ethnologie@Internet and Hauptseminar "i18n and l10n", World Wide Web Internationalization and Localization (SS 2002) and invited them to join the fieldtest. This time, we asked for the eMail adresses of those people who are interested in participating in the test so that we have a more solid contact to the participants.
We have 9 participants from i18n and l10n and 11 participants from Ethnologie@Internet.
Furthermore we have a single person that actually participated in the previous test with 19 sessions. A little informal interview brought to our knowledge that the most important feature for him was having the bookmarks available wherever he logged in. As a major disadvantage he considered the flat hierarchy for the bookmarks. None of the collaborative features could be evaluated by that participant as he used the system on his own.
Furthermore, we have had 2 people who logged into the system 2 times and 6 people who logged into the system a single time. We can contact 5 of the participants of the first fieldtest via eMail and will send them the first questionnaire when it is ready.
On the main entry page of the project, there was just a project description and a collection of unsorted links. In the beginning, that was fine as there were not many links - but in time, many links were added and it become more and more cluttered.
Now put the links under categories, changed the title and logo of the page and made the prototype and tutorial much easier to access. Also fixed a few minor issues with the whole Web appearance (e.g. the statistics lacked a style-sheet which made them look really ugly).
Added pages which allow logging in to teamXweb (currently two) and provided information on how to include a link to teamXweb on one's own page.
Pages are now rewritten in such a way that Flash-movies also work. This was particularly important as we now use Flash-movies on our own project pages - and thus our own project pages were displayed incorrectly, what a shame! Now it works, and Java Applets have been also tested and also work. However, some pages might not yet work - so if you find any broken Flash movies or Java Applets, please file a bugreport!
Also note that Flash, JavaScript and Java navigations do not and will never work by principle. They will be displayed, but using them will cause teamXweb to behave in inpredictable ways (usually, it will simply not record the navigation steps done using the custom navigation).
Kernzeit has completed the new navigation which is implemented as a Flash movie. After some integration work, that thing rocks with teamXweb big time! The only sad affair with this is that Mozilla/Netscape 6.x do not support LiveConnect with Flash-movies, which causes some serious functionality misses. Go file some bugreports... and use Opera, Netscape 4.7 or Internet Explorer in the meantime!!!
teamXweb now also supports skins - only two for now, though. New users (and Mozilla/Netscape 6.x users) can check out the old style or choose to use the new style. Thanks again to Kernzeit (now CLICK this link) for the design!
Another big thing - more on the backend, though - is that I have improved the way links on the pages are handled. No longer are all links kept in memory and this reduces the memory consumption of teamXweb with the current user base and "known web" from around 40 MB to 12 MB. The trade-off is that teamXweb may be slower with pages that have a lot of links, e.g. the Java API Documentation...
All project information has now moved to the domain www.teamXweb.com - the old pages still exist but will no longer be updated! The prototype is now accessible via http://www.teamXweb.com/teamXweb/. All eMail addresses have also been moved to that new domain, and the name has been changed within the prototype (packages etc.)!
The functionality has not been changed much, but I have changed the colors so that they match the new logo - which makes the whole thing look much better!!!
If you find any inconsistencies (broken links or the like) that may indicate that the name change may still be somewhat broken, please let me know!
As the term TeamWeb is used quite often in the Web context, and as there is a trademark NetObjects TeamWeb™, we decided to change the name to a less used term: teamXweb. People who know set theory will see that this is the cross-product of team and Web, which is much more descriptive than TeamWeb. Changing the name within the prototype is still pending, though...
With the implementation of storing the user actions, I introduced a bug that made displaying other user's history states and bookmarks impossible. Discovered and fixed that bug today!
I'm now using Tomcat with Apache, and so the obsolete :8080
is no longer needed with the URL. The official URL of TeamWeb is
now http://ultimage.net/teamweb
(later changed to http://teamXweb.com/teamXweb/).
However, for backwards compatibility, the old address still works.
The next step is moving the domain teamXweb.com to the new
server so that the final address can be used.
Bookmarks can now be exported to the format XBEL. Importing will be implemented when the bookmark-categories have become hierarchial.
User actions were previously stored only in logfiles which made
mixing the user sessions with the actions that changed the states very
hard. Now, each state (BrowserWindow) knows how it was
entered and how it was left. That way, reconstructing the complete
user session as the user performed it becomes an easy task!
Held a presentation at the "Oberseminar" "Knowledge Representation and Markup Languages" and "Data Modelling and Inferencing". During the discussion, there was some valuable feedback. In particular, hierarchial categories for the bookmarks and importing and exporting of bookmarks is a requested feature. Furthermore, the suggestion was made that users should be helped when setting up their security profiles, e.g. by a default setting for when they join a new community.
A few weeks ago, Google has changed some policies and blocked requests from unknown user-agents. As TeamWeb did not change any of the headers of its URL requests, Google thought TeamWeb was an illegal search-engine trying to forward requests to Google. Obviously, that was not the case. To convince Google, that this is not the case, I now include all the headers from the client-requests into my requests, so that it looks as if the client (e.g. Opera) directly sends the request to the server. Google likes it ;-)
Also, I fixed a bug that prevented forms to be processed correctly in the history and statistics. Now, all queries that have been done with any search-engine are listed as "out links" in the page statistics, and the states are also handled correctly in the history. Another major milestone because of a significant improvement of the usability of TeamWeb!!!
It is now possible to visit the system and play around with all of its features without having to log in. That way, new users can see if they like it or not and don't need to go through the (actually quite simple and quick) process of adding a new user. Hopefully, this will set down the threshold for new people...
Implemented an export of all the relevant data to XML which is much smarter than simple serialization. There was quite a memory problem because it seems as if standard serialization creates multiple instances when the same object is referenced at different places. Now I'm doing it all myself, which reduced memory consumption from 70 MB to 15 MB!
Besides, all data is now available in XML and thus human readable and editable, which will make discovering and fixing problems with the data much easier! As a side-effect, bookmarks can now be exported and imported in XBEL.
This also allowed (and required) downgrading to JDK 1.3.1 which is much more stable and works with all the libraries I use (unlike JDK 1.4 beta 3 which caused a lot of trouble with some of the XML libraries)! This is definitely a major milestone in the development of TeamWeb (even though not much has changed for the users)...
As there was some trouble with SuSE 7.2 and JDK 1.3.1, the server has been down for about 2 hours yesterday. In future, testing things like that will be done in ways that do not interfere with the running system... apologies again to anybody who may have tried logging in during that time!
I'll be downgrading from JDK 1.4.0 beta 3 to JDK 1.3.1. Unfortunately, there's some trouble with converting the user data - so the server's down for an evening... probably, no data will be lost, though! The server had been down for about 6 hours, sorry to anybody who tried logging in during that time!!!
Held a presentation of about 45 minutes including a demonstration of the prototyp for the participants of the Programmierpraktikum.
Added a (secret) JSP with rather extensive statistics about TeamWeb. Including: Server Statistics, Server Environment Information, General TW Stats, Users, Communities, Visited Domains and Visited Pages.
Users can now view the notes they have sent. Depending on where they are (page/site annotations, communities, messages to user self), they can view the messages that they posted in that context. In "messages to user self", a user can view all messages he has posted (however, he cannot find out where he posted them, and reading them may come with some bugs).
Communities can now have a parent-community and are only visible to members of the parent-community. This was needed to avoid mixing the communities for the XML-Praktikum and Programmierpraktikum.
Held a little presentation and demonstration of TeamWeb to the participants of the XML-Praktikum, and invited them again to participate in the test.
Sent out an eMail inviting friends and acquaintances to try out TeamWeb. The first two users of TeamWeb: oj and everytec - WELCOME TO TEAMWEB!
The visual design of the communities (actually: the "community manager") had been quite outdated. Transformed from "everything in a single HTML page" to an explorer like view, where communities can be selected on the left hand side of the window, and the community is displayed on the right hand side of the window. This improves usability significantly!
TeamWeb is now served by a server hosted by Puretec, to which I have root access. That way, TeamWeb is accessible from anywhere on the Internet (before, it was accessible only within the MHN - a part of the Internet that belongs to the universities of Munich).
Sent out an eMail to the participants of the XML-Praktikum, inviting them to participate in the field test of TeamWeb.
Wrote a tutorial that leads new users into the system.
With each viewed page some statistics on that page are displayed in a subframe. The statistics include information on the page and domain (hits, how many users have seen this, etc.) as well as link information (links followed from that page, links that lead to that page).
Users can now post messages to other users, to communities and as annotations on WebSites and domains. The annotations can be made visible only for the user, for one of the communities he is a member of or for all TeamWeb users.
The system now provides a complete bookmark-handling with a flat categorization of bookmarks (no tree-structure for the categories for simplicity).
User profiles can now be stored and viewed.
Opera has the smartest, which made it the worst: the browser caches
every page, and whenever the page is loaded again - my server is kept
off and knows nothing about it. At first, I wondered why a lot of pages
were not included in the history - until I found it Opera sends no
request because it already has the page in its cache.
Well - smart caches need smart URLs with random dummy parameters that
make each page-call look different. Gotcha!
Today, the history feature worked for the first time! It also gets updated while the user is browsing...
Ever tried <jsp:include ...> or
RequestDispatcher.include(...) (I assume you
are a developer - if not, skip this and accept my
apologies)? Well, Tomcat's a nice baby and does not scream or cry or
give me any indication that the requested resource cannot be found.
That way, it took me HOURS and HOURS to find the source a very simple
problem. Shame shame shame...
After some fiddling with the handling of parameters on dynamic pages, queries on Google are handled correctly. That definitely was a major milestone to the usability of TeamWeb, so please celebrate this day every year!!!
For a few weeks, I was quite worried about the performance of my servlets.
With JSPs it seemed to be even worse. Sometimes, it took 2 1/2 (yeah,
two and a half) minutes, before a page was displayed - served from my
notebook and displayed on Netscape also on my notebook (no, not a
2400 baud modem connection - it's 2001, remember?)
One lucky day, I accidentally clicked on the taskbar while Tomcat was
seemingly working its a** off - and the page appeared instantly. Um...
the magic was repeatable. Starting Netscape on one computer, and having
Tomcat on another one finally revealed the dark truth: while Netscape
waits for a page, it blocks the whole system. Classic deadlock. I don't
know who to blame - but it works fine with Mozilla, Opera and
Internet Explorer - so Netscape 4.7 is definitely dead for me (also
because it crashes permanently on Linux).
Installed a few eMail-adresses for people who want to file bugreports, request features or comment generally on TeamWeb:
bugreports@teamXweb.com feature-requests@teamXweb.com comments@teamXweb.com
Far too long I had postponed this change: I had my own templates and
many other workarounds - and all this is done much better using
JSPs. So after some experiments, I changed the whole application so
that JSPs would provide the view and servlets would provide the
control/logic. Of course, at that point in time such a change caused
a lot of work - but in the end, it paid off because once it was done,
everything went much faster...
Taglibs were another story, though. While JSPs make development much
faster, creating custom tags introduces some overhead not appropriate
for the development of a prototype. Thus, I keep many things done with
JSP scriptlets even though custom tags would be a much nicer solution.
Once everything is stable, converting such parts to custom tags will be
a nice task.
The basic logic for communities was implemented - still based on simple servlet technology which was quite a mess of HTML mixed into Java code... in particular, all communities and all community administration tools were put on one single HTML page...
To avoid the overhead of working with a relational database, XML
files or object oriented database, I decided to use the simplest
mechanism for persistence available in Java: object serialization.
This turned out to be a good solution for the prototype, as design
improvements - which are quite common in that phase of development -
can be applied in a lightweight manner (no need to change database
schemes etc.)
A few traps had to be circumvented, though - e.g. that serialization
does have a few problems with reference loops where an object
is referenced to itself via some intermediary references...
but after a long search for the cause of the problem, the workaround
was a matter of minutes.
Sebastian Schaffert introduced me to JTidy, an HTML pretty printer and parser that makes any Web page accessible via its document object model (DOM). As most authors of Web pages seem to not care at all about writing their pages according to the well-known standards (excuse me: W3C recommendations), my approach with regular expressions replacing certain patterns (based on the standards) failed with many pages. Thus, having JTidy do the "dirty work" made implementing a clean page processor much easier for me.
I needed a document to introduce TeamWeb to the participants of the
fieldtest. Also, I had to decide whom to invite to that field test
and talk with the responsible staff.
On the technical side, TeamWeb had to be deployed on a server
accessible to the computer pool. The first server that ever hosted
TeamWeb was basilikum.cip.informatik.uni-muenchen.de...
After learning about servlets (and how to design an application
based on servlets), some design and implementation work - the first
experimental prototype that could track a user's navigation behavior
on the web was ready. A user could log on and view a few pages.
On the backend, I had modelled the browser (window, frames) and web
pages with objects and their relationships by then. However, the
backend was not connected to the frontend at that time...
One major question in the design of the prototype was how to model the visited web. As Michael Kraus had to deal with a similar problem in one of his projects, we worked together on the model for a while. Our goal was to find a general approach to modelling browsing behavior of web users. However, it soon turned out that no such general approach can be found, as the best model for a given application depends heavily on that application. However, different models can relatively easily be transformed into other models, so having no general model is not such a bad issue.
My first approach to rewriting URLs was using regular expressions.
As regular expressions were an extension of J2SDK 1.4 over 1.3, I
decided to use the beta version of that new J2SDK. Much later,
using a software in beta stage for development turned out to be
quite a time-killer, as some of the libraries I also needed were
incompatible with changes made to the J2SDK. It would have been
a much better decision to look for an independent regular expressions
library (there are some) and use that with the stable J2SDK 1.3.1
instead. What a painful and time-consuming lesson...
However, I soon had some working code (and back then, no idea that
I would regret that decision so bad later on).
Today, I had the idea to use servlets instead of a proxy for keeping
track of where the user currently is. All URLs of all visited pages
would be rewritten to query the document from the servlet. The
serlvet would then load the document from its original server,
rewrite the URLs and send it back to the client. That way, no
client installation or proxy configuration is necessary. However,
this approach has several limits which are almost impossible to
overcome, e.g. when navigation is done via Flash-movies or Java-Applets,
that cannot be changed by the servlet.
While other approaches like writing an own browser or browser-plugin,
or using a proxy server have been discussed in the literature, this
approach is unique. Testing such a unique approach gives the whole
work some relevance, even if it turns out that it doesn't work ;-)
After two months of literature research and review, I had found about
70 papers on subjects that were more or less closely related to my
project. There were also some interesting related projects. During
that phase the scope of my project changed a few times, because
some of the things I had intended to do had already been done in
other projects.
The major shift was from tracking the navigation behavior of web users
to providing a tool that supports collaborative web usage.
This also required to change the name of the project
from Tracking the Navigation Behavior of Web Communities
to Towards an Integrated Approach to Collaborative Web Usage.
At that time, my supervisors
Prof. Dr. Fran�ois Bry
and Michael Kraus,
and I also decided to put the writing of the project thesis on hold
and start designing and implementing the prototype first. This
decision was based on the requirement to test the prototype in the
field - so it had to be ready before the following term, starting
middle of October.
I had prepared a project presentation for the Oberseminar, which I held today. Thus, I introduced the project to more people who also gave me valuable feedback.
Today, I had the first meeting with
Prof. Dr. Fran�ois Bry, where
I introduced my ideas and talked about writing a project and
possible later on a diploma thesis about this. The agreed - and
the actual work started.
A lot of conceptual work was then done, followed by a major
literature research trying to find out about related work. Also,
I prepared the project presentation for the Oberseminar.
We decided on a working name for the project thesis:
Tracking the Navigation Behavior of Web Communities.
During a very long night, I had most of
the ideas for a
collaborative web-surfing application. Back then, the most important
"feature" was tracking web-usage and deriving a graph of the
visited web, which would make the web a spacial place, where people
have locations, and could communicate based on their proximity
(link-distance). Creating a visual representation of this graph
was also a high priority... due to personal time constraints, I
could only do conceptual work back then, though - and prepared
myself to implement a part of the ideas in a project-/diploma thesis
to finish my computer science studies.
In the following phase, I had a few new ideas, including a
keyword-system based on a semantic network. However, every once in
a while the project got "laid down" and put on hold...
|
Last modified:
Sunday June 08 2003
by Holger (David) Wagner |
|
|