VDD project is currently working on performance analysis. The whole system, including every single feature, is put under the test, both on an individual base (one user at a time) and on a collective base (several users at a time). Contrary to what happens with servers, the main requirement for interactive desktop-based applications is to respond to user-generated events with a delay that is acceptable to their perception. Refering solely to throughput, bandwidth, end-to-end latency does not provide performance indications that are relevant for the desktop.

For this reason, our benchmark set is organized into three categories:

• Microbenchmarks: useful for understanding system behavior for simple interactive operations. Eg: GUI elements behaviour.
• Task-oriented benchmark: to understand the real impact of latency on the perceived interactive responsiveness of an application. Eg: office-suite tasks.
• Application microbenchmark, to evaluate isolated interactive events from the realistic workloads. Eg: page-down on Impress.

For widening the theme, see here

In order to obtain a mapping of system performance to desktop performance, we costantly measure values of parameters such as CPU, memory, disk and network, while operations, belonging to the aforementioned benchmark categories, are performed. Those values are stored on files with time stamps and eventually analysed and commented. Whenever a threshold is overpassed, an anomaly is reported and analysed apart.

Moreover, a questionnarie is given to people doing tests in order to determine satisfaction of users during the utilization experiences, useful for evaluating the system also in a qualitative way and possibly improve it.

The results and a final report will be published soon!