Single User Load Testing KPIs

For ‘Single User Load’ testing, web analyzers like HTTP Watch, DynaTrace etc. are used while browsing through the scenarios and the page load time and various other KPIs are collected. The KPIs are analyzed further for tuning the web page response time.

Following are the key KPIs that can be used for making an analysis –

• Web Page Loading Time
• Network Time
• HTTP Requests
• Request Size

Time to First Impression

This is the time from when the URL is entered into the browser until the user has the first visual indication of the page that gets loaded. It depends on the initial HTML document when the browser can start drawing content. There are different Best Practices available that talk about different strategies. Google for example downloads a minimalist page to provide a fast first visual rendering. It then delays loads more content after ‘onLoad’ or even later when the user starts interacting with the page.

The recommended value for this KPI is like this – great if < 1s, acceptable if < 2.5s and slow if > 2.5s

DOM Ready Time

The ‘DomReady’ event is fired when the DOM content has loaded and the program can access all HTML Elements without waiting for all those images to load also, which just means finding out when the document’s DOM tree has fully initialized, versus the entire document and every object in it. A classic example is the insertion/ deletion of nodes via the DOM, which can in fact be done sooner rather than later, when the DOM has loaded versus the document itself (i.e. firing of ‘window.onload’ event).

Other resources, such as images, may not have been downloaded when this event occurs. This corresponds to the DOM Content Loaded in Mozilla Firefox and IE 9.

Page Load

This is the time until the browser triggers the onLoad event which happens when the initial document and all referenced objects are fully downloaded.
JavaScript onLoad handlers use this event to manipulate the current initial state of the page. This event is one of the options explained earlier to download additional or delay load content.
The recommended value for this KPI is like this – great if < 2s, acceptable if < 4s and slow if > 4s
The significance of an ‘onload’ event is that all resources associated with web pages are downloaded and it is in ready / working state for starting the application.

Fully Loaded

This is the time until all onLoad JavaScript handlers have finished their execution and all dynamically or delay loaded content triggered by those handlers has been retrieved. It is sometimes a bit hard to identify the exact time when the page is fully loaded especially when JavaScript handlers use reoccurring timeouts that constantly modify the page, e.g.: to implement a ticker

The recommended value for this KPI is like this – great if < 2s, acceptable if < 5s and slow if > 5s

HTTP Load

It marks the completion of all HTTP or HTTPS requests made by the page

Network Time KPIs

KPIs listed in this section measure the time taken by various activities during processing an HTTP request by Browser.

Following is a sample timeline for a fictitious http request. It contains all activities that can be associated with HTTP requests. All activities listed in the picture below are explained in detail later. It needs to be noted that not all activities are present in every HTTP request. For example, DNS Lookup Connect might not be required if it is already resolved / connected. Similarly, “Cache Read” for 304 response code would not be present if content is modified and served from web / application server. The timeline width associated with each activity is for sample purpose only.

Blocked Time

• The “Blocked Time” includes any pre-processing time (such as cache lookup) and the time spent waiting for a network connection to become available.
• Usually Browsers limit the number of concurrent network connections per host name and will queue up requests if the limit has been reached.
• IE7 and Firefox 2 create a maximum of two concurrent network connections per host name.
• Often the block time is the most significant factor in the time to download images embedded in the website.

Cache Read Time

• Cache Read Time is the time taken to read the content from the browser during (Cache) or 304 responses (i.e. Not Modified).
• Cache can be read in following three situations:
• The browser sent an ‘If-Modified-Since’ header and received a 304 Not Modified response. The content would then be read from the cache because the 304 response indicates that it is still valid. The Content-Length header in the response will always be zero or absent because the HTTP 1.1 spec defines the 304 status code as never returning content.
• The browser decided not to send an HTTP GET request because there is an entry in the browser cache that either has a valid expiration date or has been recently checked.
• The browser is in ‘Work Offline’ mode.

DNS Lookup

• DNS Lookup is the time required to resolve a host name (e.g. www.google.com) into a numeric IP address (e.g. 216.239.59.99).
• One DNS Lookup happens for every domain that hosts resources for the current web site. If you move between multiple pages the browser does not require another DNS lookup for a domain that has been resolved on the previous page. It is interesting to look at the total DNS time to identify problems with DNS Lookup Times that can be caused by DNS configuration problems.

Connect Time

Depending on the browser and the number of resources that are served by a domain the browser establishes one or multiple connections to each domain that hosts resources for the page. Connect Time is the time it takes to establish the TCP/IP connection to the web server. Connections usually stay open unless the Web Server directs the browser to close the connection (Connection HTTP Header). When using secure communication via SSL, the Connect Time also includes the time of the SSL handshake. High Connect Time can therefore have the following reasons: slow network connection to the web server, usage of SSL and not allowing the browser to keep the connection open.

An excessively high number of TCP connects may indicate that HTTP persistent connections, also known as keep-alive connections, have not been correctly configured on the server. These long-lived HTTP sessions allow multiple requests to be sent over the same TCP connection, and in some cases have been shown to result in an almost 50% speed up in latency times for HTML documents with lots of images.

Send Time

Send Time is the time required to send the HTTP request message to the server and will depend on the amount of data that is sent to the server. For example, long Send times will result from uploading files using an HTTP POST

Wait Time / Server Time

Wait is the idle time spent waiting for a response message from the server. This value includes delays introduced due to network latency and the time required to process the request on the web server.

High Server Time means that the Web/Application Server required a long time to process the request. This is particularly relevant with requests that trigger application logic to be executed on the application server where higher Server Times can be expected – especially under heavy load periods. Monitoring Server Time is important to identify bottle necks, performance and scalability problems with the application server. It is usually easier to scale static content delivery by adding more web servers with load-balancers or by using a Content Delivery Network. It is not that easy to scale a dynamic application in the same way. Keeping an eye on this metric is important.

Receive Time

Receive Time is the time taken to read the response message from the server. This value will be depending on the size of the content returned, network bandwidth and whether HTTP compression was used.

Time to First Byte (TTFB)

It is the duration from the initial network request being initiated by the browser to the first byte being received from the server. It includes TCP connection time, the time to send the request and the time taken to get the first byte of the response message.

Network Time

Network is the total duration of all network related operations for an HTTP request.

Requests Elapsed Time

It is time when http request has been invoked at the final request. It can be calculated as “Blocked Time” + “Network”. It might be the case that sending of first request takes time and that’s why ‘Blocked Time’ becomes important.

HTTP Requests KPIs

# Network Roundtrips

This is the total number of network requests that get downloaded with the website. The ultimate goal is to keep this number as low as possible in order to reduce roundtrips. Monitoring this KPI gives an early indication on newly introduced content that can negatively impact page performance.

Great sites require fewer than 40 requests, acceptable are up to 100 requests. Sites with more than 100 HTTP Roundtrips are considered bad.

Some entries do not cause a network round trip because either an error occurred (e.g. ERROR_INTERNET_NAME_NOT_RESOLVED) or because no network request was required (e.g. entries resulting in a (Cache) response

# XHR (AJAX) Requests

The total number of XmlHttpRequests (XHR) sent via JavaScript to retrieve data asynchronously from the server. Monitor this KPI to identify sudden changes in dynamic content retrieval via XHR. Some JavaScript frameworks provide update mechanisms with the server-side and use XHR for these purposes. Depending on the configuration it can end up with too many XHR requests that not only impact client side performance but also cause additional load on the application server.

Total Number of HTTP 300s/400s/500s

This is the total number of requests to the server that responded with an HTTP Status Code of 300 (Redirect), 400 (Authorization Problem) or 500 (Server Error). These are requests that should be avoided as they have a negative impact on the page load time. The root cause of these problems is often server-side related implementation, configuration or deployment issues.

Number of Domains

The number of domains that host the web sites resources is important as it affects DNS, Connect and Wait Time. Additional domains that are utilized to download resources will have a direct reduction in the wait time because the browser ultimately uses more physical connections. This can have an opposite effect when more DNS lookups are needed and more time is spent to establish the physical connections.

Number of Single Resource Domains

Single Resource Domains should be avoided as you pay a high price for performing the DNS and Connect to download a single resource. It is sometimes not avoidable when downloading content from external content providers (such as ad-services). When having the deployment under your own control you want to make sure not to have single resource domains.

# Resources with No or Past Date Caching

Number of network roundtrips increases because of it.

# Resources Having Short Expires Headers

Number of network roundtrips might increase because of it.

Hit Count

The number of times that the browser has read content from the cache entry.

Size Related KPIs

Downloaded Data

This is the total amount of data that has been received from web server(s) (i.e. response headers and content). In the case of HTTPS connections this includes the overhead of the SSL encryption process.
This is the total size of all resources that make up your page. It is important to keep track of the total payload size. The larger web sites become the longer it takes to download. Changes to the page – such as adding images or new java script libraries – can have a significant impact on download time.

Uploaded Data

The total amount of data that has been sent to web server(s) (i.e. request headers and POST data). In the case of HTTPS connections this includes the overhead of the SSL encryption process.

Total size of Image / CSS / JS

Besides keeping track of the total page size it is important to look into the sizes of the individual content types such as Images, Style Sheets and JavaScript files. With this it is easier to spot the main contributors of page size.

The AJAX Edition makes the assumption that most Images, CSS and JS files served from the same domain can be merged so that we do not end up having more than 1 CSS, 6 Images and 2 JS files from the same domain.

Average HTTPS Overhead

This is the average overhead in bytes of each HTTPS request. If this value is high (more than a few hundred bytes), it may indicate that the server is incorrectly configured and is causing an excessive number of HTTPS renegotiations.

Compression Saving

This is the reduced total number of bytes and percentage which received content was compressed by HTTP compression.(i.e. “gzip” or “deflate”).