Introduction to server side performance testing

least, plus 40 ongoing projects
Longest project: 12 years
Largest project: 22 performance analysts
Geographical distribution of performance analysts
across EPAM offices*. Majority is still in Belarus
Most of performance analysts
have experience with 2+ performance test tools
Performance analyst combines skills of
developer, tester, network engineer, DBA, etc
Men/Girls is 67%/33%**

functional requirements (expectations) or not.
Anything wrong?
One small correction makes difference:
Quality is a property which shows if product meets its requirements (expectations) or not.
Performance - The degree to which a system or component accomplishes its designated functions within given constraints* (IEEE).

consists of:
Functionality
Usability
Reliability
Efficiency
Maintainability
Portability

Functional testing process

Performance optimization process

Our descendants will do that… hopefully

Good Performance

What are usual consequences of poor performance?
Unbearable slow software product’s reaction to user requests
Unexpected application crashes
Expected (still unwanted) application crashes at the moments of extreme increase in load
Software product vulnerability to attacks
Problems with product scalability in case it gets more popular
And more

is to assure good product performance.

To do that the following steps are to be performed (often in an iterative way):
Check Define requirements (= requirement analysis)
Measure performance (= performance testing)
Find performance bottlenecks (= performance analysis)
Fix found problems, so performance increases (= tuning)

Requirements

Testing

Analysis

Tuning

a target, not as a sacred cow
System analyst
To extract information from tons of data
To match cause and effect
Programmer
To develop test script and special tools
To automate testing where it is possible
Network engineer
Because no one else cares ☹

too late because you need time for performance fix.
Better options:
Start test at same time as functional team (at least) – meaning you need to develop scripts and prepare everything even earlier
Start after functional test, but allocate time for performance fixes and functional re-test
Remember!
You’re likely to spend most of time on test results analysis. It is not scripting and test running what makes it all difficult and long lasting, it is rather searching for the performance bottlenecks.

end user

http://mynews.com

Typical first assumption:
Problem relies somewhere on the end user side.
This is not always true.

User’s computer

Bad client side code? JS, Flex, etc.
Weak modem?

Internet connection

Too many hops?
Too slow connection by design?
Broken hardware?
Too far away?

Web-server

Weak hardware?
Some hardware problems?
Improper configuration?

News
web-service

Internet / LAN connection

+ wrong topology?

Application
Server

News
database

…

Bad (ineffective, dangerous, stupid, etc.) code?
Improper configuration?

+ Versions incompatibility?

Wrong database system?
Bad indices?
Bad code?
Weak design -> locks?

Database
Server

…

Internet / LAN connection

Network again!

What? Damn web-site is so slow!

under server-side performance tests

indicates its ability to be as fast, powerful, stable, and scalable as required.

= response times

+ capacity

+ stability

+ scalability

Scalability

So there are types of tests to measure performance:

Ramp-up test

Fixed load test

Longevity test

mention:
Rush-hour
Rendezvous
Render time
Production drip
… And even more to invent!
Ask me how to join performance optimization group to know the details ☺

scalability

Fixed-load test

Assessing stability

Longevity test

Network testing

Testing from different locations

User experience testing

Cross browser testing

Mobile testing

to process some requests.
It takes 1 second to process 1 request with about 10% of resources utilization.

What happens if 1 user sends 1 request?

What happens if 2 users simultaneously send a request each?

What happens if 10 users simultaneously send a request each?

What happens if 11 users simultaneously send a request each?

Some options to consider:
Option 1: Application will crash.
Option 2: Application will process 10 requests within 1 second and throw 11th request away (not a good thing, because we make 11th user completely unhappy).
Option 3: Application will process 10 requests within 1 second and put 11th request in the queue (much better, but still not a good thing).
Option 4: Application will serve all 11 requests, but it will take more than 1 second to process each of the requests (sill not a good thing, but much better than options 1-3).

What happens if 20 users simultaneously send a request each?

We cannot load this application with more than 10 requests/sec because it is its natural capacity

capacity is named as system saturation point.

have capacity (thus saturation point)
Saturation point is evident

after saturation
Any good news?
System is stable enough to have capacity (thus saturation point)

☺
Any good news?
It was we who observed that, not real users

Perfect ramp-up test!

load
Define duration

Define the load level vs capacity:
Low-load as 1 virtual user (~10% of saturation load);
Mid-load as 4 virtual users (~45% of saturation load);
High-load as 7 virtual users (~80% of saturation load);
Or some magical level of load (e.g. production-like, expected, etc.)

Duration should be:
Long enough;
But also short enough.

one → run a longevity test
Why is it a different test?

often. In order to understand the root cause, find the bottlenecks and analyze results thoroughly, hardware resources monitoring of all system components involved is required.
Important: load generator is also a component of the system under test and bottleneck may reside there as well!

It`s a good practice to monitor resource online during a running test (for ramp-up it`s a must): that way you`ll have at least some high-level idea of what is happening inside the system.
Also, you can try using your application manually from time to time during a running test to check if it`s doing fine: sometimes it`s the only way to catch sophisticated errors that automated tools may not be able to capture.

all! It’s just what to start with.

CPU
utilization

Memory
consumption

Disk
Usage

Network traffic
via specific NIC

+ possibly

For example, in Windows, it’s Performance Monitor
(search for perfmon in Start)
In perfmon: system monitor is for online monitoring,
and log for recording data for analysis.

Perfmon works with Windows Performance Counters.

Performance object

Performance counter

Instance

Hierarchical concept here:

the test

Ramp-up test

Fixed load test

Longevity test

So, just remember what the goals of the tests are, and it’s clear what to report:

will reach the addressee

The general rules are simple again:
Remember you’re writing a report for someone who will [make an attempt to] read it
Although you might be collected tons of data, report should contain useful information, not data
Keep information clear, accurate, and consistent

Typical sections of a performance test results report are:
Title of the report, author, and date
Test setup configuration (what, when, where, and how was tested; objectives of the test)
Table with related artifacts (links to any raw data, related test results reports, etc)
Test results section describing findings with all necessary details, supporting charts and tables
Conclusions and recommendations.* The most difficult one!

and client side performance testing:
“Web load testing for dummies”, by Scott Barber and Colin Mason
www.itexpocenter.nl/iec/compuware/WebLoadTestingForDummies.pdf
Nice book about performance testing .NET web-systems (in fact, a lot of general concepts there):
“Performance testing Microsoft .NET web applications”, by Microsoft ACE team
http://www.microsoft.com/mspress/books/5788.aspx
Statistics for dummies ☺:
“The Cartoon Guide to Statistics”, by Larry Gonick and Woolcott Smith
http://www.amazon.com/Cartoon-Guide-Statistics-Larry-Gonick/dp/0062731025
And, introduction to JMeter: http://habrahabr.ru/post/140310/