Top load-testing frequently asked interview questions

Testing with JMeter: how to run N requests per second

I need to test if our system can perform N requests per second. Technically, it's 2 requests to one API, 2 requests to another, and 6 requests to third one. But the important thing that they should happen simultaneously - so 10 requests per second. So, in JMeter I've created three Thread Groups, first defines number of threads 1, and ramp-up time 0. Second thread group is the same, and third thread group defines number of threads 6 and ramp-up time 0. But that doesn't really guarantee it's going to run them per second How do I emulate that? And how do I see the results -- if it was able to perform or wasn't?

Thanks!

Source: (StackOverflow)

How to do load testing for websockets

We are building a group chat feature, which is using websockets. We want to test how many connections our current infrastucture can support.

Basically it boils down to how to simulate a websocket.

Source: (StackOverflow)

What is the highest number of threads that is reasonable to simultaneously run in Jmeter?

I want to use the highest possible number of threads (to use less computers) but without making the bottleneck to be in the client.

Source: (StackOverflow)

Load Testing with AB ... fake failed requests (length)

To do some load testing, for my own curiosity, on my server I ran:

ab -kc 50 -t 200 http://localhost/index.php

This opens up 50 keep-alive connections for 200 seconds and just slams my server with requests for index.php

In my results, I get:

Concurrency Level:      50
Time taken for tests:   200.007 seconds
Complete requests:      33106
Failed requests:        32951
   (Connect: 0, Receive: 0, Length: 32951, Exceptions: 0)
Write errors:           0
Keep-Alive requests:    0
Total transferred:      1948268960 bytes
HTML transferred:       1938001392 bytes
Requests per second:    165.52 [#/sec] (mean)
Time per request:       302.071 [ms] (mean)
Time per request:       6.041 [ms] (mean, across all concurrent requests)
Transfer rate:          9512.69 [Kbytes/sec] received

Note the 32951 "failed" requests. I cannot figure this out.

As the test was running, I was able to access my web site from my home computer perfectly, albeit page load times at the bottom of the page were reported as .5 instead of the usual .02. However I never once had a failed request.

So why is AB reporting that half of the connections fail? And what does "Length: " mean in that context?

Thanks

Source: (StackOverflow)

Failed requests by length in my ApacheBench load test result

I have a website in PHP, Lighttpd. It uses also MySQL on Centos 5. I've tested my PHP with code below with Apache Bench (ab). It resulted in some errors (Failed Requests) indicating other length than normal. I'm absolutely sure that my PHP result should always have the same exact length. I've reviewed my Lighttpd and MySQL logs and error logs and don't have any errors there.

Is there any way to check exactly what ab gets when result has other length or is there any other way to find out what is the cause or what is the "bad" result?

I need to know that because I need to have 100% good results.

-bash-3.2# ab -n 500 -c 200 http://domain.com/test/index.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking domain.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Finished 500 requests


Server Software:        lighttpd/1.4.20
Server Hostname:        domain.com
Server Port:            80

Document Path:          /test/index.php
Document Length:        15673 bytes

Concurrency Level:      200
Time taken for tests:   0.375862 seconds
Complete requests:      500
Failed requests:        499
   (Connect: 0, Length: 499, Exceptions: 0)
Write errors:           0
Total transferred:      7920671 bytes
HTML transferred:       7837000 bytes
Requests per second:    1330.28 [#/sec] (mean)
Time per request:       150.345 [ms] (mean)
Time per request:       0.752 [ms] (mean, across all concurrent requests)
Transfer rate:          20579.36 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   10   9.4      6      30
Processing:     0  113 133.5     16     342
Waiting:        0  111 134.3     12     341
Total:          0  123 138.9     16     370

Percentage of the requests served within a certain time (ms)
  50%     16
  66%    235
  75%    289
  80%    298
  90%    331
  95%    345
  98%    365
  99%    368
 100%    370 (longest request)

Source: (StackOverflow)

Tools for website/web application load testing? [closed]

Before going into production, our client demands actual numbers of how many users our web application can handle.

We have all kinds of features implemented including asset management (file uploads/downloads), documents import/export, various statistics, web-services etc.

I guess we need tool which could emulate users form submission because documents import/export as far as I noticed is the slowest part of an app because of parsing and generation.

Which tool (or set of tools) could do this?

Application details:

XHTML/jQuery
Coldfusion 8
SQL Server 2008
Windows Server 2008

Source: (StackOverflow)

Best practises and literature for web application load testing

As a web developer I've been asked (a couple of times in my career) about the performance of sites that we've built. Sometimes you'll get semi-vague questions like "will the site continue perform well, even during product launch week?", "can the site handle a million users?", and even "how is the site doing?"

Of course, these questions are very legitimate, and I have always tried to answer these questions to the best of my ability, using a combination of

historic data (google analytics / IIS logs)
web load test tools
server performance counters
experience
gut feeling
common sense
a little help from our sysadmins
my personal understanding of the software architecture in question

I have usually been able to come up with reasonable answers to these questions. However, web app performance can be influenced by many things (database dependencies, caching strategies, concurrency issues, etcetera, user behaviour).

I'm a programmer and not a statician, and my approach to this problem has always felt deeply unscientific. So I did a little more research... and all of my google results seem to focus on tools and features and metrics (and MORE metrics) when I am really looking for a way to make sense of these things.

The question: What are some good resources (books?) to read on the best practices for a developer to read on the subject of web load testing, that will help me answer these types of questions?

Source: (StackOverflow)

Access log replay for load testing? Jmeter Pitfalls and Competitors

Context

We wish to use "replay" web server access logs to generate load tests. JMeter had come to mind as I'd recently read blog posts about using jmeter in the cloud (e.g. firing up a number of Amazon EC2 instances to generate the load)

For years I had heard of JMeter's ability to replay access logs, but in reviewing this feature I found the following.

Access Log Sampler

DOES:
- recreate sessions, i.e. handle the jsessionId token (thought it tries to approximate sessions by IP address);
DOES NOT:
- handle POST data (even if you could configure apache/tomcat to write out post data to the access log, jmeter access log sampler only handles 'common' log format).

Post data would go a long way toward recreating actual load.

Additionally, the documentation describes the Access Log Sampler as "alpha code" even though it's 8 years old. It doesn't seem actively maintained. (That's longer than Gmail's beta.)

HttpPerf

Another blog post pointed me to the httpperf tool. I've started to read up on it:

blog: http://www.igvita.com/2008/09/30/load-testing-with-log-replay/
httpperf: http://code.google.com/p/httperf/

Summary

What's the best way to generate load testing 'scripts' from real user data?
What has worked best for you?
Pros and cons of various tools?

Source: (StackOverflow)

Running multiple Thread Groups sequentially in JMeter

I have a Test plan in which there are multiple Thread Groups in it.
I want to run all of the Thread Groups sequentially.
Thread Groups are as below:

Thread Group1
Thread Group2
Thread Group3
. . .
Thread GroupN

I read in different blogs/ articles on internet, people claiming that the thread groups will run in the order they are defined but apparently they are not in my case. Thread Group4 runs before Thread Group1. Thread Group4 is generating report which is wrong because it runs before Group1.

How to make sure the order of Thread Groups?

Also, I need to implement following scenarios:

Run a single request multiple time by a single user (Single user should create 1000 accounts from a single HTTP request).
Run a multiple requests multiple times by multiple users simultaneously (Multiple users should create 1000 accounts simultaneously from a single HTTP request).

How to do so?

PS: Please read and understand the query carefully before replying.

Source: (StackOverflow)

How to measure CPU and memory during load test

We are using JMeter in a heterogeneous environment and we would like to measure CPU and memory while we are producing heavy load on a server where our service or web application is running. What would you recommend to use? Is there any software solution (open source or commercial) available which records CPU and memory usage in a file or to a database?

Thank you!

Source: (StackOverflow)