Performance Analysis for Java Websites

The following E-book explains “Performance Analysis for Java Websites”

This is best book for performance analysis for java websites.

Download book from here  (free download - registration not required)

and also u can download all technology E-books from this this site (http://knowfree.net).

enjoy.....  :)

thanks,

www.testingtips4u.blogspot.com

testingblog4u@gmail.com


Related posts:


LoadRunner VUGen81 Scripting for Web E-Book
Performance testing guidence book for web applications

Loadrunner Video Tutorials

Hi friends,

Recently I have gone through very good Blogs/sites on performance testing on loadrunner.

In those I found very good information. In my blog I want to share those good articles for my blog readers. J

Here I am giving some video tutorials list that I have found….

These video tutorials are very useful and informative to people who are in starting level in loadruuner performance testing.

Tutorial 1 - Introduction To Load Runner

Tutorial 2 -  Introduction Mecury Tour Website

Tutorial 3 - Introduction Vuser,Controller , Analyzer

Tutorial 4 - Vuser Part 1

Tutorial 5 - Vuser Part 2

Tutorial 6 - Vuser Part 3

Tutorial 7 - Transactions

Tutorial 8 - Playback

Tutorial 9 - Correlation Part 1

Tutorial 10 - Correlation Part 2

Tutorial 11 - Correlation Part 3

Tutorial 12 - Correlation Part 4

Tutorial 13 - Parameterization

Tutorial 14 - Verification Checkpoints

Tutorial 15- Controller

Tutorial 16 - Analyzer

Tutorial 17 - LoadRunner Video - Parameters - part1

Tutorial 18 - LoadRunner Video - Parameters - part2

Tutorial 19 - LoadRunner Video - Parameters - part3

Tutorial 20 - LoadRunner Video - Manual Correlation

Tutorial 21 - LoadRunner Video - Automated Correlation

Tutorial 22- LoadRunner Video - checking downloaded file

Tutorial 23 - LoadRunner Video - capturing Web page

Source: http://www.motivich.blogspot.com/ , http://www.guru99.com/loadrunner-tutorials.html

In next post I will provide some of good sites on… Performance concepts, performance tool information, performance certification information and performance monitoring and analysis.

Keep visiting      www.testingtips4u.blogspot.com

Share your knowledge to the people who are in testing world…. J

Please post your comments to testingblog4u@gmail.com

Post your articles/Information on Performance testing to testingblog4u@gmail.com I will publish in my blog….. www.testingtips4u.blogspot.com... Thanks… :)

Difference between processes and threads

The major difference between threads and processes is :

1. Threads share the address space of the process that created it; processes have their own address.
2. Threads have direct access to the data segment of its process; processes have their own copy of the data segment of the parent process.
3. Threads can directly communicate with other threads of its process; processes must use inter process communication to communicate with sibling processes.
4. Threads have almost no overhead; processes have considerable overhead.

5. New threads are easily created; new processes require duplication of the parent process.
6. Threads can exercise considerable control over threads of the same process; processes can only exercise control over child processes.
7. Changes to the main thread (cancellation, priority change, etc.) may affect the behavior of the other threads of the process; changes to the parent process do not affect child processes.

What are the limitations of Run-Time Viewer at LoadRunner

The LoadRunner Run-Time Viewer is an integral part in many Vuser script development environments. It provides an easy to use verification tool for watching the replay of Web applications while creating and debugging scripts. The HTML document presentation in many cases allows the scriptwriter to easily see if a particular script has generated the appropriate responses from the server during script execution. 

However, in some cases, because of the limitations of the Run-Time Viewer it is not always possible to use it as a final verification tool. There are numerous complex Web applications that will not be correctly displayed in the Run-Time Viewer or the Run-Time Viewer in response to running the script may even generate errors. These problems do not necessarily point to problems with the script itself. It is important to be able to distinguish and isolate Run-Time Viewer errors from script errors (which can often be two distinct events).

The first step in understanding this is to gain an understanding of the basics of the way LoadRunner Web replay works when running in VuGen.

When a Web Vuser is run through VuGen, the default is to have the Run-Time Viewer open and also to have resources downloaded automatically by the script. These two settings coincide with what the user sees -- that is the script runs through and the resources are downloaded and displayed graphically in the Run-Time Viewer. Now, what is really happening here are two separate things (that are only slightly related)?

The first thing that is happening is that the VuGen Web replay engine is running the script through the HP Driver process (mdrv.exe), which actually takes the script and interprets it to run the script. This process happens independently of whether the Run-Time Viewer is opened or closed and is done the same way in VuGen as it is in the Controller (which runs a multithreaded version of the driver process called mmdrv.exe).

 

The Run-Time Viewer displays the Web documents secondarily to the underlying Run-Time Engine and there is no interdependence on the two to operate properly.
Pop-up messages are often related to this because the Run-Time Viewer is making seperate requests from the Vuser.

There is some support for scripting and client side objects (like applets) but it also suffers some limitations (cannot display multiple frames, etc.).

Errors can best be diagnosed thoroughly through the use of the Extended Execution Log (which contains all the information about the requests and responses made during script execution). Generally, for easier debugging it can be helpful to turn OFF Resource Downloading (under the Browser emulation tab in the Run-Time Settings) so that the resources (usually binary data) are not recorded into the Execution Log (which clutters it with useless data). 

Visual Studio Team System 2008 Test Load Agent - Artical

Artical:
Visual Studio Team System 2008 Test Load Agent : 

Visual Studio Team System 2008 Test Load Agent It enables testers to generate test loads for Web applications and organizations to improve quality of service.

Visual Studio includes a new load agent product for achieving scalable load generation. The Team Test Load Agent enables you to configure an arbitrary number of agents that will work together to generate load against a web server or web farm.

It enables testers to generate test loads for Web applications and organizations to improve quality of service Visual Studio Team System 2008 Test Load Agent, which includes both agent and controller software, is highly extensible and customizable, giving testers an enormous amount of flexibility. It enables testers to generate test loads for Web applications and organizations to improve quality of service by more accurately testing the performance of Web applications and servers under load.

Key features include:

Scale out load generation
• Efficient load generation (1000 users from a typical machine running a typical web test)
• IP Switching for testing web farms. Each agent can be configured to use a range of IP addresses. That way load is distributed appropriately when load testing a web farm.
• Favorable Per CPU licensing (no vUser licensing)
• Performance counters collected on Load Agents and machines under test.
• Data (performance data, perfmon counters, and logs) are rolled up and stored in central SQL Server for analysis
• Load Test Monitor enables monitoring a test in progress
• Run tests from VS or Command Line• Deployment of test code, reverse deployment of logs/test results
• Parallel distributed functional test execution
• Run a large set of unit and functional tests at the same time
• Agent selection enables matrix testing
• Code Coverage
• Better quality of service: load tests are restarted if the test process exits due to a failure.
• Share test lab hardware with a team• Multiple clients can submit test runs
• Controller queues runs and manages agents
• Platform for test tools
• Extensible architecture supports third party test types
• Team Foundation Server integration
• Reporting
• Work Item tracking• Build Server
• Source Code Control

Requirements:

Test Load Agent - Controller
Processor: 2 GHz
RAM: 1 GB
Disk Space (System): 1 GBDisk Space (Install): 48 GBTest Load Agent - Agent
Processor: 2 GHz
RAM: 1 GB
Disk Space (System): 1 GB
Disk Space (Install): 5 GB Limitations:90 days trial

Supporting O/S:

Download :

File Size : 551 MB

For download Visual Studio Team System 2008 Test Load Agent click here

Sorce: Microsoft official website 

Loadrunner Interview Questions

LoadRunner interview questions

1. What is load testing? - Load testing is to test that if the application works fine with the loads that result from large number of simultaneous users, transactions and to determine weather it can handle peak usage periods.
2. What is Performance testing? - Timing for both read and update transactions should be gathered to determine whether system functions are being performed in an acceptable timeframe. This should be done standalone and then in a multi user environment to determine the effect of multiple transactions on the timing of a single transaction.
3. Did u use LoadRunner? What version? - Yes. Version 9.0. latest version is 9.5.
4. Explain the Load testing process? -
Step 1: Planning the test. Here, we develop a clearly defined test plan to ensure the test scenarios we develop will accomplish load-testing objectives.
Step 2: Creating Vusers. Here, we create Vuser scripts that contain tasks performed by each Vuser, tasks performed by Vusers as a whole, and tasks measured as transactions.
Step 3: Creating the scenario. A scenario describes the events that occur during a testing session. It includes a list of machines, scripts, and Vusers that run during the scenario. We create scenarios using LoadRunner Controller. We can create manual scenarios as well as goal-oriented scenarios. In manual scenarios, we define the number of Vusers, the load generator machines, and percentage of Vusers to be assigned to each script. For web tests, we may create a goal-oriented scenario where we define the goal that our test has to achieve. LoadRunner automatically builds a scenario for us.
Step 4: Running the scenario.
We emulate load on the server by instructing multiple Vusers to perform tasks simultaneously. Before the testing, we set the scenario configuration and scheduling. We can run the entire scenario, Vuser groups, or individual Vusers. Step 5: Monitoring the scenario.
We monitor scenario execution using the LoadRunner online runtime, transaction, system resource, Web resource, Web server resource, Web application server resource, database server resource, network delay, streaming media resource, firewall server resource, ERP server resource, and Java performance monitors. Step 6: Analyzing test results. During scenario execution, LoadRunner records the performance of the application under different loads. We use LoadRunner’s graphs and reports to analyze the application’s performance.
5. When do you do load and performance Testing? - We perform load testing once we are done with interface (GUI) testing. Modern system architectures are large and complex. Whereas single user testing primarily on functionality and user interface of a system component, application testing focuses on performance and reliability of an entire system. For example, a typical application-testing scenario might depict 1000 users logging in simultaneously to a system. This gives rise to issues such as what is the response time of the system, does it crash, will it go with different software applications and platforms, can it hold so many hundreds and thousands of users, etc. This is when we set do load and performance testing.
6. What are the components of LoadRunner? - The components of LoadRunner are The Virtual User Generator, Controller, and the Agent process, LoadRunner Analysis and Monitoring, LoadRunner Books Online.
7. What Component of LoadRunner would you use to record a Script? - The Virtual User Generator (VuGen) component is used to record a script. It enables you to develop Vuser scripts for a variety of application types and communication protocols.
8. What Component of LoadRunner would you use to play Back the script in multi user mode? - The Controller component is used to playback the script in multi-user mode. This is done during a scenario run where a vuser script is executed by a number of vusers in a group.
9. What is a rendezvous point? - You insert rendezvous points into Vuser scripts to emulate heavy user load on the server. Rendezvous points instruct Vusers to wait during test execution for multiple Vusers to arrive at a certain point, in order that they may simultaneously perform a task. For example, to emulate peak load on the bank server, you can insert a rendezvous point instructing 100 Vusers to deposit cash into their accounts at the same time.
10. What is a scenario? - A scenario defines the events that occur during each testing session. For example, a scenario defines and controls the number of users to emulate, the actions to be performed, and the machines on which the virtual users run their emulations.
11. Explain the recording mode for web Vuser script? - We use VuGen to develop a Vuser script by recording a user performing typical business processes on a client application. VuGen creates the script by recording the activity between the client and the server. For example, in web based applications, VuGen monitors the client end of the database and traces all the requests sent to, and received from, the database server. We use VuGen to: Monitor the communication between the application and the server; Generate the required function calls; and Insert the generated function calls into a Vuser script.
12. Why do you create parameters? - Parameters are like script variables. They are used to vary input to the server and to emulate real users. Different sets of data are sent to the server each time the script is run. Better simulate the usage model for more accurate testing from the Controller; one script can emulate many different users on the system.
13. What is correlation? Explain the difference between automatic correlation and manual correlation? - Correlation is used to obtain data which are unique for each run of the script and which are generated by nested queries. Correlation provides the value to avoid errors arising out of duplicate values and also optimizing the code (to avoid nested queries). Automatic correlation is where we set some rules for correlation. It can be application server specific. Here values are replaced by data which are created by these rules. In manual correlation, the value we want to correlate is scanned and create correlation is used to correlate.
14. How do you find out where correlation is required? Give few examples from your projects? - Two ways: First we can scan for correlations, and see the list of values which can be correlated. From this we can pick a value to be correlated. Secondly, we can record two scripts and compare them. We can look up the difference file to see for the values which needed to be correlated. In my project, there was a unique id developed for each customer, it was nothing but Insurance Number, it was generated automatically and it was sequential and this value was unique. I had to correlate this value, in order to avoid errors while running my script. I did using scan for correlation.
15. Where do you set automatic correlation options? - Automatic correlation from web point of view can be set in recording options and correlation tab. Here we can enable correlation for the entire script and choose either issue online messages or offline actions, where we can define rules for that correlation. Automatic correlation for database can be done using show output window and scan for correlation and picking the correlate query tab and choose which query value we want to correlate. If we know the specific value to be correlated, we just do create correlation for the value and specify how the value to be created.
16. What is a function to capture dynamic values in the web Vuser script? - Web_reg_save_param function saves dynamic data information to a parameter.
17. When do you disable log in Virtual User Generator, When do you choose standard and extended logs? - Once we debug our script and verify that it is functional, we can enable logging for errors only. When we add a script to a scenario, logging is automatically disabled. Standard Log Option: When you select
Standard log, it creates a standard log of functions and messages sent during script execution to use for debugging. Disable this option for large load testing scenarios. When you copy a script to a scenario, logging is automatically disabled Extended Log Option: Select
extended log to create an extended log, including warnings and other messages. Disable this option for large load testing scenarios. When you copy a script to a scenario, logging is automatically disabled. We can specify which additional information should be added to the extended log using the Extended log options.
18. How do you debug a LoadRunner script? - VuGen contains two options to help debug Vuser scripts-the Run Step by Step command and breakpoints. The Debug settings in the Options dialog box allow us to determine the extent of the trace to be performed during scenario execution. The debug information is written to the Output window. We can manually set the message class within your script using the lr_set_debug_message function. This is useful if we want to receive debug information about a small section of the script only.
19. How do you write user defined functions in LR? Give me few functions you wrote in your previous project? - Before we create the User Defined functions we need to create the external
library (DLL) with the function. We add this library to VuGen bin directory. Once the library is added then we assign user defined function as a parameter. The function should have the following format: __declspec (dllexport) char* (char*, char*)Examples of user defined functions are as follows:GetVersion, GetCurrentTime, GetPltform are some of the user defined functions used in my earlier project.
20. What are the changes you can make in run-time settings? - The Run Time Settings that we make are: a) Pacing - It has iteration count. b) Log - Under this we have Disable Logging Standard Log and c) Extended Think Time - In think time we have two options like Ignore think time and Replay think time. d) General - Under general tab we can set the vusers as process or as multithreading and whether each step as a transaction.
21. Where do you set Iteration for Vuser testing? - We set Iterations in the Run Time Settings of the VuGen. The navigation for this is Run time settings, Pacing tab, set number of iterations.
22. How do you perform functional testing under load? - Functionality under load can be tested by running several Vusers concurrently. By increasing the amount of Vusers, we can determine how much load the server can sustain.
23. What is Ramp up? How do you set this? - This option is used to gradually increase the amount of Vusers/load on the server. An initial value is set and a value to wait between intervals can be
specified. To set Ramp Up, go to ‘Scenario Scheduling Options’
24. What is the advantage of running the Vuser as thread? - VuGen provides the facility to use multithreading. This enables more Vusers to be run per
generator. If the Vuser is run as a process, the same driver program is loaded into memory for each Vuser, thus taking up a large amount of memory. This limits the number of Vusers that can be run on a single
generator. If the Vuser is run as a thread, only one instance of the driver program is loaded into memory for the given number of
Vusers (say 100). Each thread shares the memory of the parent driver program, thus enabling more Vusers to be run per generator.
25. If you want to stop the execution of your script on error, how do you do that? - The lr_abort function aborts the execution of a Vuser script. It instructs the Vuser to stop executing the Actions section, execute the vuser_end section and end the execution. This function is useful when you need to manually abort a script execution as a result of a specific error condition. When you end a script using this function, the Vuser is assigned the status "Stopped". For this to take effect, we have to first uncheck the “Continue on error” option in Run-Time Settings.
26. What is the relation between Response Time and Throughput? - The Throughput graph shows the amount of data in bytes that the Vusers received from the server in a second. When we compare this with the transaction response time, we will notice that as throughput decreased, the response time also decreased. Similarly, the peak throughput and highest response time would occur approximately at the same time.
27. Explain the Configuration of your systems? - The configuration of our systems refers to that of the client machines on which we run the Vusers. The configuration of any client machine includes its hardware settings, memory, operating system, software applications, development tools, etc. This system component configuration should match with the overall system configuration that would include the network infrastructure, the web server, the database server, and any other components that go with this larger system so as to achieve the load testing objectives.
28. How do you identify the performance bottlenecks? - Performance Bottlenecks can be detected by using monitors. These monitors might be application server monitors, web server monitors, database server monitors and network monitors. They help in finding out the troubled area in our scenario which causes increased response time. The measurements made are usually performance response time, throughput, hits/sec, network delay graphs, etc.
29. If web server, database and Network are all fine where could be the problem? - The problem could be in the system itself or in the application server or in the code written for the application.
30. How did you find web server related issues? - Using Web resource monitors we can find the performance of web servers. Using these monitors we can analyze throughput on the web server, number of hits per second that
occurred during scenario, the number of http responses per second, the number of downloaded pages per second.
31. How did you find database related issues? - By running “Database” monitor and help of “Data Resource Graph” we can find database related issues. E.g. You can specify the resource you want to measure on before running the controller and than you can see database related issues
32. Explain all the web recording options?
33. What is the difference between Overlay graph and Correlate graph? - Overlay Graph: It overlay the content of two graphs that shares a common x-axis. Left Y-axis on the merged graph show’s the current graph’s value & Right Y-axis show the value of Y-axis of the graph that was merged. Correlate Graph: Plot the Y-axis of two graphs against each other. The active graph’s Y-axis becomes X-axis of merged graph. Y-axis of the graph that was merged becomes merged graph’s Y-axis.
34. How did you plan the Load? What are the Criteria? - Load test is planned to decide the number of users, what kind of machines we are going to use and from where they are run. It is based on 2 important documents, Task Distribution Diagram and Transaction profile. Task Distribution Diagram gives us the information on number of users for a particular transaction and the time of the load. The peak usage and off-usage are decided from this Diagram. Transaction profile gives us the information about the transactions name and their priority levels with regard to the scenario we are deciding.
35. What does vuser_init action contain? - Vuser_init action contains procedures to login to a server.
36. What does vuser_end action contain? - Vuser_end section contains log off procedures.
37. What is think time? How do you change the threshold? - Think time is the time that a real user waits between actions. Example: When a user receives data from a server, the user may wait several seconds to review the data before responding. This delay is known as the think time. Changing the Threshold: Threshold level is the level below which the recorded think time will be ignored. The default value is five (5) seconds. We can change the think time threshold in the Recording options of the Vugen.
38. What is the difference between standard log and extended log? - The standard log sends a subset of functions and messages sent during script execution to a log. The subset depends on the Vuser type Extended log sends a detailed script execution messages to the output log. This is mainly used during debugging when we want information about: Parameter substitution. Data returned by the server. Advanced trace.
39. Explain the following functions: - lr_debug_message - The lr_debug_message function sends a debug message to the output log when the specified message class is set. lr_output_message - The lr_output_message function sends notifications to the Controller Output window and the Vuser log file. lr_error_message - The lr_error_message function sends an error message to the LoadRunner Output window. lrd_stmt - The lrd_stmt function associates a character string (usually a SQL statement) with a cursor. This function sets a SQL statement to be processed. lrd_fetch - The lrd_fetch function fetches the next row from the result set.
40. Throughput - If the throughput scales upward as time progresses and the number of Vusers increase, this indicates that the bandwidth is sufficient. If the graph were to remain relatively flat as the number of Vusers increased, it would
be reasonable to conclude that the bandwidth is constraining the volume of
data delivered.
41. Types of Goals in Goal-Oriented Scenario - Load Runner provides you with five different types of goals in a goal oriented scenario:
 The number of concurrent Vusers
 The number of hits per second
 The number of transactions per second
 The number of pages per minute
 The transaction response time that you want your scenario
42. Analysis Scenario (Bottlenecks): In Running Vuser graph correlated with the response time graph you can see that as the number of Vusers increases, the average response time of the check itinerary transaction very gradually increases. In other words, the average response time steadily increases as the load
increases. At 56 Vusers, there is a sudden, sharp increase in the average response
time. We say that the test broke the server. That is the mean time before failure (MTBF). The response time clearly began to degrade when there were more than 56 Vusers running simultaneously.
43. What is correlation? Explain the difference between automatic correlation and manual correlation? - Correlation is used to obtain data which are unique for each run of the script and which are generated by nested queries. Correlation provides the value to avoid errors arising out of duplicate values and also optimizing the code (to avoid nested queries). Automatic correlation is where we set some rules for correlation. It can be application server specific. Here values are replaced by data which are created by these rules. In manual correlation, the value we want to correlate is scanned and create correlation is used to correlate.
44. Where do you set automatic correlation options? - Automatic correlation from web point of view, can be set in recording options and correlation tab. Here we can enable correlation for the entire script and choose either issue online messages or offline actions, where we can define rules for that correlation. Automatic correlation for database, can be done using show output window and scan for correlation and picking the correlate query tab and choose which query value we want to correlate. If we know the specific value to be correlated, we just do create correlation for the value and specify how the value to be created.
45. What is a function to capture dynamic values in the web vuser script? - Web_reg_save_param function saves dynamic data information to a parameter.

Loadrunner Groups - Mail Archive

Loadrunner Groups - Mail Archive:

You can find loadrunner questions and experts answers at one place. The following sites are not only for loadrunner ...we can get more information on different technologies(ex:loadrunner,winrunner,silktest,jmeter,webload,winrunner,linux,...and many) All these questions are gathered from different groups like ., google groups,yahoo groups,....etc... and placed @ one place.


URL for mail archive : http://www.mail-archive.com

(Search 60901782 archived postings, 6527 active mailing lists)






The following site also same as above ...

URL for markmail is : http://markmail.org

(Searching 6,988 lists and 36,944,333 messages. First list started in November 1992. There are 3,794 active lists, recently accumulating 27,823 messages per day. You can browse recent emails. )










Share your valuable information to testingblog4u@gmail.com

Load and Performance Test Tools

Program testing and fault detection can be aided significantly by testing tools and debuggers. Types of testing/debug tools include features such as:


Load and Performance Test Tools :

BrowserMob - On-demand, self-service, low-cost, pay-as-you-go service enables simulation of large volumes of real browsers hitting a website. Utilizes Amazon Web Services, Selenium. Uses real browsers for each virtual user so that traffic is realistic, AJAX & Flash support is automatic. Browser screen shots of errors included in reports.

Pylot - Open source tool by Corey Goldberg for generating concurrent http loads. Define test cases in an XML file - specify requests - url, method, body/payload, etc - and verifications. Verification is by matching content to regular expressions and with HTTP status codes. HTTP and HTTPS (SSL) support. Monitor and execute test suites from GUI (wxPython), and adjust load, number of agents, request intervals, rampup time, test duration. Real-time stats and error reporting are displayed.

AppLoader - Load testing app from NRG Global for web and other applications accessible from a Windows desktop; generates load from the end user's perspective. Protocol independent and supports a wide variety of enterprise class applications. Integrates with their Chroniker monitoring suite so results of load testing can be correlated with system behavior as load is increased. Runs from Win platforms.

fwptt - Open source tool by Bogdan Damian for load testing web applications. Capabilities include handling of Ajax. Generates tests in C#. For Windows platforms

JCrawler - An open-source stress-testing tool for web apps; includes crawling/exploratory features. User can give JCrawler a set of starting URLs and it will begin crawling from that point onwards, going through any URLs it can find on its way and generating load on the web application. Load parameters (hits/sec) are configurable via central XML file; fires up as many threads as needed to keep load constant; includes self-testing unit tests. Handles http redirects and cookies; platform independent.

vPerformer - Performance and load testing tool from Verisium Inc. to assess the performance and scalability of web apps. Use recorded scripts or customized scripts using Javascript. Targeted platforms: Windows

Curl-Loader - Open-source tool written in 'C', simulating application load and behavior of tens of thousand HTTP/HTTPS and FTP/FTPS clients, each with its own source IP-address. In contrast to other tools curl-loader is using real C-written client protocol stacks, namely, HTTP and FTP stacks of libcurl and TLS/SSL of openssl. Activities of each virtual client are logged and collected statistics include information about: resolving, connection establishment, sending of requests, receiving responses, headers and data received/sent, errors from network, TLS/SSL and application (HTTP, FTP) level events and errors.

RealityLoad XF On-Demand Load Testing - An on-demand load testing service (no licenses) from Gomez.com. Leverages Gomez' peer panel, which consists of over 15,000 end-user desktop testing locations distributed across the world, to provide distributed load tests that accurately reproduce the network and latency characteristics encountered by real users in a live environment.

OpNet LoadScaler - Load test tool from OpNet Technologies Inc. Create tests without programming; generate loads against web applications, and other services including Web Services, FTP, and Email. Record end-user browser activity in the OPNET TestCreatorTM authoring environment to automatically generate test scripts in industry-standard JavaScript. Modify, extend and debug tests with the included JavaScript editor. Alternatively, drag and drop icons onto the test script tree. No knowledge of a scripting language is required to customize test scripts.

StressTester - Enterprise load and performance testing tool for web applications from Reflective Solutions Ltd. Advanced user journey modeling, scalable load, system resources monitors and results analysis. No scripting required. Suitable for any Web, JMS, IP or SQL Application. OS independent.

The Grinder - A Java-based load-testing framework freely available under a BSD-style open-source license. Orchestrate activities of a test script in many processes across many machines, using a graphical console application. Test scripts make use of client code embodied in Java plug-ins. Most users do not write plug-ins themselves, instead using one of the supplied plug-ins. Comes with a mature plug-in for testing HTTP services, as well as a tool which allows HTTP scripts to be automatically recorded.

Proxy Sniffer - Web load and stress testing tool from from Ingenieurb?vid Fischer GmbH Capabilities include: HTTP/S Web Session Recorder that can be used with any web browser; recordings can then be used to automatically create optimized Java-based load test programs; automatic protection from "false positive" results by examining actual web page content; detailed Error Analysis using saved error snapshots; real-time statistics.

Testing Master - Load test tool from Novosoft, capabilities include IP spoofing, multiple simultaneous test cases and website testing features for sites with dynamic content and secure HTTPS pages.

Funkload - Web load testing, stress testing, and functional testing tool written in Python and distributed as free software under the GNU GPL. Emulates a web browser (single-threaded) using webunit; https support; produces detailed reports in ReST, HTML, or PDF.

Avalanche - Load-testing appliance from Spirent Communications, designed to stress-test security, network, and Web application infrastructures by generating large quantities of user and network traffic. Simulates as many as two million concurrently-connected users with unique IP addresses, emulates multiple Web browsers, supports Web Services testing Supports HTTP 1.0/1.1, SSL, FTP, RTSP/ RTP, MS Win Media, SMTP, POP3, DNS, Telnet, and Video on Demand over Multicast protocols.

Loadea - Stress testing tool runs on WinXP; free evaluation version for two virtual users. Capture module provides a development environment, utilizes C# scripting and XML based data. Control module defines, schedules, and deploys tests, defines number of virtual users, etc. Analysis module analyzes results and provides reporting capabilities.

LoadManager - Load, Stress, Stability and Performance testing tool from Alvicom. Runs on all platforms supported by Eclipse and Java.

QEngine Web Performance Testing - Automated testing tool from AdventNet for performance testing (load and stress testing) of web applications and web services. For Linux and Windows.

NeoLoad - Load testing tool for web applications from Neotys with clear and intuitive graphical interface, no scripting/fast learning curve, clear and comprehensive reports and test results. Can design complex scenarios to handle real world applications. Features include data replacement, data extraction, SOAP support, system monitoring (Windows, Linux, IIS, Apache, WebLogic, Websphere...), SSL recording, PDF/HTML/Word reporting, IP spoofing, and more. Multi-platform: Windows, Linux, Solaris.

Test Complete Enterprise - Automated test tool from AutomatedQA Corp. includes web load testing capabilities.

WebPartner Test and Performance Center - Test tool from WebPartner for stress tests, load performance testing, transaction diagnostics and website monitoring of HTTP/HTTPS web transactions and XML/SOAP/WSDL web services.

QTest - Web load testing tool from Quotium Technologies SA. Capabilities include: cookies managed natively, making the script modelling phase shorter; HTML and XML parser, allowing display and retrieval of any element from a HTML page or an XML flux in test scripts; option of developing custom monitors using supplied APIs; more.

LoadDriver - Load test tool from Inforsolutions emphasizes ease of use; directly drives multiple instances of MSIE, rather than simulating browsers. Supports browser-side scripts/objects, HTTP 1.0/1.1, HTTPS, cookies, cache, Windows authentication. Tests can be scriptlessly parameterized with data from text files or custom ODBC data source, for individual userID, password, page to start, think times, data to enter, links to click, cache, initial cache state, etc.

Test Perspective Load Test - Do-it-yourself load testing service from Keynote Systems for Web applications. Utilizes Keynote's load-generating infrastructure on the Internet; conduct realistic outside-the-firewall load and stress tests to validate performance of entire Web application infrastructure.

SiteTester1 - Load test tool from Pilot Software Ltd. Allows definition of requests, jobs, procedures and tests, HTTP1.0/1.1 compatible requests, POST/GET methods, cookies, running in multi-threaded or single-threaded mode, generates various reports in HTML format, keeps and reads XML formatted files for test definitions and test logs. Requires JDK1.2 or higher.

httperf - Web server performance/benchmarking tool from HP Research Labs. Provides a flexible facility for generating various HTTP workloads and measuring server performance. Focus is not on implementing one particular benchmark but on providing a robust, high-performance, extensible tool. Available free as source code.

NetworkTester - Tool (formerly called 'NetPressure') from Agilent Technologies uses real user traffic, including DNS, HTTP, FTP, NNTP, streaming media, POP3, SMTP, NFS, CIFS, IM, etc. - through access authentication systems such as PPPOE, DHCP, 802.1X, IPsec, as necessary. Unlimited scalability; GUI-driven management station; no scripting; open API. Errors isolated and identified in real-time; traffic monitored at every step in a protocol exchange (such as time of DNS lookup, time to logon to server, etc.). All transactions logged, and detailed reporting available.

WAPT - Web load and stress testing tool from SoftLogica LLC. Handles dynamic content and HTTPS/SSL; easy to use; support for redirects and all types of proxies; clear reports and graphs.

Visual Studio Team System 2008 Test Edition - A suite of testing tools for Web applications and services that are integrated into the Microsoft Visual Studio environment. These enable testers to author, execute, and manage tests and related work items all from within Visual Studio.

OpenLoad - Affordable and completely web-based load testing tool from OpenDemand; knowledge of scripting languages not required - web-based recorder can capture and translate any user action from any website or web application. Generate up to 1000 simultaneous users with minimum hardware.

Apache JMeter - Java desktop application from the Apache Software Foundation designed to load test functional behavior and measure performance. Originally designed for testing Web Applications but has since expanded to other test functions; may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types; can make a graphical analysis of performance or test server/script/object behavior under heavy concurrent load.

TestMaker - Free open source utility maintained by PushToTest.com and Frank Cohen, for performance, scalability, and functional testing of Web application. Features test authoring of Web applications, Rich Internet Applications (RIA) using Ajax, Service Oriented Architecture, and Business Process Management environments. Integrates Selenium, soapUI, TestGen4Web, and HTMLUnit to make test development faster/easier. Repurposes tests from these tools into load and performance tests, functional tests, and business service monitors with no coding. Repurposes unit tests written in Java, Jython, JRuby, Groovy, and other dynamic scripting languages. Runs on any platform.

SiteStress - Remote, consultative load testing service by Webmetrics. Simulates end-user activity against designated websites for performance and infrastructure reliability testing. Can generate an infinitely scalable user load from GlobalWatch Network, and provide performance reporting, analysis, and optimization recommendations.

Siege - Open source stress/regression test and benchmark utility; supports basic authentication, cookies, HTTP and HTTPS protocols. Enables testing a web server with a configurable number of concurrent simulated users. Stress a single URL with a specified number of simulated users or stress multiple URL's simultaneously. Reports total number of transactions, elapsed time, bytes transferred, response time, transaction rate, concurrency, and server response. Developed by Jeffrey Fulmer, modeled in part after Lincoln Stein's torture.pl, but allows stressing many URLs simultaneously. Distributed under terms of the GPL; written in C; for UNIX and related platforms.

JBlitz - Load, performance and functional test tool from Clan Productions. Runs multiple concurrent virtual users.to simulate heavy load. Validates each response using plain text or regular expression searches, or by calling out to your own custom code. Full Java API. For testing and 'bullet-proofing' server side software - ASPs, JSPs, servlets, EJBs, Perl / PHP / C / C++ / CGI scripts etc.

WebServer Stress Tool - Web stress test tool from Paessler AG handles proxies, passwords, user agents, cookies, AAL.

Web Polygraph - Freely available benchmarking tool for caching proxies, origin server accelerators, L4/7 switches, and other Web intermediaries. Other features: for high-performance HTTP clients and servers, realistic traffic generation and content simulation, ready-to-use standard workloads, powerful domain-specific configuration language, and portable open-source implementation. C++ source available; binaries avail for Windows.

OpenSTA - 'Open System Testing Architecture' is a free, open source web load/stress testing application, licensed under the Gnu GPL. Utilizes a distributed software architecture based on CORBA. OpenSTA binaries available for Windows.

PureLoad - Java-based multi-platform performance testing and analysis tool from Minq Software. Includes 'Comparer' and 'Recorder' capabilities, dynamic input data, scenario editor/debugger, load generation for single or distributed sources.

ApacheBench - Perl API for Apache benchmarking and regression testing. Intended as foundation for a complete benchmarking and regression testing suite for transaction-based mod_perl sites. For stress-testing server while verifying correct HTTP responses. Based on the Apache 1.3.12 ab code. Available via CPAN as .tar.gz file.

Torture - Bare-bones Perl script by Lincoln Stein for testing web server speed and responsiveness and test stability and reliability of a particular Web server. Can send large amounts of random data to a server to measure speed and response time of servers, CGI scripts, etc.

WebSpray - Low-cost load testing tool from CAI Networks; includes link testing capabilities; can simulate up to 1,000 clients from a single IP address; also supports multiple IP addresses with or without aliases. For Windows.

eValid LoadTest - Web test tool from Software Research, Inc that uses a 'Test Enabled Web Browser' test engine that provides browser based 100% client side quality checking, dynamic testing, content validation, page performance tuning, and webserver loading and capacity analysis.

WebPerformance Load Tester - Load test tool emphasizing ease-of-use, from WebPerformance Inc. Supports all browsers and web servers; records and allows viewing of exact bytes flowing between browser and server; no scripting required. Modem simulation allows each virtual user to be bandwidth limited. Can automatically handle variations in session-specific items such as cookies, usernames, passwords, IP addresses, and any other parameter to simulate multiple virtual users. For Windows, Linux, Solaris, most UNIX variants.

WebSuite - A collection of load testing, capture/playback, and related tools from Technovations for performance testing of web sites. Modules include WebCorder, Load Director, Report Generator, Batch, Manager, and others. WebSizr load testing tool supports authentication, SSL, cookies, redirects. Recorded scripts can be modified manually. For Windows.

FORECAST - Load testing tool from Facilita Software for web, client-server, network, and database systems. Capabilities include proprietary, Java, or C++ scripting; windows browser or network recording/playback. Supports binary encoded data such as Adobe Flex/AMF, Serialised Java objects etc.SSL; supports NTLM, kerberos, proxies, authentication, redirects, certificates, cookies, caching, bandwidth limitation and page validation. Virtual user data can be parameterized. Works with a wide variety of platforms.

http-Load - Free load test application from ACME Labs to generate web server loads, from ACME Software. Handles HTTP and HTTPS; for Unix.

QALoad - Compuware's tool for load/stress testing of web, database, and character-based systems. Supports HTTP, SSL, SOAP, XML, Streaming Media. Works with a variety of databases, middleware, ERP.

Microsoft WCAT load test tool - Web load test tool from Microsoft for load testing of MS IIS servers; other MS stress tools also listed.

IBM Rational Performance Tester - Performance testing tool from IBM/Rational; has optional extensions to Seibel applications and SAP Solutions. Supports Windows, Linux and z/OS as distributed controller agents; provides high-level and detailed views of tests.

SilkPerformer - Enterprise-class load-testing tool from Borland (formerly Segue). Can simulate thousands of users working with multiple protocols and computing environments. Allows prediction of behavior of e-business environment before it is deployed, regardless of size and complexity.

Radview's WebLoad - Load testing tool from Radview Software, also available as part of their TestView web testing suite. Basic version available as open source, add-ons available for purchase. Capabilities include over 75 Performance Metrics; can view global or detailed account of transaction successes/failures on individual Virtual Client level, assisting in capturing intermittent errors; allows comparing of running test vs. past test metrics. Test scripting via visual tool or Javascript. Wizard for automating non-GUI-based services testing; DoS security testing.

Loadrunner - HP's (formerly Mercury's) load/stress testing tool for web and other applications; supports a wide variety of application environments, platforms, and databases. Large suite of network/app/server monitors to enable performance measurement of each tier/server/component and tracing of bottlenecks.

Post your comments to testingblog4u@gmail.com

Requirement gathering for Performance testing: Part-3

This posting explains about "Requirement gathering for Performance testing - Business Process"

Business Process:

1. What are the user initiated activities that you want to simulate in the script? Please list.
• Will you be providing the use cases?
• Will you be providing the load testing scenarios?
2. What is the user ratio for each activity?
(i.e. activity1 – 70% of load, activity2 – 15%, and activity3 – 15%)
3. Are there any business process dependencies that impact the user activities you defined above?
• Does one process need to run before another process?
• Can the activities run concurrently?
4. We generally measure each transaction sent/received. Are there other requirements that you are looking for?
• Measure each page or groups of pages that comprise a business flow?
5. What are the expected response times?
• What are the maximum acceptable response times?
6. How do you see your users accessing the application?
• Do they all log in at the same time?
• Sporadically throughout the day?
• Certain number at specified intervals?
7. What is the activity execution frequency?
• How often are the users doing each activity in an hour, a day, etc.
• How many transactions occur in an hour, a day, etc.
8. What is the expected throughput of the architecture?
• Throughput can be expressed in terms of hits per hour, day, etc.
• It can also be expressed in terms of how many bytes per second or transactions per hour.
9. Are there specific requirements for the input data?
• Data validation (credit cards, zip codes, etc.).
• Data uniqueness (you cannot enter the same data more than once).
• Is it time sensitive?
• Is the data reusable within the test?
• Is there cleanup—and how often?
10. How long will it take to obtain all input data?
• Customer records
• Logins and Passwords
11. Are there any input data dependencies?
• Is data prepping required?
• Does form input data have to be unique? (I.e. can all users have the same address?)
12. What is the data refresh strategy? Is one required?
• Does the DB need to be restored between runs?
• Is it possible to do a rollback?
13. Has disk space been evaluated to assure that there is sufficient disk space to support the high volume of users and activities? (Ex. log files, other flat files)

14. Will batch processes be executing concurrently with the performance testing? Are there any you think should be included on at least one test run?

15. Is there sufficient Database space to support high volume/users?

Related Articals:

Requirement gathering for Performance testing: Part-1

Requirement gathering for Performance testing: Part-2

post your comments to testingblog4u@gmail.com

Requirement gathering for Performance testing: Part-2

In last posting a have posted some points on "Performance Testing Requirement gathering"...

In this posting i am giving more information about "Architecture" & "Monitoring resouces" requirements...

Architecture:

1. Will functional testing be completed before performance testing begins?
2. Will all the functional testing defects of the application be fixed?
3. What are the components under test? (Provide a matrix indicating the description, make, model number, number of CPUs, and speed in MHz, IP Address, and Operating System for each applicable component.)
• Application
• Application Servers
• Web Servers
• Database Servers
• Others (Please specify)

4. Who are the persons responsible for supporting and monitoring all components under test? (Provide list of names, work and pager numbers, and include off -hour numbers, if applicable.)
• Developers
• System Administrators
• Network Administrators
• DBA’s Others

5. If the environment under test (EUT) is not the actual production environment, is the EUT identical to production?
6. Will the EUT be dedicated to performance testing during the testing period?
7. Will cache memory be turned off on servers during testing?
8. Can access to servers under test be provided for monitoring purposes in the form of rstat daemons, administrator rights, or other?
9. Are there any known network limitations on the bandwidth available to the application?
10. What are the client side components? (Java, JavaScript, VBScript, ActiveX, etc.)
• These components can affect the way we construct the script.
• There may be some Business Process logic that we have to simulate in the script.
• Does the client application depend on any locally installed components?
11. What are the communication protocol(s) between the client and the backend servers?
• Such as HTTP, FTP, TCP/IP, IIOP, GIOP, TUXEDO, MQ-Series, TIBCO, RTE, Citrix ICA, Telnet, proprietary protocols (via sockets), etc.
12. How does the client application maintain state information?
• Is it done by the backend application or the DB server store?
• Does the application track connections based on IP?
13. Is it required that the load be generated from multiple geographical regions? If yes, which?
14. Is the application customer facing? Is it accessed from outside the company firewall?
15. Is there CORBA, DCOM, or RMI in the architecture?
• If there is, where does it reside?
• We are not concerned if the client component is not directly involved.

Monitoring resouces:

16. Do you want us to monitor the system resources for each component under test?
• System Resources
• Unix And Windows
• Database Resource Monitoring
• DB2,Oracle,SQl server, Sybase
• Application Deployment Solutions
• Citrix Metaframe Server Monitor
• Network Monitoring
• Web Application Server Resources
• WebLogic Monitor, WebSphere Monitor etc.

Related Articals:
Requirement gathering for Performance testing: Part-1
Requirement gathering for Performance testing: Part-3

Post your comments to testingblog4u@gmail.com

Requirement gathering for Performance testing: Part-1

In order to define the scope of the performance/load testing, this questionnaire will be useful for gathering requirements for performance testing:

General Requirements:

1. What is the test requirement/objective in terms of load, capacity, and/or performance?
• Establish baseline results/bench marks?
• Compare previous environment bench mark results with new environment?
• Reach certain level of transactions/virtual users in a given period?

2. What is the current test window?
• Please provide the project timeline. Specify the test window.
• What limitations exist on the execution times? Hours of day (8 to 5 or off hours?)
• Will there be any Limitation on the connectivity of the application.

3. What is the “go live” date?
• If there are multiple “go live” phases, please indicate so.

4. What is the test environment for the application? What is the schedule for the application being in this environment? Which environment can we develop our tests in?
• Production
• QA / Test
• Development
• Other (Please specify)

5. How is the application accessed? Can we obtain immediate access to the application for test feasibility purposes?
• Is there a Login and password requirement?
• Is software installation involved on the client side?

6. Are the applications accessible from off-shore?
• Do we get access to the application directly or
• Do we need to access the application using CITRIX/Terminal Services clients?
• Others, Please specify

7. What is the implication of crashing the application?
• Who needs to be notified?
• How does this affect normal business operation?

8. Application knowledge
• How does the application functional knowledge training is conducted?
• Is it required to have the LR expert to visit your location to impart knowledge and study the application and the environment?

Related Articals:
Requirement gathering for Performance testing: Part-2
Requirement gathering for Performance testing: Part-3

You will get more information in feature postings :)
Please post your valuable comments to testingblog4u@gmail.com