The Library of Software Testing

How can I tell if a web page is secure?

Pavankumar Pothuraju — Thu, 23 Aug 2007 09:29:57 GMT

The below takes you ssl.com aritcle on web page secure. This brilliantly written article answers almost everything you need to know bout how to tell if a website is secure.

http://info.ssl.com/Article.aspx?id=10068

Also read the below releated articles on subject:

http://www.iisfaq.com/default.aspx?View=P20&P=145

http://www.lloydstsb.com/security/web_page_security.asp [click 'check a site certificate' link at the end of description]

Below article explains the process of procuring and installing a SSL server certificate [article focus on SSL.com products, but worth to read as the process should be common to all vendors]

http://info.ssl.com/article.aspx?id=10694

Cheers

What are Load, Stress, Volume, etc. testing?

Pavankumar Pothuraju — Wed, 16 Nov 2005 20:14:00 GMT

- Volume testing is a way to test functionality.

- Stress testing is a way to test reliability.

- Load testing is a way to test performance.

Testing an application under heavy but expected loads is known as load
testing. It generally refers to the practice of modeling the expected usage of a software program by simulating multiple users accessing the system's services concurrently. As such, load testing is most relevant for a multi-user system, often one built using a client/server model, such as a web server tested under a range of loads to determine at what point the system's response time degrades or fails. Although you could perform a load test on a word processor by or graphics editor forcing it read in an extremely large document; on a financial package by forcing to generate a report based on several years' worth of data, etc.

When the load placed on the system is accelerated beyond normal usage
patterns, in order to test the system's response at unusually high or peak loads, it is known as stress testing. Stress testing is often incorrectly used interchangeably with load and performance testing.

In a stress test, the load is usually so great that error conditions are the expected result, although there is a grey area between the two domains and no clear boundary exists where you could say that an activity ceases to be a load test and becomes a stress test.

There is little agreement on what the specific goals of load testing are. The term is often used synonymously with performance testing, reliability testing, and volume testing.

Performance testing is usually performed to determine how fast some aspect of a system performs under a particular workload. It can serve different purposes: to demonstrate that the system meets performance criteria; to compare two systems to find which performs better; to measure what parts of the system or workload cause the system to perform badly.

In the diagnostic case, software engineers use tools such as profilers to measure what parts of a device or software contribute most to the poor performance. In performance testing, it is often crucial (and often difficult to arrange) for the test conditions to be similar to the expected actual use.

A reliability test determines how likely a piece of hardware (or sometimes software) is to fail. It is part of reliability theory, used originally as a tool to help nineteenth century insurance companies compute profitable rates to charge their customers. Statistical models appropriate for any of these are generically called "time-to-event" models. Death or failure is called an "event", and the goal is to project or forecast the rate of events for a given population or probability of an event for an individual.

Volume testing is used to determine if the system under test can handle the required amounts of data, user requests, etc.

Soak testing occurs when running a system at normal to high levels of load for prolonged periods of time. A soak test would normally execute several times more transactions in an entire day than would be expected in a busy day. This should identify any performance problems that appear after a large number of transactions have been executed.

For some of the basics on this type of testing, please read the following series:

Web Page Response Time 101
http://www.stickyminds.com/r.asp?F=DART_5030

Web Load Test Planning
http://www.stickyminds.com/r.asp?F=DART_5084

The Science and Art of Web Site Load Testing
http://www.stickyminds.com/r.asp?F=DART_1939

Getting Started : Stressing Web Applications Stress Early -- Stress Often
http://www.amibug.com/getinfo.shtm?present=webstress

http://www.perftestplus.com
http://www.performancetester.com

HTTP Error Codes

Pavankumar Pothuraju — Thu, 06 Oct 2005 14:39:00 GMT

HTTP Error Codes
Information Codes
Error #	Error Code	Description
100	Continue	The request was successful. The process can now continue.
101	Switching Protocols	The request for the server to switch protocols was accepted, such as a switch from ftp to http.

Success Codes
Error #	Error Code	Description
200	OK	The item requested of the server is available (keep in mind, available, not accepted or completed).
201	Created	A new address has been created through the use of form posting, perl, cgi, etc.
202	Accepted	The request has been accepted (keep in mind that the request has been accepted, not completed)
203	Non-Authoritative Information	The information received is not from the server that the information was requested from, but from another source.
204	No Content	There was no content to be given for the request. For instance if you click on a hyperlink, imagemap, or button that isn't linked to anything or doesn't do anything.
205	Reset Content	A script has reset the displayed content.
206	Partial Content	Only partial content has been displayed. This could be due to bandwidth, poor caching, bad html, or other reasons.

Redirection Codes
Error #	Error Code	Description
300	Multiple choices	You will either get a choice of pages or an error message when this occurs. The address is actually pointing to two multiple files and/or locations.
301	Moved Permanently	The requested page has been permanently moved. The server will automatically redirect you to the new location.
302	Found	The requested page has been temporarily moved. The server will automatically redirect you to the new location.
303	See Other	The requested data is stored in an alternate location and the GET method will be used to retrieve the data. If the actual error is returned then this may be due to a web server misconfiguration.
304	Not Modified	The requested data has not been modified since the last request.
305	Use Proxy	The requested data may only be accessed via the use of a proxy server.
307	Temporary Redirect	The requested page has been moved. The server will automatically redirect you to the new location. Unlike Error 301 and 302 however, the server has not specified whether the move is temporary or permanent.

Client Error Codes
Error #	Error Code	Description
400	Bad Request	The request was denied due to a syntax error in the request.
401	Unauthorized	Your IP address or the username/password you entered were not correct. Your request was denied as you have no permission to access the data.
402	Payment Required	The data is not accessible at the time. The owner of the space has not yet payed their service provider.
403	Forbidden	Your IP address or the username/password you entered were not correct. Your request was denied as you have no permission to access the data. OR The server was unable to serve the data that was requested.
404	Not Found	The document that has been requested either no longer exists, or has never existed on the server.
405	Method Not Allowed	The document that has been requested either no longer exists, or has never existed on the server.
406	Not Acceptable	The client (webbrowser) does not accept the document format. The formats that may be specified not to accept are charset, encoding, certain file types, languages, or ranges.
407	Proxy Authentication Required	The browser has not been authenticated on the required proxy server to access the data. This error is probably most commonly returned by content filters/parental controls.
408	Request Timeout	The server has closed the socket due to communications between the client and server taking too long. This could be due to server load, bandwidth issues, the client being disconnected from the internet, etc.
409	Conflict	Too many requests for the same file at one time. OR There is a conflict with an established software rule. (ie: you are trying to copy over a file with an older version, or you do not have permissions to delete a file) OR This could be caused by a DNS issue
410	Gone	This is like a 404 error in that the document requested is not on the server, however this differs in that the server 'knows' that the file used to be there and 'believes' that the file may be back, so it returns 410 rather 404.
411	Length Required	When trying to send a document to the server the server did not recieve a Content-Length specification in the header.
412	Precondition Failed	A precondition setting required by the client or server has not been met.
413	Request Entity Too Large	The process is too large to process. (ie: a file you are trying to upload is too large to fit on the server, or a webpage you are trying to download is too large for the server to process)
414	Request-URI Too Large	The URL requested is simply too long. It is most likely more than 1024, 2048, or 4096 characters in length.
415	Unsupported Media Type	This usually occurs if the server does not support the type of media the client is requesting. (ie: the server does not support streaming media, but streaming media is on the server and the client is attempting to access it)
416	Requested Range Not Satisfiable	The client request included a range for acceptable file size, however the document requested did not fit into that range.
417	Expectation Failed	The client's expect header requested certain server behaviors that the server could not perform.

Server Error Codes
Error #	Error Code	Description
500	Internal Server Error	The server encountered an error. This is most often caused by a scripting problem, a failed database access attempt, or other similar reasons.
501	Not Implemented	The method you are using to access the document can not be performed by the server. Possible methods include: CONNECT DELETE GET HEAD OPTIONS POST PUT TRACE
502	Bad Gateway	The document requested resides on a 3rd party server and the original server received an error from the 3rd party server.
503	Service Unavailable	The server is overloaded or down for maintenance and due to this was unable to process the client request.
504	Gateway Timeout	Most likely the client has lost connectivity (disconnected from the internet) or the cleint's host is having technical difficulties. This could also mean that a server that allows access to the requested server is down, having bandwidth/load issues, or otherwise unavailable.
505	HTTP version not supported	The server does not support the HTTP version used by the client. (This usually occurs if the server is using an OLDER version of HTTP than the client.)

Computer Bug

Pavankumar Pothuraju — Mon, 26 Sep 2005 14:41:00 GMT

Computer Bug

http://en.wikipedia.org/wiki/Software_bug

Bug Tracking and Reporting

http://www.stickyminds.com/r.asp?F=DART_5898

http://www.stickyminds.com/r.asp?F=DART_5043

http://www.cs.uwm.edu/classes/cs657/cs657-7/library/writingEffectiveDefectReports.pdf

http://www.daveeaton.com/scm/PMTools.html

http://testingfaqs.org/t-track.html

Testing Client/Server Systems

Pavankumar Pothuraju — Thu, 21 Apr 2005 02:35:00 GMT

Courtesy: Paul Gerrard, Systeme Evolutif Ltd. www.evolutif.co.uk

What's different about testing client/server?

Client/server architectures allow complex systems to be assembled from components. However, multiple operating systems, changing technologies and greater architectural complexity make integration more difficult. Risks such as poor reliability, performance, configuration management, security and other non-functional issues are more prominent. None of the risks in client/server are new, but there has been a change in emphasis. Since its purpose is to address risk, the emphasis of testing in client/server must change.

The complexity of client/server also makes testing more difficult. Complex systems can be delivered faster because they are assembled from bought-in components. Testing is often estimated to be in proportion with the development cost but in a typical client/server system, the development cost may be small compared with the overall cost of the system. The end-result in such projects, is that testers are presented with more complex systems, but the time left for testing is squeezed. In client/server, testers often have to do MORE WITH LESS!

Client/server test process

A client/server test strategy must identify the risks of concern and define a test process that ensures these risks are addressed. It is axiomatic that a problem is cheaper to fix if identified early, so the test process should be aligned very closely to the development process. Testing of a deliverable should occur as soon as possible after it has been built.

Functionality is usually tested in three stages: a unit or component test, often performed by programmers; a system test, of the complete system in a controlled environment performed by a dedicated test resource; finally, an acceptance test under close to production conditions, often performed by users. The objectives, techniques and responsibility for these three test stages are directly comparable to test stages in more traditional host-based systems.

The changed emphasis in testing client/server is associated with integration and non-functional testing.

Integration is a big issue because client/server systems are usually assembled from around twelve components (for a simple 2-Tier system) to perhaps twenty components for a complex architecture. These components are usually sourced from multiple suppliers. Although standards are emerging, client/server architectures often use components which have never been used before in combination.

The sheer number of interfaces involved make interface problems and inter-component conflicts more likely. Assumptions made by developers of one component from one supplier may contradict assumptions made by another developer. These problems may be encountered for the first time ever in your installation. Getting suppliers to take such problems seriously may be difficult because in client/server, it is the system integrator who takes responsibility for integration testing.

Performance consistently presents a problem in client/server. `Intelligent clients' may call for and process large volumes of data across networks. The amount of code exercised in large numbers of architectural layers may be substantial. Delays between distributed processes may be only ten or twenty milliseconds, but when a transaction requires hundreds of network messages, delays can be considerable.

Other non-functional issues such as security, backup and recovery and system administration also present risks. What was taken for granted in a mainframe environment often presents problems in client/server.

The test strategy must address all these risks, but testing does not happen `at the end'. Testing occurs at all stages and includes reviews, walkthroughs and inspections. Developers should be responsible for the products they deliver and should test their own code. System tests should cover non-functional areas as well as the functionality. The overarching principle is test early, test often.

Managing testing

To ensure testing gets attention, define testing as a specific activity within the development phase. Instil a regime that says components are delivered and acceptable only when tested without error and signed off.

Promote better test practices within the development team. Developers are often reluctant (or incapable) of testing their own code, so implement a `buddy scheme', where pairs of programmers test each others code. If practical, put testers into the development team to obtain the benefits of independence.

When a project moves from the development phase into system and acceptance testing, the nature of the project changes. Incident logging, management and reporting become the project manager's main method of project control. Progress is measured by tests performed, errors detected, fixed and retested. A simple PC database is usually adequate for recording incidents and providing useful progress reports and statistics.

Without a strong configuration management and change control regime, systematic testing can be ineffective. Complexity makes this more important, but also more difficult. Ensure a definitive inventory of components exists, system builds are automated where possible and releases are base-levelled.

Test automation

If you are not planning to conduct formal, documented functional tests, do not buy a tool to `do regression testing'. It will probably be a distraction, and lose you both time and money. If you do intend to implement regression tests, formulate a highly selective and economically viable regression test policy and only attempt to create automated regression tests when the software is sufficiently stable.

Integration and interface testing using test drivers or harnesses should be encouraged. Given test running and dynamic analysis tools, your developers can usually build test harnesses quickly. Where object-oriented languages, such as C++ are used, dynamic analysis tools are essential to detect memory leaks and give programmers invaluable insights into the behaviour of their code. Consider developing extended or `soak' tests of components where bespoke, low level code has been written.

Architectural components should be load and stress tested individually and as a complete system if resilience or performance are of concern. Whether you adopt a top-down, bottom-up or other integration strategy, the principle is to gain confidence in one build configuration, before integrating the next components for test.

System performance testing requires considerable tool support. Load generation, response time measurement, server, database and client monitoring and performance analysis may all be required. Although several vendors offer sophisticated test running tools with load regulation and performance analysis facilities, there is no one solution available. A combination of proprietary test tools, in-house utilities and innovation are usually needed to build, execute and analyse comprehensive tests.

Performance testing and tuning tends to follow three stages. Early tests tend to cause failures in the system caused by bugs or gross mis-setting of system parameters. In stage two, application design errors, such as inefficient SQL are identified. Finally, when the system is reliable and the application tuned, the database and server operating system parameters can then be tuned to achieve optimal performance.

Testing skills

Testing is a skill which requires a disciplined, systematic approach. Few developers, testers or users have ever received formal training in testing. Training helps testers not only to do better testing but helps them to be more motivated. Users, in particular, can benefit from testing training because they will better understand how to organise themselves, to prepare more comprehensive tests and be more critical of the systems they are expected to accept. If RAD is used, responsibility for testing shifts from the developer to the user, so user testing skills are essential.

Defect Severity and Defet Priority

Pavankumar Pothuraju — Sat, 22 Jan 2005 20:27:00 GMT

This document defines the defect Severity scale for determining defect criticality and the associated defect Priority levels to be assigned to errors found in software. It is a scale which can be easily adapted to other automated test management tools.

ANSI/IEEE Std 729-1983 Glossary of Software Engineering Terminology defines Criticality as,

"A classification of a software error or fault based on an evaluation of the degree of impact that error or fault on the development or operation of a system (often used to determine whether or when a fault will be corrected)."

The severity framework for assigning defect criticality that has proven most useful in actual testing practice is a five level scale. The criticality associated with each level is based on the answers to several questions.

First, it must be determined if the defect resulted in a system failure. ANSI/IEEE Std 729-1983 defines a failure as,

"The termination of the ability of a functional unit to perform its required function."

Second, the probability of failure recovery must be determined. ANSI/IEEE 729-1983 defines failure recovery as,

"The return of a system to a reliable operating state after failure."

Third, it must be determined if the system can do this on its own or if remedial measures must be implemented in order to return the system to reliable operation.

Fourth, it must be determined if the system can operate reliably with the defect present if it is not manifested as a failure.

Fifth, it must be determined if the defect should or should not be repaired.

The following five level scale of defect criticality addresses the these questions.

The five Levels are:

1. Critical

2. Major

3. Average

4. Minor

5. Exception

1. Critical - The defect results in the failure of the complete software system, of a subsystem, or of a software unit (program or module) within the system.

2. Major - The defect results in the failure of the complete software system, of a subsystem, or of a software unit (program or module) within the system. There is no way to make the failed component(s), however, there are acceptable processing alternatives which will yield the desired result.

3. Average - The defect does not result in a failure, but causes the system to produce incorrect, incomplete, or inconsistent results, or the defect impairs the systems usability.

4. Minor - The defect does not cause a failure, does not impair usability, and the desired processing results are easily obtained by working around the defect.

5. Exception - The defect is the result of non-conformance to a standard, is related to the aesthetics of the system, or is a request for an enhancement. Defects at this level may be deferred or even ignored.

In addition to the defect severity level defined above, defect priority level can be used with severity categories to determine the immediacy of repair. A five repair priority scale has also be used in common testing practice. The levels are:

1. Resolve Immediately

2. Give High Attention

3. Normal Queue

4. Low Priority

5. Defer

1. Resolve Immediately - Further development and/or testing cannot occur until the defect has been repaired. The system cannot be used until the repair has been effected.

2. Give High Attention - The defect must be resolved as soon as possible because it is impairing development/and or testing activities. System use will be severely affected until the defect is fixed.

3. Normal Queue - The defect should be resolved in the normal course of development activities. It can wait until a new build or version is created.

4. Low Priority - The defect is an irritant which should be repaired but which can be repaired after more serious defect have been fixed.

5. Defer - The defect repair can be put of indefinitely. It can be resolved in a future major system revision or not resolved at all.

Professional Tester - Magazine for testing professionals

Pavankumar Pothuraju — Tue, 29 Mar 2005 21:01:00 GMT

Professional Tester is a magazine for testing professionals. Anyone can subscribe to this magazine and receive by email. Articles from the January 2004 are available for DOWNLOAD

Some good links on performance testing (Load testing)

Pavankumar Pothuraju — Tue, 29 Mar 2005 20:29:00 GMT

1) http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vsent7/html/vxcontestingforperformance.asp

2) http://msdn.microsoft.com/msdnmag/issues/03/01/LoadTesting/

3) http://www.softwaretesting.nildram.co.uk/cont383.htm (Good site, refer for other testing concepts too)

4) http://www.theserverside.com/news/thread.tss?thread_id=3396 (Performance Testing with the MS Web Application Stress Tool)

5) http://www.theserverside.com/news/thread.tss?thread_id=23988 (Performance Testing your Distributed J2EE Applications)

6) http://www.theserverside.com/articles/article.tss?l=Tips-On-Performance-Testing-And-Optimization (Tips on Performance Testing and Optimization)

Requirements Traceability Matrix

Pavankumar Pothuraju — Fri, 10 Dec 2004 17:50:00 GMT

Project Name

Requirements Traceability Matrix

Unique No.

Requirement

Source of Requirement

Software Reqs. Spec / Functional Req. Doc.

Design Spec.

Program Module

Test Spec.

Test Case(s)

Successful Test Verification

Modification of Requirement

Remarks

Objective 1:

Description of Matrix Fields

Develop a matrix to trace the requirements back to the project objectives identified in the Project Plan and forward through the remainder of the project life cycle stages. Place a copy of the matrix in the Project File. Expand the matrix in each stage to show traceability of work products to the requirements and vice versa. The requirements traceability matrix should contain the following fields:

A unique identification number containing the general category of the requirement (e.g., SYSADM) and a number assigned in ascending order (e.g., 1.0; 1.1; 1.2).
The requirement statement.
Requirement source (Conference; Configuration Control Board; Task Assignment, etc.).
Software Requirements Specification/Functional Requirements Document paragraph number containing the requirement.
Design Specification paragraph number containing the requirement.
Program Module containing the requirement.
Test Specification containing the requirement test.
Test Case number(s) where requirement is to be tested (optional).
Verification of successful testing of requirements.
Modification field. If requirement was changed, eliminated, or replaced, indicate disposition and authority for modification.
Remarks.