Manual Testing Course-Day 16

Error, Defect, Fault, Bug and failure

Most of these terms- error, defect, fault, failure and bugs are used interchangeably but there is difference between them. Some of these terms are very much different from others.

Error

An error is a mistake made by human that leads to discrepancy between the actual and the expected result.

Defect

A defect is a problem in the functioning of a software system during testing. ISTQB defines a defect as “A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g., an incorrect statement or data definition.”

Fault

A fault is an incorrect step, process or data definition in a software product.

Bug

A bug is a flaw in a software system that causes the system to behave in an unintended manner.

Failure

A failure is the inability of a software system to perform its operations within the specified performance benchmark. As per ISTQB, “a defect, if encountered during execution, may cause a failure of the component or system”.

So, we can say that a mistake made by humans during coding is called error, an error found during the testing phase is called a defect, a defect to be resolved by the development team is called a bug and when a build does not meet its specifications then it is termed as failure.

Difference between Assert and Verify

Both Assert and Verify statements are used in the test suites for adding validations to the test methods. Testing frameworks like TestNG and JUnit are used with Selenium to provide assertions.
Coming to the difference between Assert and Verify commands. In the case of the “Assert” command, as soon as the validation fails the execution of that particular test method is stopped. Following that the test method is marked as failed.
Whereas, in the case of “Verify”, the test method continues execution even after the failure of an assertion statement. Although the test method will still be marked as failed but the remaining statements of the test method will be executed normally.
In TestNG, the Verify functionality is provided by means of the Soft Assertions or using SoftAssert class.
Now, let’s get deeper into Assert and Verify (or soft assert).
Assert

We use Assert when we have to validate critical functionality. If the feature fails then this makes the execution of further statements irrelevant. Hence, the test method is aborted as soon as failure occurs.

Example

@Test
public void assertionTest(){
   
   //Assertion Passing
   Assert.assertTrue(1+2 == 3);
   System.out.println("Passing 1");
   
   //Assertion failing
   Assert.fail("Failing second assertion");
   System.out.println("Failing 2");
}

Output

Passing 1
FAILED: assertionTest
java.lang.AssertionError: Failing second assertion

Here, we can observe that only the text “Passing 1” gets printed. The second assertion aborts the test method as it fails to prevent further statements from getting executed.

Verify

At times, we might require the test method to continue execution even after the failure of the assertion statements. In TestNG, Verify is implemented using SoftAssert class.
In the case of SoftAssert, all the statements in the test method are executed (including multiple assertions). Once, all the statements are executed, the test results are collated based on the assertion results. And then the tests are marked as passed or fail.

Example

@Test
public void softAssertionTest(){
   
   //Creating softAssert object
   SoftAssert softAssert = new SoftAssert();
   
   //Assertion failing
   softAssert.fail("Failing first assertion");
   System.out.println("Failing 1");
   
   //Assertion failing
   softAssert.fail("Failing second assertion");
   System.out.println("Failing 2");

   //Collates the assertion results and marks test as pass or fail
   softAssert.assertAll();
}

Output

Failing 1
Failing 2
FAILED: softAssertionTest
java.lang.AssertionError: The following asserts failed:
	Failing first assertion,
	Failing second assertion

Here, we can see that even though both the test methods are bound to fail. Still, the test continues to execute.

Difference Between Web Service and Website

A Web service is a web application component that uses a standardized format like XML to interact with other web applications over the internet. 
Whereas a website is a collection of webpages put together which are accessed by a web browser.

Web Service	Website
A web service doesn’t have a user interface.	A website has a user interface or GUI.
Web services are meant for other applications to be interacted with over the internet.	Websites are meant for use by humans.
Web services are platform independent as they use open protocols	Websites are cross-platform as they require tweaking to operate on different browsers, operating systems, etc.
Web services are accessed by HTTP methods – GET, POST, PUT, DELETE, etc.	Websites are accessed by using their GUI components – buttons, text boxes, forms, etc.
E.g. Google maps API is a web service that can be used by websites to display Maps by passing coordinates to it.	E.g. www.facebook.com is a website that has a collection of related web pages.

Difference between Web Application and Desktop Application

As we all are aware that a web application is an application stored at a remote server that requires a web browser as a client to run and internet for data/resource transfer.
Whereas, a desktop application is an application designed to serve standalone machines that don’t require the internet for its operation.

Web Applications	Desktop Applications
Deployment and up-gradation for a web-based application require deployment on a single set of server machines.	Deployment and any up-gradation/patch are done on individual client machines separately.
Web applications can be accessed from anywhere, so there is no location constraint.	As desktop are confined to a standalone machine, so they can be only accessed from the machines they are deployed in.
Web applications are platform-independent, they can work in different types of platforms with the only requirement of a web browser.	Desktop applications need to be developed separately for different platform machines.
Web applications are at higher security risks as they are inherently designed to increase accessibility.	Desktop applications, on the other hand, have better authorization and administrators have better control, hence more secure.
Web applications rely heavily on internet connectivity, for there operation.	Desktop applications don’t require the internet for their operations. Some applications just require internet connectivity at the time of updations.

REST vs SOAP 

Difference between REST and SOAP

REST Web Service	SOAP Web Service
REST is an architectural style, any web service following REST architecture is called RESTFul Web Service.	SOAP is a protocol, a set of rules which must be followed while creating a web service
Doesn’t impose any constraint in data format, supports multiple data formats – JSON, XML, CSV etc.	Supports XML message format only
REST resources are exposed by the service URI and HTTP Verbs- GET, PUT, POST and DELETE	SOAP exposes its named methods to be consumed by a client. For example – a SOAP web service “WeatherWS” exposes its various operations as methods like – GetWeatherInformationResponse, GetCityForecastByZIP, WeatherDescription etc.
REST is considered light weight and faster(no XML parsing is required)	Slower than REST because of the use of verbose XML format. But it is considered more secure. This is because it uses WS-security, a component of Web Service Specification for improving web service security.
REST APIs consume less bandwidth. This is because the request doesn’t require SOAP headers in every message. REST make use of HTTP headers for any meta information	SOAP Web services use SOAP headers with their requests. These headers contain meta/additional information about the request
RESTful web services are easier and faster to create	SOAP web service follow a contract and are preferred for implementing complex logic
Last difference, REST stands for Representational State Transfer	SOAP stands for Simple Object Access Protocol

Difference Between Agile and Scrum methodology

.

Agile is a software development methodology based on the iterative and incremental approach in which small incremental builds are delivered in multiple iterations to the customer for their feedback. So, basically, it is an approach or a fundamental principle for software development.

Whereas, Scrum is one of the implementations of agile methodology in which incremental builds are delivered to the customer in every one to three weeks time interval called a sprint.

In short, agile and scrum are not comparable. Agile is an approach and scrum is its implementation. Let’s see how we can map the principles of agile with scrum-

• Agile involves collaborations and face-to-face interactions among the members of different cross-functional teams. In scrum, this collaboration is achieved in daily stand up meetings with fixed roles assigned to individuals – scrum master, product owner, and team members.
  
• Agile requires frequent delivery to the end-user for their feedback. In scrum, after each sprint, a build is delivered to the client for their incremental feedback.
  
• One of the major principles of agile methodology is emphasis on quality of the product. In scrum, this is achieved by sprint review and retrospective meetings. In these meetings, after the sprint end, all the members of the team discuss the things that went well and things that didn’t go well during the sprint. The strategies to remove the shortcomings in the sprint and improvement of the overall process are decided in these meetings.
  

Difference Between Test Plan and Test Strategy

Test plan and test strategy are 2 of the most confusing terms in software testing. Hence, the difference between test plan and test strategy is one of the most common testing interview questions.

In this article, we will study the key differences between the two. Let’s start with quick formal definitions first.

A test plan is a formal document derived from requirement documents. It describes in detail, the scope of testing and the different activities performed during testing.

Whereas, a test strategy is a high-level document describing the way testing will be carried out in an organization. It basically aims at providing a systematic approach to the software testing process.

Test Plan vs Test Strategy

Test Plan	Test Strategy
A test plan describes in detail the scope of testing and the different activities performed in testing.	A test strategy is a high-level document containing some guidelines about the way testing will be carried out.
A test plan is specific to a particular project.	A test strategy is usually for a complete organization.
It describes the whole testing activities in detail – the techniques used, schedule, resources, etc.	It describes the high-level test design techniques to be used, environment specifications, etc.
It is prepared by the test lead or test manager.	It is generally prepared by the project manager.
A test plan document includes components like – features to be tested, components not to be tested, approach to testing, pass-fail criteria, test deliverables, estimates, assumptions, etc.	A test strategy document includes – scope, test approach, testing tools, test environment specifications, release control, risk analysis, etc.
Test plans can be changed or updated.	Test strategy is usually not changed.
It is about the details and specifics.	It is more about general approaches and methodologies.

Difference Between Test Case and Test Scenario

A test case is a set of conditions for evaluating a particular feature of a software product to determine its compliance with the business requirements.
Whereas, a test scenario is generally a one-line statement describing a feature of the application to be tested. It is used for end to end testing of a feature and is generally derived from the use cases.

Let’s now see the difference between the two –

Test Case	Test Scenario
A test case contains clearly defined test steps for testing a feature of an application.	A test scenario contains a high level documentation, describing an end to end functionality to be tested.
Test cases focus on “what to test” and “how to test”.	Test scenarios just focus on “what to test”.
Test cases have clearly defined step, pre-requisites, expected results etc. Hence, there is no ambiguity.	Test scenarios are generally one-liner. Hence, there is always possibility of ambiguity during testing.
Test cases can be derived from test scenarios and have many to one relationship with them.	Test scenarios are derived from use cases.
Test cases are efficient in exhaustive testing of application.	Test scenarios are beneficial in quick testing of end to end functionality of the application.
More resources are required for documentation and execution of test cases.	Relatively less time and resources are required for creating and testing using scenarios.

Waterfall vs Agile 

Waterfall Methodology

The Waterfall methodology is a software development methodology that is based on the sequential-linear approach of software development. In the case of the Waterfall model, all the software development activities are divided into different phases namely – Requirement Gathering & Analysis, System Design, Implementation, Testing, Deployment, and Maintenance. All these phases are performed sequentially in the given order.

Agile Methodology

Agile Methodology is based on an incremental-iterative approach where the requirements of the complex projects are divided into smaller iterations involving collaboration between the different stakeholders of the project. In each iteration, a deliverable is developed, tested, and demonstrated to the client.

Difference between Agile and Waterfall

Now, let’s see – What differentiates Agile from the Waterfall methodology?

Waterfall	Agile
The waterfall methodology is sequential and linear.	Agile methodology is incremental and iterative.
Requirements have to be frozen at the beginning of SDLC.	Requirements are expected to change and changes are incorporated at any point.
The working model of software is delivered at the later phases of SDLC.	The working model is delivered during the initial phases and successive iteration of the model is delivered to the client for feedback.
It is difficult to scale-up projects based on waterfall methodology.	Scaling up of products is easy because of the iterative approach.
Customers or end-user doesn’t have a say after the requirements are frozen during the initial phases. They only get to know the product once it is built completely.	Frequent customer interaction and feedbacks are involved in agile methodology.
Waterfall requires formalized documentations.	In agile documentation is often neglected and a working prototype serves as the basis for customer evaluation and feedback.
Testing is performed once the software is built.	Continuous testing is performed during each iteration.

Key Differences Agile vs Waterfall

1. Waterfall follows a sequential approach while Agile follows an incremental approach to software development.

2. Requirements are required to be frozen before development whereas requirements are expected to get changed during the course of development.

3. Testing is performed after the development phase in Waterfall while it can be performed concurrently in the case of Agile.

4. The project is divided into different phases in the Waterfall model whereas the same is divided into different sprints in the case of Agile.

5. Waterfall model brings a project mindset with a focus on successful project delivery whereas Agile brings a product mindset with a focus on product and customer satisfaction.

Choosing between Agile and Waterfall for your Project

There are different factors like customer availability, team size, the anticipation of change, etc which should be considered while selecting the right methodology for your project.

Difference between SDLC & STLC

What is SDLC?

SDLC stands for Software Development Life Cycle. It describes the various phases involved in the software development process. With the help of SDLC, we can create software applications in a well-defined and systematic way.

The different phases of the Software Development Life Cycle are-

• Requirement Gathering and Analysis – In this phase, all the requirements are gathered and analyzed for their feasibility.
• Designing – In this phase, the requirement specifications are converted into design specifications.
• Coding/Implementation – Actual coding is done here.
• Testing – This phase involves testing of the software product.
• Deployment – The software is deployed to production for the end-user.
• Maintenance – Due to change in the environment and for continuous improvement maintenance is required.
  
  

What is STLC?

STLC stands for the Software testing life cycle. It refers to all these activities performed during the testing of a software product. Basically, it provides a sequence of activities performed to ensure the quality of the software application.

The different phases of Software Testing Life Cycle are-

• Requirement Analysis – In this phase, the high-level analysis of the requirements is done.
• Test Planning – In this phase, a test strategy and approach is defined.
• Test Case Development – The test cases are created in this phase.
• Test Environment Setup – Here, the test environment is created in which the test execution will be performed.
• Test Execution – Test cases are executed and defects are logged for the failed tests.
• Exit Criteria Evaluation and Reporting – Based on the agreed-upon exit criteria, the testing activities are marked as complete.
• Test Closure – A test closure document is prepared which contains all the testing activities performed and the bugs found. This phase marks the formal closure of the testing phase.
  
  

Mapping of SDLC and STLC Phases

As we know that development and testing are carried out parallelly. So, now let’s see the mapping between the phases of SDLC and STLC-

SDLC	STLC
Requirements Gathering	Requirement Analysis- The requirements documents are analyzed and validated and the scope of testing is defined.
Designing	Test Planning, Analysis, and Design- Defining test specifications, analysis, and reviewing of requirement documents, risk analysis reports, and other design specifications.
Coding	Test Case Development, Test Environment Setup- Involves the test case creation, test data and environment setup.
Testing	Test Execution- Involves manual and automated test case execution and result logging.
Deployment	Exit Criteria Evaluation, Reporting, and Test Closure- Involves checking if all the project deliverables are delivered, archiving the test ware, test environment, and documenting the learnings.
Maintenance	Maintenance testing- Testing required after an upgrade, correction, or any change in the software under maintenance.

Difference Between QA and QC

QA	QC
QA stands for Quality Assurance.	QC stands for Quality Control.
QA is the set of activities using which we analyze the processes used in software development.	QC is the set of activities using which we analyze the quality of the product build.
It is a static process of analyzing the documents and not the actual end product.	It involves dynamic testing of a software product by running it.
Verification comes under QA.	Validation comes under QC.
Answers the question – “Are we building the product right?”	Answers the question – “Are we building the right product?”
It is a preventive measure as it identifies the weakness in the process to build software to prevent defects.	It is a corrective measure as it tests the built product to find defects.
It involves activitie like document review, test case review, walkthroughs, inspection etc.	It involves activites like functional testing, automation testing etc.
Carrying out QA activities is the responsibilty of whole team involved in Software development Life Cycle(SDLC).	Carrying out QC activities is the responsibility of testing team, involved in the Software Testing Life Cycle(STLC).

Difference between Verification and Validation

Verification and validation are the two activities/phases that together lead to the whole software testing. Let’s see the difference between the two-

Verification	Validation
Verification is the process of evaluating the artifacts of software development in order to ensure that the product being developed will comply to the standards.	Validation is the process of validating that the developed software product conforms to the specified business requirements.
It is a static process of analysing the documents and not the actual end product.	It involves dynamic testing of a software product by running it.
Verification is a process oriented approach.	Validation is a product-oriented approach.
Answers the question – “Are we building the product right?”	Answers the question – “Are we building the right product?”
Errors found during verification require lesser cost/resources to get fixed as compared to be found during the validation phase.	Errors found during validation require more cost/resources. Later the error is discovered higher is the cost to fix it.
It involves activities like document review, test case review, walk-throughs, inspection etc.	It involves activities like functional testing, automation testing etc.

Difference Between System and Integration Testing

System testing is a level of testing where the application as a whole is tested for its compliance to functional and non-functional requirements.

Whereas, in integration testing, we test the interfacing between the modules which are interconnected with each other.

Let’s now see the difference between the two.

System Testing	Integration Testing
In system testing, we check the system as a whole.	In integration testing, we check the interfacing between the inter-connected components.
It is performed after integration testing.	It is performed after unit testing.
It is carried out for performing both functional and non-functional testing(performance, usability, etc).	It is generally limited to functional aspects of the integrated components.
Since the testing is limited to the evaluation of functional requirements, hence, it includes black-box testing techniques only.	Since the interfacing logic is required to perform this testing, hence, it requires white/grey box testing techniques along with black-box techniques.
The different types of system testing are- Functional testing, Performance testing, Usability testing, Reliability testing, Security testing, Scalability testing, Installation testing, etc.	The different approaches of performing integration testing namely – Top-down, bottom-up, big-bang and hybrid integration.

Check the infographic below for a quick recap.

Differences between Black Box Testing vs White Box Testing

• 
  

Software Testing can be majorly classified into two categories: 
 

1. Black Box Testing is a software testing method in which the internal structure/ design/ implementation of the item being tested is not known to the tester 
    
2. White Box Testing is a software testing method in which the internal structure/ design/ implementation of the item being tested is known to the tester. 
    

Differences between Black Box Testing vs White Box Testing: 

 

Black Box Testing	White Box Testing
It is a way of software testing in which the internal structure or the program or the code is hidden and nothing is known about it.	It is a way of testing the software in which the tester has knowledge about the internal structure or the code or the program of the software.
It is mostly done by software testers.	It is mostly done by software developers.
No knowledge of implementation is needed.	Knowledge of implementation is required.
It can be referred as outer or external software testing.	It is the inner or the internal software testing.
It is functional test of the software.	It is structural test of the software.
This testing can be initiated on the basis of requirement specifications document.	This type of testing of software is started after detail design document.
No knowledge of programming is required.	It is mandatory to have knowledge of programming.
It is the behavior testing of the software.	It is the logic testing of the software.
It is applicable to the higher levels of testing of software.	It is generally applicable to the lower levels of software testing.
It is also called closed testing.	It is also called as clear box testing.
It is least time consuming.	It is most time consuming.
It is not suitable or preferred for algorithm testing.	It is suitable for algorithm testing.
Can be done by trial and error ways and methods.	Data domains along with inner or internal boundaries can be better tested.
Example: search something on google by using keywords	Example: by input to check and verify loops
Types of Black Box Testing:    A. Functional Testing    B. Non-functional testing    C. Regression Testing	Types of White Box Testing:    A. Path Testing    B. Loop Testing    C. Condition testing

 

Difference Between Smoke And Sanity Testing

Most of the times we get confused between the meaning of Sanity Testing and Smoke Testing. First of all, these two testings are way “different” and performed during different stages of a testing cycle.

S. No.	Smoke Testing	Sanity Testing
1	Smoke testing means to verify (basic) that the implementations done in a build are working fine.	Sanity testing means to verify the newly added functionalities, bugs etc. are working fine.
2	This is the first testing on the initial build.	Done when the build is relatively stable.
3	Done on every build.	Done on stable builds post regression.

Following is a diagrammatic representation of their differences:

SMOKE TESTING

• This testing originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch fire and smoke. In the software industry, this testing is a shallow and wide approach whereby all the areas of the application without getting into too deep, is tested.
• A smoke test is scripted, either using a written set of tests or an automated test
• A Smoke test is designed to touch every part of the application in a cursory way. It’s shallow and wide.
• This testing is conducted to ensure whether the most crucial functions of a program are working, but not bothering with the finer details. (Such as build verification).
• This testing is a normal health check-up to the build of an application before taking it to test in-depth.

SANITY TESTING

• A sanity test is a narrow regression test that focuses on one or a few areas of functionality. Sanity Testing is usually narrow and deep.
• This test is usually unscripted.
• This test is used to determine that a small section of the application is still working after a minor change.
• This testing is cursory testing, it is performed whenever a cursory testing is sufficient to prove that the application is functioning according to specifications. This level of testing is a subset of regression testing.
• This is to verify whether the requirements are met or not, checking all the features breadth-first.

Differences between Manual and Automation Testing

• 
  
• 
  
• Manual Testing:

Manual testing is a type of testing in which we do not take the help of any tools (automation) to perform the testing. In this testing, testers make test cases for the codes and test the software and give the final report about that software. Manual testing is a time consuming testing because it is done by humans and there is a chance of human errors.

Automation Testing:
Automation testing is a type of testing in which we take the help of tools (automation) to perform the testing. It is faster than manual testing because it is done by some automation tools. There is no chance of any human errors.

Manual Testing	Automation Testing
For observation manual testing allow human factor.	For observation automation testing does not allow human factor.
Manual testing checks the user-friendliness.	Automation testing can not give the guarantee of user-friendliness.
In manual testing investment is required for humans resources because here humans test the software instead of any software tools.	In automation testing investment is required for automation tools because here testing is done by some software tools.
Manual testing is time consuming because here process is done by humans or manual.	Automation testing is not time consuming because here process is done by some software tools.
In manual testing, there is a chance of human errors because here testing is done by humans.	In automation testing, there is no chance of human errors because here testing is done by tools.
In manual testing, there is a possibility of Exploratory testing.	In automation testing, there is no permission of random testing.
In manual testing, there is no need of programming knowledge.	In automation testing, there is must a need of programming knowledge.
Manual testing doesn’t use frameworks.	Automation testing uses frameworks like Data Drive,